[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:00:14.823009   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:46723
I20260709 13:00:14.824620   431 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:00:14.825786   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:14.845166   437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:14.845608   438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:14.848760   431 server_base.cc:1061] running on GCE node
W20260709 13:00:14.847478   440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:16.090720   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:16.091122   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:16.091382   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602016091353 us; error 0 us; skew 500 ppm
I20260709 13:00:16.092362   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:16.099978   431 webserver.cc:533] Webserver started at http://127.0.107.254:45561/ using document root <none> and password file <none>
I20260709 13:00:16.101016   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:16.101292   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:16.101841   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:16.108778   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/master-0-root/instance:
uuid: "af712a29c564424d9045cda87c4aacb7"
format_stamp: "Formatted at 2026-07-09 13:00:16 on dist-test-slave-cq4t"
I20260709 13:00:16.118269   431 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.010s	sys 0.000s
I20260709 13:00:16.124063   446 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:16.125562   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:00:16.126061   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/master-0-root
uuid: "af712a29c564424d9045cda87c4aacb7"
format_stamp: "Formatted at 2026-07-09 13:00:16 on dist-test-slave-cq4t"
I20260709 13:00:16.126461   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:16.192663   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:16.194772   431 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:00:16.195349   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:16.284788   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:46723
I20260709 13:00:16.284881   497 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:46723 every 8 connection(s)
I20260709 13:00:16.294808   498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:16.321727   498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7: Bootstrap starting.
I20260709 13:00:16.328981   498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:16.331055   498 log.cc:826] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7: Log is configured to *not* fsync() on all Append() calls
I20260709 13:00:16.336428   498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7: No bootstrap required, opened a new log
I20260709 13:00:16.358289   498 raft_consensus.cc:359] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af712a29c564424d9045cda87c4aacb7" member_type: VOTER }
I20260709 13:00:16.358862   498 raft_consensus.cc:385] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:16.359234   498 raft_consensus.cc:740] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af712a29c564424d9045cda87c4aacb7, State: Initialized, Role: FOLLOWER
I20260709 13:00:16.360257   498 consensus_queue.cc:260] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af712a29c564424d9045cda87c4aacb7" member_type: VOTER }
I20260709 13:00:16.360805   498 raft_consensus.cc:399] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:00:16.361122   498 raft_consensus.cc:493] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:00:16.361508   498 raft_consensus.cc:3060] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:16.366393   498 raft_consensus.cc:515] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af712a29c564424d9045cda87c4aacb7" member_type: VOTER }
I20260709 13:00:16.367347   498 leader_election.cc:304] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af712a29c564424d9045cda87c4aacb7; no voters: 
I20260709 13:00:16.369128   498 leader_election.cc:290] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:00:16.369570   501 raft_consensus.cc:2804] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:16.373232   501 raft_consensus.cc:697] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [term 1 LEADER]: Becoming Leader. State: Replica: af712a29c564424d9045cda87c4aacb7, State: Running, Role: LEADER
I20260709 13:00:16.374280   501 consensus_queue.cc:237] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af712a29c564424d9045cda87c4aacb7" member_type: VOTER }
I20260709 13:00:16.375322   498 sys_catalog.cc:565] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:00:16.404799   503 sys_catalog.cc:455] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af712a29c564424d9045cda87c4aacb7. Latest consensus state: current_term: 1 leader_uuid: "af712a29c564424d9045cda87c4aacb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af712a29c564424d9045cda87c4aacb7" member_type: VOTER } }
I20260709 13:00:16.406037   503 sys_catalog.cc:458] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:16.412194   509 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:00:16.427970   509 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:00:16.460821   502 sys_catalog.cc:455] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af712a29c564424d9045cda87c4aacb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af712a29c564424d9045cda87c4aacb7" member_type: VOTER } }
I20260709 13:00:16.461326   502 sys_catalog.cc:458] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:16.479595   509 catalog_manager.cc:1383] Generated new cluster ID: 5615de9b99064214bd0206501a15d5bb
I20260709 13:00:16.480057   509 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:00:16.484002   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:00:16.503123   509 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:00:16.504789   509 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:00:16.519279   509 catalog_manager.cc:6070] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7: Generated new TSK 0
I20260709 13:00:16.520412   509 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:00:16.550767   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:16.560985   520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:16.563819   521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:16.566583   523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:16.568053   431 server_base.cc:1061] running on GCE node
I20260709 13:00:16.569442   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:16.569723   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:16.569936   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602016569918 us; error 0 us; skew 500 ppm
I20260709 13:00:16.570751   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:16.575069   431 webserver.cc:533] Webserver started at http://127.0.107.193:44457/ using document root <none> and password file <none>
I20260709 13:00:16.575802   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:16.576068   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:16.576452   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:16.578552   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-0-root/instance:
uuid: "49c3856ee800410f95f6b9695bd926d6"
format_stamp: "Formatted at 2026-07-09 13:00:16 on dist-test-slave-cq4t"
I20260709 13:00:16.586007   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:00:16.592238   528 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:16.593214   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:00:16.593600   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-0-root
uuid: "49c3856ee800410f95f6b9695bd926d6"
format_stamp: "Formatted at 2026-07-09 13:00:16 on dist-test-slave-cq4t"
I20260709 13:00:16.594007   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:16.640033   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:16.641955   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:16.644201   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:16.649742   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:16.650086   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:16.650385   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:16.650581   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:16.765797   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:44271
I20260709 13:00:16.765873   590 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:44271 every 8 connection(s)
I20260709 13:00:16.773908   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:16.792114   596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:16.793176   595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:16.799222   598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:16.805160   431 server_base.cc:1061] running on GCE node
I20260709 13:00:16.806684   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:16.806926   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:16.807183   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602016807162 us; error 0 us; skew 500 ppm
I20260709 13:00:16.807847   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:16.811031   431 webserver.cc:533] Webserver started at http://127.0.107.194:45265/ using document root <none> and password file <none>
I20260709 13:00:16.811559   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:16.811724   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:16.812103   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:16.813730   591 heartbeater.cc:344] Connected to a master server at 127.0.107.254:46723
I20260709 13:00:16.813760   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-1-root/instance:
uuid: "75b161200bff4cc8a1789a7247d37fea"
format_stamp: "Formatted at 2026-07-09 13:00:16 on dist-test-slave-cq4t"
I20260709 13:00:16.814332   591 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:16.815598   591 heartbeater.cc:507] Master 127.0.107.254:46723 requested a full tablet report, sending...
I20260709 13:00:16.819439   463 ts_manager.cc:194] Registered new tserver with Master: 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271)
I20260709 13:00:16.821038   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.000s	sys 0.006s
I20260709 13:00:16.822762   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42236
I20260709 13:00:16.827208   603 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:16.828277   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:00:16.828641   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-1-root
uuid: "75b161200bff4cc8a1789a7247d37fea"
format_stamp: "Formatted at 2026-07-09 13:00:16 on dist-test-slave-cq4t"
I20260709 13:00:16.829011   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:16.844524   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:16.846288   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:16.849460   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:16.853400   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:16.853695   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:16.853981   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:16.854194   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:16.971107   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:40969
I20260709 13:00:16.971246   665 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:40969 every 8 connection(s)
I20260709 13:00:16.979813   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:16.988195   669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:16.991937   670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:17.001152   666 heartbeater.cc:344] Connected to a master server at 127.0.107.254:46723
I20260709 13:00:17.001631   666 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:17.002869   666 heartbeater.cc:507] Master 127.0.107.254:46723 requested a full tablet report, sending...
W20260709 13:00:17.003934   672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:17.004101   431 server_base.cc:1061] running on GCE node
I20260709 13:00:17.005669   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:17.006022   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:17.006031   463 ts_manager.cc:194] Registered new tserver with Master: 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:17.006413   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602017006391 us; error 0 us; skew 500 ppm
I20260709 13:00:17.007306   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:17.008242   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42244
I20260709 13:00:17.011842   431 webserver.cc:533] Webserver started at http://127.0.107.195:42021/ using document root <none> and password file <none>
I20260709 13:00:17.012420   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:17.012674   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:17.012971   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:17.014165   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-2-root/instance:
uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
format_stamp: "Formatted at 2026-07-09 13:00:17 on dist-test-slave-cq4t"
I20260709 13:00:17.020542   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.003s
I20260709 13:00:17.024734   677 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.025781   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:00:17.026124   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-2-root
uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
format_stamp: "Formatted at 2026-07-09 13:00:17 on dist-test-slave-cq4t"
I20260709 13:00:17.026439   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602014632082-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:17.056093   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:17.058343   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:17.060470   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:17.063393   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:17.063621   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.063967   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:17.064163   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:17.189522   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:40989
I20260709 13:00:17.189673   739 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:40989 every 8 connection(s)
I20260709 13:00:17.212182   740 heartbeater.cc:344] Connected to a master server at 127.0.107.254:46723
I20260709 13:00:17.212634   740 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:17.213666   740 heartbeater.cc:507] Master 127.0.107.254:46723 requested a full tablet report, sending...
I20260709 13:00:17.215869   463 ts_manager.cc:194] Registered new tserver with Master: e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:17.216010   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019632462s
I20260709 13:00:17.217482   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42246
I20260709 13:00:17.274828   463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42252:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:00:17.279805   463 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260709 13:00:17.419240   511 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:17.427048   630 tablet_service.cc:1511] Processing CreateTablet for tablet f32ed3eafdb743e0b5527a057250e80d (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:00:17.432183   627 tablet_service.cc:1511] Processing CreateTablet for tablet 175f13d32fad40c8a7cb768f55aca725 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
W20260709 13:00:17.439432   494 debug-util.cc:398] Leaking SignalData structure 0x7b0800062060 after lost signal to thread 434
I20260709 13:00:17.445530   629 tablet_service.cc:1511] Processing CreateTablet for tablet 2b999efdb1de432fa180c502b4b2e0e2 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:00:17.444332   630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f32ed3eafdb743e0b5527a057250e80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.443192   631 tablet_service.cc:1511] Processing CreateTablet for tablet ec5a7dee7d3b4b2cb847171ad1e80e98 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:00:17.447257   628 tablet_service.cc:1511] Processing CreateTablet for tablet a80f43d3599a4018921d0f12ea30d25f (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:00:17.447595   625 tablet_service.cc:1511] Processing CreateTablet for tablet acad783b77864a29a0bfcfdd201e12f6 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:00:17.453742   627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 175f13d32fad40c8a7cb768f55aca725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.444350   624 tablet_service.cc:1511] Processing CreateTablet for tablet 2865aa77674948499e8d5e89dd117de7 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:00:17.447759   626 tablet_service.cc:1511] Processing CreateTablet for tablet 725a4a0982b1482eacea62cdebb7cb3e (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:00:17.462136   629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b999efdb1de432fa180c502b4b2e0e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.464557   705 tablet_service.cc:1511] Processing CreateTablet for tablet ec5a7dee7d3b4b2cb847171ad1e80e98 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:00:17.462121   704 tablet_service.cc:1511] Processing CreateTablet for tablet f32ed3eafdb743e0b5527a057250e80d (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:00:17.471711   631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec5a7dee7d3b4b2cb847171ad1e80e98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.470324   701 tablet_service.cc:1511] Processing CreateTablet for tablet 175f13d32fad40c8a7cb768f55aca725 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:00:17.471611   703 tablet_service.cc:1511] Processing CreateTablet for tablet 2b999efdb1de432fa180c502b4b2e0e2 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:00:17.475798   623 tablet_service.cc:1511] Processing CreateTablet for tablet 9f1df71364ad4b3dbdd84404e9bf9b44 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:00:17.477087   625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acad783b77864a29a0bfcfdd201e12f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.479879   626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 725a4a0982b1482eacea62cdebb7cb3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.474390   702 tablet_service.cc:1511] Processing CreateTablet for tablet a80f43d3599a4018921d0f12ea30d25f (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:00:17.482471   704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f32ed3eafdb743e0b5527a057250e80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.481495   624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2865aa77674948499e8d5e89dd117de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.485502   699 tablet_service.cc:1511] Processing CreateTablet for tablet acad783b77864a29a0bfcfdd201e12f6 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:00:17.480314   700 tablet_service.cc:1511] Processing CreateTablet for tablet 725a4a0982b1482eacea62cdebb7cb3e (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:00:17.493177   703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b999efdb1de432fa180c502b4b2e0e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.492765   698 tablet_service.cc:1511] Processing CreateTablet for tablet 2865aa77674948499e8d5e89dd117de7 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:00:17.496439   697 tablet_service.cc:1511] Processing CreateTablet for tablet 9f1df71364ad4b3dbdd84404e9bf9b44 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:00:17.498818   700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 725a4a0982b1482eacea62cdebb7cb3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.499881   699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acad783b77864a29a0bfcfdd201e12f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.500587   702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80f43d3599a4018921d0f12ea30d25f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.501116   628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80f43d3599a4018921d0f12ea30d25f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.504583   698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2865aa77674948499e8d5e89dd117de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.505347   705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec5a7dee7d3b4b2cb847171ad1e80e98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.507804   623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f1df71364ad4b3dbdd84404e9bf9b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.510035   701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 175f13d32fad40c8a7cb768f55aca725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.516418   556 tablet_service.cc:1511] Processing CreateTablet for tablet ec5a7dee7d3b4b2cb847171ad1e80e98 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:00:17.526660   697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f1df71364ad4b3dbdd84404e9bf9b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.522133   554 tablet_service.cc:1511] Processing CreateTablet for tablet 2b999efdb1de432fa180c502b4b2e0e2 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:00:17.540436   550 tablet_service.cc:1511] Processing CreateTablet for tablet acad783b77864a29a0bfcfdd201e12f6 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:00:17.558737   552 tablet_service.cc:1511] Processing CreateTablet for tablet 175f13d32fad40c8a7cb768f55aca725 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:00:17.561301   552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 175f13d32fad40c8a7cb768f55aca725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.565900   553 tablet_service.cc:1511] Processing CreateTablet for tablet a80f43d3599a4018921d0f12ea30d25f (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:00:17.571992   551 tablet_service.cc:1511] Processing CreateTablet for tablet 725a4a0982b1482eacea62cdebb7cb3e (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:00:17.579782   549 tablet_service.cc:1511] Processing CreateTablet for tablet 2865aa77674948499e8d5e89dd117de7 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:00:17.581883   556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec5a7dee7d3b4b2cb847171ad1e80e98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.575031   555 tablet_service.cc:1511] Processing CreateTablet for tablet f32ed3eafdb743e0b5527a057250e80d (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:00:17.588477   548 tablet_service.cc:1511] Processing CreateTablet for tablet 9f1df71364ad4b3dbdd84404e9bf9b44 (DEFAULT_TABLE table=test-workload [id=f3d9771b49d54ba381bf9f849c0add3f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:00:17.596676   553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80f43d3599a4018921d0f12ea30d25f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.606442   551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 725a4a0982b1482eacea62cdebb7cb3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.628432   555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f32ed3eafdb743e0b5527a057250e80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.648388   554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b999efdb1de432fa180c502b4b2e0e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.667426   549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2865aa77674948499e8d5e89dd117de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.675364   548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f1df71364ad4b3dbdd84404e9bf9b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.687561   550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acad783b77864a29a0bfcfdd201e12f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:17.827196   758 tablet_bootstrap.cc:492] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea: Bootstrap starting.
I20260709 13:00:17.827651   591 heartbeater.cc:499] Master 127.0.107.254:46723 was elected leader, sending a full tablet report...
I20260709 13:00:17.976548   760 tablet_bootstrap.cc:492] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759: Bootstrap starting.
I20260709 13:00:17.994012   759 tablet_bootstrap.cc:492] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6: Bootstrap starting.
I20260709 13:00:17.999212   760 tablet_bootstrap.cc:654] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.001391   758 tablet_bootstrap.cc:654] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.012622   666 heartbeater.cc:499] Master 127.0.107.254:46723 was elected leader, sending a full tablet report...
I20260709 13:00:18.033263   759 tablet_bootstrap.cc:654] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.036541   760 tablet_bootstrap.cc:492] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759: No bootstrap required, opened a new log
I20260709 13:00:18.037114   760 ts_tablet_manager.cc:1403] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759: Time spent bootstrapping tablet: real 0.105s	user 0.017s	sys 0.013s
I20260709 13:00:18.040923   760 raft_consensus.cc:359] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.041934   760 raft_consensus.cc:385] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.042276   760 raft_consensus.cc:740] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.043293   760 consensus_queue.cc:260] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.045217   758 tablet_bootstrap.cc:492] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea: No bootstrap required, opened a new log
I20260709 13:00:18.045872   758 ts_tablet_manager.cc:1403] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea: Time spent bootstrapping tablet: real 0.219s	user 0.018s	sys 0.025s
I20260709 13:00:18.050966   758 raft_consensus.cc:359] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.052135   758 raft_consensus.cc:385] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.052580   758 raft_consensus.cc:740] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.053642   758 consensus_queue.cc:260] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.057485   740 heartbeater.cc:499] Master 127.0.107.254:46723 was elected leader, sending a full tablet report...
I20260709 13:00:18.058297   760 ts_tablet_manager.cc:1434] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759: Time spent starting tablet: real 0.021s	user 0.012s	sys 0.009s
I20260709 13:00:18.060504   760 tablet_bootstrap.cc:492] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759: Bootstrap starting.
I20260709 13:00:18.078614   759 tablet_bootstrap.cc:492] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6: No bootstrap required, opened a new log
I20260709 13:00:18.079299   759 ts_tablet_manager.cc:1403] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6: Time spent bootstrapping tablet: real 0.086s	user 0.007s	sys 0.023s
I20260709 13:00:18.105628   760 tablet_bootstrap.cc:654] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.107867   758 ts_tablet_manager.cc:1434] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea: Time spent starting tablet: real 0.062s	user 0.024s	sys 0.014s
I20260709 13:00:18.118077   759 raft_consensus.cc:359] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.120079   759 raft_consensus.cc:385] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.120435   759 raft_consensus.cc:740] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.121265   759 consensus_queue.cc:260] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.119675   758 tablet_bootstrap.cc:492] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea: Bootstrap starting.
I20260709 13:00:18.141566   758 tablet_bootstrap.cc:654] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.145296   760 tablet_bootstrap.cc:492] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759: No bootstrap required, opened a new log
I20260709 13:00:18.145915   760 ts_tablet_manager.cc:1403] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759: Time spent bootstrapping tablet: real 0.086s	user 0.009s	sys 0.003s
I20260709 13:00:18.149801   760 raft_consensus.cc:359] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.160363   759 ts_tablet_manager.cc:1434] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6: Time spent starting tablet: real 0.081s	user 0.002s	sys 0.011s
I20260709 13:00:18.161978   759 tablet_bootstrap.cc:492] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6: Bootstrap starting.
I20260709 13:00:18.150743   760 raft_consensus.cc:385] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.166714   760 raft_consensus.cc:740] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.167970   760 consensus_queue.cc:260] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.171597   759 tablet_bootstrap.cc:654] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.174324   760 ts_tablet_manager.cc:1434] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759: Time spent starting tablet: real 0.028s	user 0.010s	sys 0.000s
I20260709 13:00:18.175750   760 tablet_bootstrap.cc:492] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759: Bootstrap starting.
I20260709 13:00:18.188768   760 tablet_bootstrap.cc:654] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.200073   758 tablet_bootstrap.cc:492] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea: No bootstrap required, opened a new log
W20260709 13:00:18.200049   741 tablet.cc:2406] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:00:18.200749   758 ts_tablet_manager.cc:1403] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea: Time spent bootstrapping tablet: real 0.081s	user 0.013s	sys 0.006s
I20260709 13:00:18.205953   758 raft_consensus.cc:359] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.206847   758 raft_consensus.cc:385] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.208844   758 raft_consensus.cc:740] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.209681   758 consensus_queue.cc:260] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.211887   758 ts_tablet_manager.cc:1434] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:00:18.213593   758 tablet_bootstrap.cc:492] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea: Bootstrap starting.
I20260709 13:00:18.217685   760 tablet_bootstrap.cc:492] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759: No bootstrap required, opened a new log
I20260709 13:00:18.218227   760 ts_tablet_manager.cc:1403] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.014s
I20260709 13:00:18.221742   760 raft_consensus.cc:359] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.222535   760 raft_consensus.cc:385] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.225021   759 tablet_bootstrap.cc:492] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6: No bootstrap required, opened a new log
I20260709 13:00:18.225521   759 ts_tablet_manager.cc:1403] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6: Time spent bootstrapping tablet: real 0.064s	user 0.020s	sys 0.000s
I20260709 13:00:18.222818   760 raft_consensus.cc:740] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.228253   760 consensus_queue.cc:260] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.229218   759 raft_consensus.cc:359] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.229965   759 raft_consensus.cc:385] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.230695   759 raft_consensus.cc:740] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.231693   759 consensus_queue.cc:260] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.232571   758 tablet_bootstrap.cc:654] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.233901   759 ts_tablet_manager.cc:1434] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:00:18.236074   760 ts_tablet_manager.cc:1434] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
I20260709 13:00:18.238880   760 tablet_bootstrap.cc:492] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759: Bootstrap starting.
I20260709 13:00:18.246670   760 tablet_bootstrap.cc:654] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.237900   759 tablet_bootstrap.cc:492] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6: Bootstrap starting.
I20260709 13:00:18.253140   759 tablet_bootstrap.cc:654] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.258505   758 tablet_bootstrap.cc:492] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea: No bootstrap required, opened a new log
I20260709 13:00:18.259096   758 ts_tablet_manager.cc:1403] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.006s
I20260709 13:00:18.262868   758 raft_consensus.cc:359] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.267705   758 raft_consensus.cc:385] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.268108   758 raft_consensus.cc:740] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.268908   758 consensus_queue.cc:260] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.270035   760 tablet_bootstrap.cc:492] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759: No bootstrap required, opened a new log
I20260709 13:00:18.270509   760 ts_tablet_manager.cc:1403] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.008s
I20260709 13:00:18.271716   758 ts_tablet_manager.cc:1434] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.006s
I20260709 13:00:18.272689   758 tablet_bootstrap.cc:492] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea: Bootstrap starting.
I20260709 13:00:18.273908   760 raft_consensus.cc:359] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.274603   760 raft_consensus.cc:385] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.274942   760 raft_consensus.cc:740] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.275709   760 consensus_queue.cc:260] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.280617   764 raft_consensus.cc:493] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.281148   764 raft_consensus.cc:515] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.282105   758 tablet_bootstrap.cc:654] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.292471   760 ts_tablet_manager.cc:1434] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 13:00:18.292392   765 raft_consensus.cc:493] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.293778   760 tablet_bootstrap.cc:492] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759: Bootstrap starting.
I20260709 13:00:18.293200   765 raft_consensus.cc:515] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.311115   765 leader_election.cc:290] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:18.311193   764 leader_election.cc:290] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:18.313148   764 raft_consensus.cc:493] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.324200   760 tablet_bootstrap.cc:654] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.323855   764 raft_consensus.cc:515] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.360553   778 raft_consensus.cc:493] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.361330   778 raft_consensus.cc:515] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.363766   778 leader_election.cc:290] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:18.368512   758 tablet_bootstrap.cc:492] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea: No bootstrap required, opened a new log
I20260709 13:00:18.371639   765 raft_consensus.cc:493] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.372223   765 raft_consensus.cc:515] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.369866   764 leader_election.cc:290] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:18.370587   758 ts_tablet_manager.cc:1403] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea: Time spent bootstrapping tablet: real 0.097s	user 0.027s	sys 0.049s
I20260709 13:00:18.376572   767 raft_consensus.cc:493] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.377418   759 tablet_bootstrap.cc:492] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6: No bootstrap required, opened a new log
I20260709 13:00:18.377189   765 leader_election.cc:290] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:18.377702   767 raft_consensus.cc:515] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.378391   759 ts_tablet_manager.cc:1403] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6: Time spent bootstrapping tablet: real 0.141s	user 0.024s	sys 0.015s
I20260709 13:00:18.383230   767 leader_election.cc:290] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:18.394259   759 raft_consensus.cc:359] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.395383   759 raft_consensus.cc:385] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.395805   759 raft_consensus.cc:740] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.397598   759 consensus_queue.cc:260] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.400938   759 ts_tablet_manager.cc:1434] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.001s
I20260709 13:00:18.401046   758 raft_consensus.cc:359] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.401942   758 raft_consensus.cc:385] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.402267   758 raft_consensus.cc:740] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.403028   758 consensus_queue.cc:260] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.405323   758 ts_tablet_manager.cc:1434] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 13:00:18.406834   758 tablet_bootstrap.cc:492] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea: Bootstrap starting.
I20260709 13:00:18.414233   759 tablet_bootstrap.cc:492] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6: Bootstrap starting.
W20260709 13:00:18.422011   511 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:18.452893   760 tablet_bootstrap.cc:492] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759: No bootstrap required, opened a new log
I20260709 13:00:18.453625   760 ts_tablet_manager.cc:1403] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759: Time spent bootstrapping tablet: real 0.160s	user 0.018s	sys 0.005s
I20260709 13:00:18.490191   760 raft_consensus.cc:359] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.494334   760 raft_consensus.cc:385] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.496918   767 raft_consensus.cc:493] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.498070   767 raft_consensus.cc:515] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.497453   760 raft_consensus.cc:740] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.507741   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f1df71364ad4b3dbdd84404e9bf9b44" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6" is_pre_election: true
I20260709 13:00:18.506407   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6" is_pre_election: true
I20260709 13:00:18.508214   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec5a7dee7d3b4b2cb847171ad1e80e98" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6" is_pre_election: true
I20260709 13:00:18.516253   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725a4a0982b1482eacea62cdebb7cb3e" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6" is_pre_election: true
I20260709 13:00:18.518496   767 leader_election.cc:290] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:18.516829   760 consensus_queue.cc:260] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.523450   563 raft_consensus.cc:2468] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 0.
I20260709 13:00:18.517383   566 raft_consensus.cc:2468] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 0.
W20260709 13:00:18.529043   606 leader_election.cc:343] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.520774   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6" is_pre_election: true
I20260709 13:00:18.531453   606 leader_election.cc:304] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea; no voters: 
I20260709 13:00:18.540678   765 raft_consensus.cc:2804] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:00:18.539875   680 leader_election.cc:343] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.544466   765 raft_consensus.cc:493] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.547451   765 raft_consensus.cc:3060] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.549664   680 leader_election.cc:304] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:18.551157   760 ts_tablet_manager.cc:1434] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759: Time spent starting tablet: real 0.097s	user 0.015s	sys 0.000s
I20260709 13:00:18.552891   764 raft_consensus.cc:2804] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:00:18.553674   680 leader_election.cc:343] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.554546   764 raft_consensus.cc:493] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.557818   760 tablet_bootstrap.cc:492] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759: Bootstrap starting.
I20260709 13:00:18.557749   764 raft_consensus.cc:3060] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.555984   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea" is_pre_election: true
I20260709 13:00:18.555226   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f1df71364ad4b3dbdd84404e9bf9b44" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea" is_pre_election: true
I20260709 13:00:18.561846   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725a4a0982b1482eacea62cdebb7cb3e" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea" is_pre_election: true
I20260709 13:00:18.564471   640 raft_consensus.cc:2468] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 0.
W20260709 13:00:18.569427   679 leader_election.cc:343] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:18.570559   679 leader_election.cc:343] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.571012   679 leader_election.cc:304] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4f3f8d22f334cfeb9dc7316c2771759; no voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea
I20260709 13:00:18.573356   758 tablet_bootstrap.cc:654] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.576174   777 raft_consensus.cc:2749] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:18.579149   759 tablet_bootstrap.cc:654] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.579977   679 leader_election.cc:304] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 49c3856ee800410f95f6b9695bd926d6
I20260709 13:00:18.585413   777 raft_consensus.cc:2804] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:18.587457   777 raft_consensus.cc:493] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.588047   777 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.596076   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acad783b77864a29a0bfcfdd201e12f6" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea" is_pre_election: true
I20260709 13:00:18.596192   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725a4a0982b1482eacea62cdebb7cb3e" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea" is_pre_election: true
I20260709 13:00:18.594436   765 raft_consensus.cc:515] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
W20260709 13:00:18.603058   530 leader_election.cc:343] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:18.606659   530 leader_election.cc:343] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.611207   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec5a7dee7d3b4b2cb847171ad1e80e98" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:18.612411   765 leader_election.cc:290] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:18.613538   563 raft_consensus.cc:3060] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.617795   758 tablet_bootstrap.cc:492] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea: No bootstrap required, opened a new log
I20260709 13:00:18.616078   764 raft_consensus.cc:515] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.618528   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec5a7dee7d3b4b2cb847171ad1e80e98" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" is_pre_election: true
I20260709 13:00:18.621112   758 ts_tablet_manager.cc:1403] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea: Time spent bootstrapping tablet: real 0.214s	user 0.003s	sys 0.024s
I20260709 13:00:18.618654   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" is_pre_election: true
I20260709 13:00:18.622090   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec5a7dee7d3b4b2cb847171ad1e80e98" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
W20260709 13:00:18.625865   605 leader_election.cc:343] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:18.627835   605 leader_election.cc:343] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.632293   764 leader_election.cc:290] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:18.633167   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725a4a0982b1482eacea62cdebb7cb3e" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:18.633868   566 raft_consensus.cc:3060] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.634436   563 raft_consensus.cc:2468] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 1.
I20260709 13:00:18.636083   606 leader_election.cc:304] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea; no voters: e4f3f8d22f334cfeb9dc7316c2771759
I20260709 13:00:18.640785   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725a4a0982b1482eacea62cdebb7cb3e" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:18.636269   758 raft_consensus.cc:359] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.641353   765 raft_consensus.cc:2804] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:18.642795   758 raft_consensus.cc:385] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:00:18.645638   679 leader_election.cc:343] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.645392   758 raft_consensus.cc:740] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.649515   765 raft_consensus.cc:697] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Becoming Leader. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:18.645638   777 raft_consensus.cc:515] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.651199   760 tablet_bootstrap.cc:654] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.652729   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acad783b77864a29a0bfcfdd201e12f6" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" is_pre_election: true
I20260709 13:00:18.653841   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725a4a0982b1482eacea62cdebb7cb3e" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" is_pre_election: true
I20260709 13:00:18.654241   759 tablet_bootstrap.cc:492] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6: No bootstrap required, opened a new log
I20260709 13:00:18.655207   758 consensus_queue.cc:260] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.659811   715 raft_consensus.cc:2393] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49c3856ee800410f95f6b9695bd926d6 in current term 1: Already voted for candidate e4f3f8d22f334cfeb9dc7316c2771759 in this term.
I20260709 13:00:18.662196   714 raft_consensus.cc:2393] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75b161200bff4cc8a1789a7247d37fea in current term 1: Already voted for candidate e4f3f8d22f334cfeb9dc7316c2771759 in this term.
I20260709 13:00:18.662947   759 ts_tablet_manager.cc:1403] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6: Time spent bootstrapping tablet: real 0.251s	user 0.014s	sys 0.009s
I20260709 13:00:18.661005   713 raft_consensus.cc:2468] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 0.
I20260709 13:00:18.667851   530 leader_election.cc:304] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6; no voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759
I20260709 13:00:18.667680   765 consensus_queue.cc:237] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.674510   758 ts_tablet_manager.cc:1434] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.005s
I20260709 13:00:18.671049   792 raft_consensus.cc:493] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.675246   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:18.677290   530 leader_election.cc:304] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 75b161200bff4cc8a1789a7247d37fea
I20260709 13:00:18.678349   777 leader_election.cc:290] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:18.679570   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:18.683486   640 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:18.680363   680 leader_election.cc:343] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.688905   566 raft_consensus.cc:2468] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 1.
I20260709 13:00:18.689960   767 raft_consensus.cc:2749] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:18.687250   759 raft_consensus.cc:359] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.693225   680 leader_election.cc:304] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 75b161200bff4cc8a1789a7247d37fea
I20260709 13:00:18.695503   759 raft_consensus.cc:385] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.702273   777 raft_consensus.cc:2804] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:18.702503   759 raft_consensus.cc:740] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.702836   777 raft_consensus.cc:697] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Becoming Leader. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:18.681532   792 raft_consensus.cc:515] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.709728   759 consensus_queue.cc:260] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.721817   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" is_pre_election: true
I20260709 13:00:18.717593   792 leader_election.cc:290] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:18.715545   777 consensus_queue.cc:237] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.725256   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6" is_pre_election: true
I20260709 13:00:18.728194   795 raft_consensus.cc:2804] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:00:18.730294   606 leader_election.cc:343] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.739420   760 tablet_bootstrap.cc:492] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759: No bootstrap required, opened a new log
I20260709 13:00:18.735337   795 raft_consensus.cc:493] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.746253   640 raft_consensus.cc:2468] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 1.
I20260709 13:00:18.750947   713 raft_consensus.cc:2468] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 0.
I20260709 13:00:18.753713   760 ts_tablet_manager.cc:1403] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759: Time spent bootstrapping tablet: real 0.197s	user 0.033s	sys 0.022s
I20260709 13:00:18.753321   795 raft_consensus.cc:3060] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.756505   679 leader_election.cc:304] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 49c3856ee800410f95f6b9695bd926d6
I20260709 13:00:18.758052   759 ts_tablet_manager.cc:1434] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6: Time spent starting tablet: real 0.092s	user 0.019s	sys 0.000s
I20260709 13:00:18.757109   605 leader_election.cc:304] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 49c3856ee800410f95f6b9695bd926d6
I20260709 13:00:18.758850   777 raft_consensus.cc:2804] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:18.761802   792 raft_consensus.cc:2804] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:18.761582   777 raft_consensus.cc:697] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Becoming Leader. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:18.763103   759 tablet_bootstrap.cc:492] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6: Bootstrap starting.
I20260709 13:00:18.763296   792 raft_consensus.cc:493] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.764921   792 raft_consensus.cc:3060] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.767323   777 consensus_queue.cc:237] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.772262   760 raft_consensus.cc:359] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.773242   760 raft_consensus.cc:385] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.773579   760 raft_consensus.cc:740] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.776118   760 consensus_queue.cc:260] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.785522   758 tablet_bootstrap.cc:492] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea: Bootstrap starting.
I20260709 13:00:18.792124   792 raft_consensus.cc:515] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.796721   605 leader_election.cc:304] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea; no voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759
I20260709 13:00:18.797787   792 leader_election.cc:290] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:18.797899   765 raft_consensus.cc:2749] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:18.803462   760 ts_tablet_manager.cc:1434] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.007s
I20260709 13:00:18.803668   765 raft_consensus.cc:493] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.788426   461 catalog_manager.cc:5697] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 75b161200bff4cc8a1789a7247d37fea (127.0.107.194). New cstate: current_term: 1 leader_uuid: "75b161200bff4cc8a1789a7247d37fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:18.804229   765 raft_consensus.cc:515] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.804785   760 tablet_bootstrap.cc:492] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759: Bootstrap starting.
I20260709 13:00:18.807121   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:18.807751   765 leader_election.cc:290] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
W20260709 13:00:18.811419   606 leader_election.cc:343] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.812448   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6" is_pre_election: true
I20260709 13:00:18.813284   562 raft_consensus.cc:2468] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 0.
I20260709 13:00:18.816258   606 leader_election.cc:304] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea; no voters: 
I20260709 13:00:18.817095   765 raft_consensus.cc:2804] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:18.817492   765 raft_consensus.cc:493] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.817840   765 raft_consensus.cc:3060] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.817795   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:18.818497   714 raft_consensus.cc:3060] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.821265   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" is_pre_election: true
W20260709 13:00:18.822688   605 leader_election.cc:343] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.828266   765 raft_consensus.cc:515] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.823350   795 raft_consensus.cc:515] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.836426   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:18.833973   460 catalog_manager.cc:5697] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:18.837978   562 raft_consensus.cc:3060] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.840785   792 raft_consensus.cc:493] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.840371   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:18.843498   792 raft_consensus.cc:515] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
W20260709 13:00:18.845798   605 leader_election.cc:343] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:18.848097   792 leader_election.cc:290] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:18.849022   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acad783b77864a29a0bfcfdd201e12f6" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6" is_pre_election: true
I20260709 13:00:18.839460   765 leader_election.cc:290] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:18.847805   714 raft_consensus.cc:2468] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 1.
I20260709 13:00:18.850317   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acad783b77864a29a0bfcfdd201e12f6" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" is_pre_election: true
I20260709 13:00:18.850710   566 raft_consensus.cc:2393] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75b161200bff4cc8a1789a7247d37fea in current term 1: Already voted for candidate 49c3856ee800410f95f6b9695bd926d6 in this term.
I20260709 13:00:18.853510   562 raft_consensus.cc:2468] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 1.
I20260709 13:00:18.853348   713 raft_consensus.cc:2468] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 0.
I20260709 13:00:18.854092   758 tablet_bootstrap.cc:654] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.854159   605 leader_election.cc:304] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 49c3856ee800410f95f6b9695bd926d6
I20260709 13:00:18.838465   460 catalog_manager.cc:5697] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:18.854739   606 leader_election.cc:304] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea; no voters: e4f3f8d22f334cfeb9dc7316c2771759
I20260709 13:00:18.840746   759 tablet_bootstrap.cc:654] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.855641   792 raft_consensus.cc:2804] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:18.855906   605 leader_election.cc:304] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 49c3856ee800410f95f6b9695bd926d6
I20260709 13:00:18.856365   792 raft_consensus.cc:697] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Becoming Leader. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:18.856614   793 raft_consensus.cc:2804] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:18.857419   760 tablet_bootstrap.cc:654] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:18.857506   792 consensus_queue.cc:237] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.861372   765 raft_consensus.cc:2804] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:18.871977   765 raft_consensus.cc:493] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.873359   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acad783b77864a29a0bfcfdd201e12f6" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:18.871474   793 raft_consensus.cc:697] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Becoming Leader. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:18.874234   765 raft_consensus.cc:3060] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.875344   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acad783b77864a29a0bfcfdd201e12f6" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:18.874100   795 leader_election.cc:290] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 election: Requested vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:18.875015   793 consensus_queue.cc:237] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.875911   714 raft_consensus.cc:3060] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:18.918582   765 raft_consensus.cc:515] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.920815   714 raft_consensus.cc:2468] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 1.
I20260709 13:00:18.924005   640 raft_consensus.cc:2393] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 49c3856ee800410f95f6b9695bd926d6 in current term 1: Already voted for candidate 75b161200bff4cc8a1789a7247d37fea in this term.
I20260709 13:00:18.925755   530 leader_election.cc:304] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:18.925793   765 leader_election.cc:290] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:18.926653   795 raft_consensus.cc:2804] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:18.927299   795 raft_consensus.cc:697] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 1 LEADER]: Becoming Leader. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:18.928390   795 consensus_queue.cc:237] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.928810   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acad783b77864a29a0bfcfdd201e12f6" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:18.932917   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acad783b77864a29a0bfcfdd201e12f6" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:18.935236   714 raft_consensus.cc:2393] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 75b161200bff4cc8a1789a7247d37fea in current term 1: Already voted for candidate 49c3856ee800410f95f6b9695bd926d6 in this term.
I20260709 13:00:18.938357   760 tablet_bootstrap.cc:492] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759: No bootstrap required, opened a new log
I20260709 13:00:18.938644   605 leader_election.cc:304] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea; no voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759
I20260709 13:00:18.941238   765 raft_consensus.cc:2749] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:18.943451   759 tablet_bootstrap.cc:492] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6: No bootstrap required, opened a new log
I20260709 13:00:18.944103   759 ts_tablet_manager.cc:1403] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6: Time spent bootstrapping tablet: real 0.182s	user 0.018s	sys 0.018s
I20260709 13:00:18.945173   760 ts_tablet_manager.cc:1403] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759: Time spent bootstrapping tablet: real 0.141s	user 0.025s	sys 0.006s
I20260709 13:00:18.952961   760 raft_consensus.cc:359] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.957095   758 tablet_bootstrap.cc:492] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea: No bootstrap required, opened a new log
I20260709 13:00:18.956622   760 raft_consensus.cc:385] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.955638   759 raft_consensus.cc:359] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.960915   760 raft_consensus.cc:740] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.961175   759 raft_consensus.cc:385] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.958359   758 ts_tablet_manager.cc:1403] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea: Time spent bootstrapping tablet: real 0.173s	user 0.012s	sys 0.029s
I20260709 13:00:18.961639   759 raft_consensus.cc:740] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.964339   759 consensus_queue.cc:260] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.964898   758 raft_consensus.cc:359] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.965757   758 raft_consensus.cc:385] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:18.966138   758 raft_consensus.cc:740] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Initialized, Role: FOLLOWER
I20260709 13:00:18.966552   760 consensus_queue.cc:260] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.969223   462 catalog_manager.cc:5697] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "49c3856ee800410f95f6b9695bd926d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:18.970908   760 ts_tablet_manager.cc:1434] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.002s
I20260709 13:00:18.972064   760 tablet_bootstrap.cc:492] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759: Bootstrap starting.
I20260709 13:00:18.975765   758 consensus_queue.cc:260] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:18.969203   759 ts_tablet_manager.cc:1434] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.004s
I20260709 13:00:18.977892   759 tablet_bootstrap.cc:492] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6: Bootstrap starting.
I20260709 13:00:18.981130   758 ts_tablet_manager.cc:1434] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:00:18.988503   758 tablet_bootstrap.cc:492] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea: Bootstrap starting.
I20260709 13:00:18.988561   760 tablet_bootstrap.cc:654] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.002775   759 tablet_bootstrap.cc:654] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.003185   760 tablet_bootstrap.cc:492] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759: No bootstrap required, opened a new log
I20260709 13:00:19.003626   760 ts_tablet_manager.cc:1403] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.008s
I20260709 13:00:19.008136   760 raft_consensus.cc:359] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.011657   760 raft_consensus.cc:385] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.011973   760 raft_consensus.cc:740] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.012741   760 consensus_queue.cc:260] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.015008   760 ts_tablet_manager.cc:1434] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:00:19.015980   760 tablet_bootstrap.cc:492] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759: Bootstrap starting.
I20260709 13:00:19.019912   461 catalog_manager.cc:5697] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 75b161200bff4cc8a1789a7247d37fea (127.0.107.194). New cstate: current_term: 1 leader_uuid: "75b161200bff4cc8a1789a7247d37fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.022094   461 catalog_manager.cc:5697] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 75b161200bff4cc8a1789a7247d37fea (127.0.107.194). New cstate: current_term: 1 leader_uuid: "75b161200bff4cc8a1789a7247d37fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.029309   760 tablet_bootstrap.cc:654] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.047377   760 tablet_bootstrap.cc:492] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759: No bootstrap required, opened a new log
I20260709 13:00:19.047885   760 ts_tablet_manager.cc:1403] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.005s
I20260709 13:00:19.050117   758 tablet_bootstrap.cc:654] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.050798   760 raft_consensus.cc:359] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.051620   760 raft_consensus.cc:385] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.051918   760 raft_consensus.cc:740] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.052735   760 consensus_queue.cc:260] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.057348   759 tablet_bootstrap.cc:492] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6: No bootstrap required, opened a new log
I20260709 13:00:19.057936   759 ts_tablet_manager.cc:1403] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6: Time spent bootstrapping tablet: real 0.080s	user 0.009s	sys 0.020s
I20260709 13:00:19.061275   760 ts_tablet_manager.cc:1434] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:00:19.061676   759 raft_consensus.cc:359] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.063959   759 raft_consensus.cc:385] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.066888   759 raft_consensus.cc:740] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.067798   759 consensus_queue.cc:260] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.070297   759 ts_tablet_manager.cc:1434] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6: Time spent starting tablet: real 0.012s	user 0.000s	sys 0.009s
I20260709 13:00:19.071365   759 tablet_bootstrap.cc:492] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6: Bootstrap starting.
I20260709 13:00:19.063473   758 tablet_bootstrap.cc:492] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea: No bootstrap required, opened a new log
I20260709 13:00:19.072332   758 ts_tablet_manager.cc:1403] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea: Time spent bootstrapping tablet: real 0.084s	user 0.012s	sys 0.002s
I20260709 13:00:19.075801   758 raft_consensus.cc:359] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.076632   758 raft_consensus.cc:385] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.076928   758 raft_consensus.cc:740] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.085649   758 consensus_queue.cc:260] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.086198   714 raft_consensus.cc:3060] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.088738   758 ts_tablet_manager.cc:1434] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:00:19.089895   758 tablet_bootstrap.cc:492] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea: Bootstrap starting.
I20260709 13:00:19.092290   759 tablet_bootstrap.cc:654] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.097879   758 tablet_bootstrap.cc:654] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.100433   765 consensus_queue.cc:1048] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.121647   759 tablet_bootstrap.cc:492] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6: No bootstrap required, opened a new log
I20260709 13:00:19.122264   759 ts_tablet_manager.cc:1403] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6: Time spent bootstrapping tablet: real 0.051s	user 0.017s	sys 0.006s
I20260709 13:00:19.122948   758 tablet_bootstrap.cc:492] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea: No bootstrap required, opened a new log
I20260709 13:00:19.123481   758 ts_tablet_manager.cc:1403] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.007s
I20260709 13:00:19.125494   759 raft_consensus.cc:359] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.126292   759 raft_consensus.cc:385] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.126717   759 raft_consensus.cc:740] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.126174   758 raft_consensus.cc:359] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.127312   758 raft_consensus.cc:385] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.127676   758 raft_consensus.cc:740] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.127658   759 consensus_queue.cc:260] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.130499   759 ts_tablet_manager.cc:1434] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:00:19.132174   759 tablet_bootstrap.cc:492] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6: Bootstrap starting.
I20260709 13:00:19.139811   759 tablet_bootstrap.cc:654] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.128918   758 consensus_queue.cc:260] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.149035   777 raft_consensus.cc:493] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.149204   758 ts_tablet_manager.cc:1434] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:00:19.149686   777 raft_consensus.cc:515] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.150775   758 tablet_bootstrap.cc:492] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea: Bootstrap starting.
I20260709 13:00:19.151757   793 consensus_queue.cc:1048] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:19.154389   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2865aa77674948499e8d5e89dd117de7" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6" is_pre_election: true
W20260709 13:00:19.156273   680 leader_election.cc:343] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.157401   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2865aa77674948499e8d5e89dd117de7" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea" is_pre_election: true
I20260709 13:00:19.157624   758 tablet_bootstrap.cc:654] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.158106   640 raft_consensus.cc:2468] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 0.
I20260709 13:00:19.162773   764 consensus_queue.cc:1048] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.163173   679 leader_election.cc:304] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 49c3856ee800410f95f6b9695bd926d6
I20260709 13:00:19.164479   764 raft_consensus.cc:2804] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.164911   764 raft_consensus.cc:493] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.165287   777 leader_election.cc:290] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:19.165306   764 raft_consensus.cc:3060] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.176213   777 raft_consensus.cc:493] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.177350   777 raft_consensus.cc:515] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.178745   758 tablet_bootstrap.cc:492] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea: No bootstrap required, opened a new log
I20260709 13:00:19.179551   758 ts_tablet_manager.cc:1403] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.016s
I20260709 13:00:19.180962   777 leader_election.cc:290] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:19.181156   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6" is_pre_election: true
I20260709 13:00:19.182029   563 raft_consensus.cc:2468] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 0.
I20260709 13:00:19.183606   680 leader_election.cc:304] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:19.184508   777 raft_consensus.cc:2804] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.184280   758 raft_consensus.cc:359] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.184973   777 raft_consensus.cc:493] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.185554   758 raft_consensus.cc:385] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.185642   777 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.186111   758 raft_consensus.cc:740] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.187301   758 consensus_queue.cc:260] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.190267   758 ts_tablet_manager.cc:1434] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 13:00:19.198530   764 raft_consensus.cc:515] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.202126   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2865aa77674948499e8d5e89dd117de7" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:19.202189   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea" is_pre_election: true
I20260709 13:00:19.203231   640 raft_consensus.cc:2468] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 0.
W20260709 13:00:19.203924   680 leader_election.cc:343] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:19.243005   777 raft_consensus.cc:515] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.247370   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:19.248096   562 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.261896   562 raft_consensus.cc:2468] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 1.
I20260709 13:00:19.262717   792 consensus_queue.cc:1048] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.277427   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2865aa77674948499e8d5e89dd117de7" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:19.278230   640 raft_consensus.cc:3060] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.279392   795 raft_consensus.cc:493] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.279903   795 raft_consensus.cc:515] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.282423   795 leader_election.cc:290] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:19.284639   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea" is_pre_election: true
I20260709 13:00:19.287348   640 raft_consensus.cc:2468] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 1.
I20260709 13:00:19.288933   679 leader_election.cc:304] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 49c3856ee800410f95f6b9695bd926d6
I20260709 13:00:19.302515   777 leader_election.cc:290] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:19.293052   764 leader_election.cc:290] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:19.319034   795 raft_consensus.cc:493] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.320580   795 raft_consensus.cc:515] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.341382   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" is_pre_election: true
I20260709 13:00:19.341496   831 raft_consensus.cc:2804] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.342227   795 leader_election.cc:290] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:19.342408   831 raft_consensus.cc:697] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Becoming Leader. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:19.345403   530 leader_election.cc:304] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6; no voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759
I20260709 13:00:19.343578   831 consensus_queue.cc:237] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.350250   759 tablet_bootstrap.cc:492] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6: No bootstrap required, opened a new log
W20260709 13:00:19.353931   679 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40969, user_credentials={real_user=slave}} blocked reactor thread for 65114.7us
I20260709 13:00:19.354117   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f1df71364ad4b3dbdd84404e9bf9b44" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea" is_pre_election: true
I20260709 13:00:19.348682   795 raft_consensus.cc:2749] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:19.355059   680 leader_election.cc:304] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:19.356911   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f1df71364ad4b3dbdd84404e9bf9b44" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" is_pre_election: true
I20260709 13:00:19.365717   714 raft_consensus.cc:2468] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 0.
I20260709 13:00:19.366452   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:19.366691   566 raft_consensus.cc:3060] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.367178   641 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.361660   759 ts_tablet_manager.cc:1403] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6: Time spent bootstrapping tablet: real 0.230s	user 0.073s	sys 0.102s
I20260709 13:00:19.369457   530 leader_election.cc:304] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:19.368165   640 raft_consensus.cc:2468] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 0.
I20260709 13:00:19.370507   831 raft_consensus.cc:2804] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.371114   831 raft_consensus.cc:697] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Becoming Leader. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:19.371456   795 raft_consensus.cc:2804] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:19.371835   795 raft_consensus.cc:493] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.372236   795 raft_consensus.cc:3060] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.379423   759 raft_consensus.cc:359] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.380183   759 raft_consensus.cc:385] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.374626   831 consensus_queue.cc:237] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.380499   759 raft_consensus.cc:740] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.383133   759 consensus_queue.cc:260] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
W20260709 13:00:19.388089   680 consensus_peers.cc:597] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 -> Peer 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271): Couldn't send request to peer 49c3856ee800410f95f6b9695bd926d6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:19.384788   641 raft_consensus.cc:2468] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 1.
I20260709 13:00:19.398618   759 ts_tablet_manager.cc:1434] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 13:00:19.403455   759 tablet_bootstrap.cc:492] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6: Bootstrap starting.
I20260709 13:00:19.413208   792 consensus_queue.cc:1048] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.413816   795 raft_consensus.cc:515] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.417714   641 raft_consensus.cc:3060] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.419931   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f1df71364ad4b3dbdd84404e9bf9b44" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:19.425390   639 raft_consensus.cc:3060] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.429153   831 raft_consensus.cc:493] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.434567   831 raft_consensus.cc:515] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.420408   764 consensus_queue.cc:1048] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:00:19.438010   831 leader_election.cc:290] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:19.439721   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea" is_pre_election: true
W20260709 13:00:19.440874   511 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:19.451157   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f1df71364ad4b3dbdd84404e9bf9b44" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:19.452225   714 raft_consensus.cc:3060] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.456318   563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6" is_pre_election: true
I20260709 13:00:19.460879   680 leader_election.cc:304] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e4f3f8d22f334cfeb9dc7316c2771759; no voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea
I20260709 13:00:19.462255   764 raft_consensus.cc:2749] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:19.463419   792 consensus_queue.cc:1048] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.463759   714 raft_consensus.cc:2468] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 1.
I20260709 13:00:19.466631   461 catalog_manager.cc:5697] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:19.471066   836 consensus_queue.cc:1048] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:00:19.458535   795 leader_election.cc:290] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 election: Requested vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:19.477501   639 raft_consensus.cc:2468] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 1.
I20260709 13:00:19.478325   530 leader_election.cc:304] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:19.479264   795 raft_consensus.cc:2804] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:19.479866   795 raft_consensus.cc:697] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 1 LEADER]: Becoming Leader. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:19.480865   795 consensus_queue.cc:237] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.485307   759 tablet_bootstrap.cc:654] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:19.474517   461 catalog_manager.cc:5697] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:19.504040   714 raft_consensus.cc:3060] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.517855   767 consensus_queue.cc:1048] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.556066   792 raft_consensus.cc:493] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.557175   792 raft_consensus.cc:515] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.565730   764 consensus_queue.cc:1048] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.568869   765 consensus_queue.cc:1048] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.577061   759 tablet_bootstrap.cc:492] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6: No bootstrap required, opened a new log
I20260709 13:00:19.577566   759 ts_tablet_manager.cc:1403] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6: Time spent bootstrapping tablet: real 0.178s	user 0.016s	sys 0.023s
I20260709 13:00:19.578338   792 leader_election.cc:290] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:19.580274   759 raft_consensus.cc:359] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.580960   759 raft_consensus.cc:385] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:19.580883   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725a4a0982b1482eacea62cdebb7cb3e" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c3856ee800410f95f6b9695bd926d6" is_pre_election: true
I20260709 13:00:19.581306   759 raft_consensus.cc:740] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Initialized, Role: FOLLOWER
I20260709 13:00:19.582017   759 consensus_queue.cc:260] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.602092   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725a4a0982b1482eacea62cdebb7cb3e" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" is_pre_election: true
I20260709 13:00:19.642057   843 raft_consensus.cc:493] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.642728   843 raft_consensus.cc:515] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.643646   605 leader_election.cc:304] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea; no voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759
I20260709 13:00:19.610899   759 ts_tablet_manager.cc:1434] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.001s
I20260709 13:00:19.655368   847 raft_consensus.cc:2749] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:19.656957   843 leader_election.cc:290] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:19.659101   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea" is_pre_election: true
I20260709 13:00:19.658967   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" is_pre_election: true
I20260709 13:00:19.663628   530 leader_election.cc:304] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6; no voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759
I20260709 13:00:19.664798   843 raft_consensus.cc:2749] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:19.713130   462 catalog_manager.cc:5697] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "49c3856ee800410f95f6b9695bd926d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:19.724572   843 raft_consensus.cc:493] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.725170   843 raft_consensus.cc:515] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:19.727764   843 leader_election.cc:290] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:19.730150   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2865aa77674948499e8d5e89dd117de7" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b161200bff4cc8a1789a7247d37fea" is_pre_election: true
I20260709 13:00:19.731154   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2865aa77674948499e8d5e89dd117de7" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" is_pre_election: true
I20260709 13:00:19.731580   641 raft_consensus.cc:2393] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49c3856ee800410f95f6b9695bd926d6 in current term 1: Already voted for candidate e4f3f8d22f334cfeb9dc7316c2771759 in this term.
I20260709 13:00:19.733366   530 leader_election.cc:304] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6; no voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759
I20260709 13:00:19.734532   843 raft_consensus.cc:3060] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.742753   843 raft_consensus.cc:2749] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:19.745651   764 consensus_queue.cc:1048] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:19.762300   830 consensus_queue.cc:1048] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.781946   764 consensus_queue.cc:1048] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:19.795550   777 consensus_queue.cc:1048] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:19.876993   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec5a7dee7d3b4b2cb847171ad1e80e98"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
mode: GRACEFUL
new_leader_uuid: "49c3856ee800410f95f6b9695bd926d6"
 from {username='slave'} at 127.0.0.1:54560
I20260709 13:00:19.877767   640 raft_consensus.cc:606] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Received request to transfer leadership to TS 49c3856ee800410f95f6b9695bd926d6
I20260709 13:00:19.884267   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
mode: GRACEFUL
new_leader_uuid: "49c3856ee800410f95f6b9695bd926d6"
 from {username='slave'} at 127.0.0.1:54560
I20260709 13:00:19.885109   640 raft_consensus.cc:606] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Received request to transfer leadership to TS 49c3856ee800410f95f6b9695bd926d6
I20260709 13:00:19.891065   562 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:19.922262   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2"
dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
mode: GRACEFUL
new_leader_uuid: "49c3856ee800410f95f6b9695bd926d6"
 from {username='slave'} at 127.0.0.1:37456
I20260709 13:00:19.923440   714 raft_consensus.cc:606] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Received request to transfer leadership to TS 49c3856ee800410f95f6b9695bd926d6
I20260709 13:00:19.931020   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
mode: GRACEFUL
new_leader_uuid: "49c3856ee800410f95f6b9695bd926d6"
 from {username='slave'} at 127.0.0.1:54560
I20260709 13:00:19.931859   640 raft_consensus.cc:606] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Received request to transfer leadership to TS 49c3856ee800410f95f6b9695bd926d6
I20260709 13:00:19.939828   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725"
dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
mode: GRACEFUL
new_leader_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:37456
I20260709 13:00:19.940589   714 raft_consensus.cc:606] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Received request to transfer leadership to TS 75b161200bff4cc8a1789a7247d37fea
I20260709 13:00:19.948323   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "725a4a0982b1482eacea62cdebb7cb3e"
dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
mode: GRACEFUL
new_leader_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:37456
I20260709 13:00:19.949061   714 raft_consensus.cc:606] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Received request to transfer leadership to TS 75b161200bff4cc8a1789a7247d37fea
I20260709 13:00:19.989740   562 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "acad783b77864a29a0bfcfdd201e12f6"
dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
mode: GRACEFUL
new_leader_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:60674
I20260709 13:00:19.990459   562 raft_consensus.cc:606] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 1 LEADER]: Received request to transfer leadership to TS 75b161200bff4cc8a1789a7247d37fea
I20260709 13:00:19.995595   831 raft_consensus.cc:993] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759: : Instructing follower 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:19.996127   831 raft_consensus.cc:1081] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Signalling peer 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:19.998376   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "725a4a0982b1482eacea62cdebb7cb3e"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:54536
I20260709 13:00:19.999086   640 raft_consensus.cc:493] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:19.999492   640 raft_consensus.cc:3060] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.000681   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2865aa77674948499e8d5e89dd117de7"
dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
mode: GRACEFUL
new_leader_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:37456
I20260709 13:00:20.001356   714 raft_consensus.cc:606] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Received request to transfer leadership to TS 75b161200bff4cc8a1789a7247d37fea
I20260709 13:00:20.008385   640 raft_consensus.cc:515] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.009141   562 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f1df71364ad4b3dbdd84404e9bf9b44"
dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
mode: GRACEFUL
new_leader_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
 from {username='slave'} at 127.0.0.1:60674
I20260709 13:00:20.009907   562 raft_consensus.cc:606] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 1 LEADER]: Received request to transfer leadership to TS e4f3f8d22f334cfeb9dc7316c2771759
I20260709 13:00:20.012897   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725a4a0982b1482eacea62cdebb7cb3e" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:20.013525   714 raft_consensus.cc:3055] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.013634   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725a4a0982b1482eacea62cdebb7cb3e" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:20.013919   714 raft_consensus.cc:740] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:20.014329   562 raft_consensus.cc:3060] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.014783   714 consensus_queue.cc:260] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.016093   714 raft_consensus.cc:3060] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.022995   640 leader_election.cc:290] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 2 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:20.025553   562 raft_consensus.cc:2468] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 2.
I20260709 13:00:20.027096   606 leader_election.cc:304] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea; no voters: 
I20260709 13:00:20.028121   765 raft_consensus.cc:2804] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:20.028650   765 raft_consensus.cc:697] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [term 2 LEADER]: Becoming Leader. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:20.028752   714 raft_consensus.cc:2468] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 2.
I20260709 13:00:20.029531   765 consensus_queue.cc:237] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.039237   460 catalog_manager.cc:5697] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea reported cstate change: term changed from 1 to 2, leader changed from e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195) to 75b161200bff4cc8a1789a7247d37fea (127.0.107.194). New cstate: current_term: 2 leader_uuid: "75b161200bff4cc8a1789a7247d37fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:20.050441   847 raft_consensus.cc:993] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea: : Instructing follower 49c3856ee800410f95f6b9695bd926d6 to start an election
I20260709 13:00:20.050998   847 raft_consensus.cc:1081] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Signalling peer 49c3856ee800410f95f6b9695bd926d6 to start an election
I20260709 13:00:20.053288   562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d"
dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
 from {username='slave'} at 127.0.0.1:60664
I20260709 13:00:20.053983   562 raft_consensus.cc:493] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:20.054397   562 raft_consensus.cc:3060] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.062433   562 raft_consensus.cc:515] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.066001   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:20.066730   640 raft_consensus.cc:3055] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.067142   640 raft_consensus.cc:740] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:20.068084   640 consensus_queue.cc:260] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.069291   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:20.069554   640 raft_consensus.cc:3060] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.070060   714 raft_consensus.cc:3060] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.075075   562 leader_election.cc:290] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 2 election: Requested vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:20.077589   640 raft_consensus.cc:2468] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 2.
I20260709 13:00:20.079176   530 leader_election.cc:304] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea; no voters: 
I20260709 13:00:20.080375   852 raft_consensus.cc:2804] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:20.080942   852 raft_consensus.cc:697] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Becoming Leader. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:20.081049   714 raft_consensus.cc:2468] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 2.
I20260709 13:00:20.082039   852 consensus_queue.cc:237] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.093503   852 consensus_queue.cc:1048] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:20.093201   460 catalog_manager.cc:5697] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 reported cstate change: term changed from 1 to 2, leader changed from 75b161200bff4cc8a1789a7247d37fea (127.0.107.194) to 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "49c3856ee800410f95f6b9695bd926d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:20.099876   843 consensus_queue.cc:1048] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:20.118698   836 raft_consensus.cc:993] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6: : Instructing follower e4f3f8d22f334cfeb9dc7316c2771759 to start an election
I20260709 13:00:20.119386   836 raft_consensus.cc:1081] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 1 LEADER]: Signalling peer e4f3f8d22f334cfeb9dc7316c2771759 to start an election
I20260709 13:00:20.124778   713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f1df71364ad4b3dbdd84404e9bf9b44"
dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
 from {username='slave'} at 127.0.0.1:37446
I20260709 13:00:20.125452   713 raft_consensus.cc:493] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:20.125892   713 raft_consensus.cc:3060] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.139756   713 raft_consensus.cc:515] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.146309   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f1df71364ad4b3dbdd84404e9bf9b44" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:20.146991   562 raft_consensus.cc:3055] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.147413   562 raft_consensus.cc:740] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:20.148365   562 consensus_queue.cc:260] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.149734   562 raft_consensus.cc:3060] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.159868   562 raft_consensus.cc:2468] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 2.
I20260709 13:00:20.161345   680 leader_election.cc:304] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:20.162814   777 raft_consensus.cc:2804] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:20.163349   777 raft_consensus.cc:697] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 LEADER]: Becoming Leader. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:20.164094   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f1df71364ad4b3dbdd84404e9bf9b44" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:20.164184   777 consensus_queue.cc:237] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.164896   640 raft_consensus.cc:3060] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.165061   713 leader_election.cc:290] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 2 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:20.179538   640 raft_consensus.cc:2468] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 2.
I20260709 13:00:20.180756   460 catalog_manager.cc:5697] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 reported cstate change: term changed from 1 to 2, leader changed from 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193) to e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:20.213631   793 raft_consensus.cc:993] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea: : Instructing follower 49c3856ee800410f95f6b9695bd926d6 to start an election
I20260709 13:00:20.214095   765 raft_consensus.cc:1081] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Signalling peer 49c3856ee800410f95f6b9695bd926d6 to start an election
I20260709 13:00:20.215812   562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f"
dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
 from {username='slave'} at 127.0.0.1:60664
I20260709 13:00:20.216437   562 raft_consensus.cc:493] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:20.216766   562 raft_consensus.cc:3060] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.222282   562 raft_consensus.cc:515] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.224217   562 leader_election.cc:290] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 2 election: Requested vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:20.225399   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:20.225852   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:20.226025   640 raft_consensus.cc:3055] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.226377   640 raft_consensus.cc:740] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:20.226431   713 raft_consensus.cc:3060] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.227306   640 consensus_queue.cc:260] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.228677   640 raft_consensus.cc:3060] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.234999   713 raft_consensus.cc:2468] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 2.
I20260709 13:00:20.235956   640 raft_consensus.cc:2468] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 2.
I20260709 13:00:20.236469   530 leader_election.cc:304] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:20.237329   836 raft_consensus.cc:2804] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:20.237819   836 raft_consensus.cc:697] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Becoming Leader. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:20.238605   836 consensus_queue.cc:237] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.246906   460 catalog_manager.cc:5697] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 reported cstate change: term changed from 1 to 2, leader changed from 75b161200bff4cc8a1789a7247d37fea (127.0.107.194) to 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "49c3856ee800410f95f6b9695bd926d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:20.278208   831 raft_consensus.cc:993] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759: : Instructing follower 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:20.278717   831 raft_consensus.cc:1081] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Signalling peer 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:20.280925   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2865aa77674948499e8d5e89dd117de7"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:54536
I20260709 13:00:20.281566   640 raft_consensus.cc:493] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:20.282013   640 raft_consensus.cc:3060] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.289788   640 raft_consensus.cc:515] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.293078   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2865aa77674948499e8d5e89dd117de7" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:20.293746   562 raft_consensus.cc:3060] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.301234   562 raft_consensus.cc:2468] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 2.
I20260709 13:00:20.302838   606 leader_election.cc:304] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea; no voters: 
I20260709 13:00:20.303366   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2865aa77674948499e8d5e89dd117de7" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:20.303858   765 raft_consensus.cc:2804] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:20.304343   640 leader_election.cc:290] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 2 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:20.304706   765 raft_consensus.cc:697] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 2 LEADER]: Becoming Leader. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:20.304072   713 raft_consensus.cc:3055] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.306308   713 raft_consensus.cc:740] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:20.305785   765 consensus_queue.cc:237] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.307379   713 consensus_queue.cc:260] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.308787   713 raft_consensus.cc:3060] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.316175   460 catalog_manager.cc:5697] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea reported cstate change: term changed from 1 to 2, leader changed from e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195) to 75b161200bff4cc8a1789a7247d37fea (127.0.107.194). New cstate: current_term: 2 leader_uuid: "75b161200bff4cc8a1789a7247d37fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:20.318758   713 raft_consensus.cc:2468] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 2.
I20260709 13:00:20.353811   793 raft_consensus.cc:993] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea: : Instructing follower 49c3856ee800410f95f6b9695bd926d6 to start an election
I20260709 13:00:20.354436   765 raft_consensus.cc:1081] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Signalling peer 49c3856ee800410f95f6b9695bd926d6 to start an election
I20260709 13:00:20.356761   562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec5a7dee7d3b4b2cb847171ad1e80e98"
dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
 from {username='slave'} at 127.0.0.1:60664
I20260709 13:00:20.357391   562 raft_consensus.cc:493] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:20.357756   562 raft_consensus.cc:3060] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.365183   562 raft_consensus.cc:515] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.367564   562 leader_election.cc:290] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 2 election: Requested vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:20.369387   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec5a7dee7d3b4b2cb847171ad1e80e98" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:20.369530   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec5a7dee7d3b4b2cb847171ad1e80e98" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:20.370074   713 raft_consensus.cc:3060] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.370296   640 raft_consensus.cc:3055] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.370756   640 raft_consensus.cc:740] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:20.371862   640 consensus_queue.cc:260] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.373333   640 raft_consensus.cc:3060] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.377430   713 raft_consensus.cc:2468] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 2.
I20260709 13:00:20.379070   530 leader_election.cc:304] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:20.380105   836 raft_consensus.cc:2804] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:20.380651   836 raft_consensus.cc:697] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Becoming Leader. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:20.381649   836 consensus_queue.cc:237] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.386422   640 raft_consensus.cc:2468] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 2.
I20260709 13:00:20.386786   777 raft_consensus.cc:993] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759: : Instructing follower 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:20.387431   777 raft_consensus.cc:1081] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Signalling peer 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:20.389605   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:54536
I20260709 13:00:20.390332   640 raft_consensus.cc:493] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:20.390816   640 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.399539   460 catalog_manager.cc:5697] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 reported cstate change: term changed from 1 to 2, leader changed from 75b161200bff4cc8a1789a7247d37fea (127.0.107.194) to 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "49c3856ee800410f95f6b9695bd926d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:20.400635   640 raft_consensus.cc:515] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.404095   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:20.404795   562 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.411067   640 leader_election.cc:290] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 2 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:20.413228   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:20.414156   713 raft_consensus.cc:3055] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.414582   713 raft_consensus.cc:740] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:20.415773   713 consensus_queue.cc:260] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.416930   562 raft_consensus.cc:2468] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 2.
I20260709 13:00:20.417505   713 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.418571   606 leader_election.cc:304] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea; no voters: 
I20260709 13:00:20.419715   765 raft_consensus.cc:2804] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:20.420359   765 raft_consensus.cc:697] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 2 LEADER]: Becoming Leader. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:20.421253   765 consensus_queue.cc:237] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.430748   713 raft_consensus.cc:2468] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 2.
I20260709 13:00:20.431566   460 catalog_manager.cc:5697] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea reported cstate change: term changed from 1 to 2, leader changed from e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195) to 75b161200bff4cc8a1789a7247d37fea (127.0.107.194). New cstate: current_term: 2 leader_uuid: "75b161200bff4cc8a1789a7247d37fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:20.440016   764 raft_consensus.cc:993] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759: : Instructing follower 49c3856ee800410f95f6b9695bd926d6 to start an election
I20260709 13:00:20.440588   764 raft_consensus.cc:1081] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Signalling peer 49c3856ee800410f95f6b9695bd926d6 to start an election
I20260709 13:00:20.447736   562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2"
dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
 from {username='slave'} at 127.0.0.1:60656
I20260709 13:00:20.448484   562 raft_consensus.cc:493] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:20.449070   562 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.457639   562 raft_consensus.cc:515] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.462805   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:20.463492   640 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.467548   562 leader_election.cc:290] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 2 election: Requested vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:20.467782   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:20.468447   713 raft_consensus.cc:3055] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.468801   713 raft_consensus.cc:740] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:20.469660   713 consensus_queue.cc:260] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.470973   713 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.473855   640 raft_consensus.cc:2468] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 2.
I20260709 13:00:20.477247   714 raft_consensus.cc:1275] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.479331   765 consensus_queue.cc:1048] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:20.481374   530 leader_election.cc:304] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea; no voters: 
I20260709 13:00:20.481782   713 raft_consensus.cc:2468] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 2.
I20260709 13:00:20.483866   795 raft_consensus.cc:2804] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:20.484411   795 raft_consensus.cc:697] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Becoming Leader. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:20.485316   795 consensus_queue.cc:237] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.491828   713 raft_consensus.cc:1275] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Refusing update from remote peer 49c3856ee800410f95f6b9695bd926d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.494143   795 consensus_queue.cc:1048] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:20.495826   460 catalog_manager.cc:5697] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 reported cstate change: term changed from 1 to 2, leader changed from e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195) to 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "49c3856ee800410f95f6b9695bd926d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:20.523622   562 raft_consensus.cc:1275] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.530550   847 consensus_queue.cc:1048] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:20.538520   640 raft_consensus.cc:1275] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Refusing update from remote peer 49c3856ee800410f95f6b9695bd926d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.540360   795 consensus_queue.cc:1048] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:20.576876   795 raft_consensus.cc:993] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6: : Instructing follower 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:20.577368   795 raft_consensus.cc:1081] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 1 LEADER]: Signalling peer 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:20.579180   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "acad783b77864a29a0bfcfdd201e12f6"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:54548
I20260709 13:00:20.579813   641 raft_consensus.cc:493] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:20.580194   641 raft_consensus.cc:3060] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.584906   641 raft_consensus.cc:515] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.587512   641 leader_election.cc:290] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 2 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:20.587677   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acad783b77864a29a0bfcfdd201e12f6" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:20.587575   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acad783b77864a29a0bfcfdd201e12f6" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:20.588369   715 raft_consensus.cc:3060] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.588455   562 raft_consensus.cc:3055] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:20.588964   562 raft_consensus.cc:740] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:20.589972   562 consensus_queue.cc:260] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.591339   562 raft_consensus.cc:3060] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:20.595494   715 raft_consensus.cc:2468] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 2.
I20260709 13:00:20.596869   605 leader_election.cc:304] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:20.597800   847 raft_consensus.cc:2804] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:20.598260   847 raft_consensus.cc:697] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 2 LEADER]: Becoming Leader. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:20.599099   847 consensus_queue.cc:237] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:20.599575   562 raft_consensus.cc:2468] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 2.
I20260709 13:00:20.609167   460 catalog_manager.cc:5697] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea reported cstate change: term changed from 1 to 2, leader changed from 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193) to 75b161200bff4cc8a1789a7247d37fea (127.0.107.194). New cstate: current_term: 2 leader_uuid: "75b161200bff4cc8a1789a7247d37fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:20.714699   562 raft_consensus.cc:1275] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Refusing update from remote peer e4f3f8d22f334cfeb9dc7316c2771759: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.716486   831 consensus_queue.cc:1048] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:20.729527   641 raft_consensus.cc:1275] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Refusing update from remote peer e4f3f8d22f334cfeb9dc7316c2771759: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.732393   764 consensus_queue.cc:1048] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:20.824025   641 raft_consensus.cc:1275] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Refusing update from remote peer 49c3856ee800410f95f6b9695bd926d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.825600   852 consensus_queue.cc:1048] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:20.828035   562 raft_consensus.cc:1275] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.829650   847 consensus_queue.cc:1048] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:20.830767   640 raft_consensus.cc:1275] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Refusing update from remote peer 49c3856ee800410f95f6b9695bd926d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.832801   852 consensus_queue.cc:1048] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:20.858767   715 raft_consensus.cc:1275] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Refusing update from remote peer 49c3856ee800410f95f6b9695bd926d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.863396   852 consensus_queue.cc:1048] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:20.871954   713 raft_consensus.cc:1275] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.874967   765 consensus_queue.cc:1048] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:20.891690   562 raft_consensus.cc:1275] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.910159   793 consensus_queue.cc:1048] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:20.928081   713 raft_consensus.cc:1275] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Refusing update from remote peer 49c3856ee800410f95f6b9695bd926d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.930059   795 consensus_queue.cc:1048] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:20.946668   715 raft_consensus.cc:1275] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.951830   793 consensus_queue.cc:1048] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:20.970882   713 raft_consensus.cc:1275] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Refusing update from remote peer 49c3856ee800410f95f6b9695bd926d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.973106   852 consensus_queue.cc:1048] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:20.983680   640 raft_consensus.cc:1275] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Refusing update from remote peer 49c3856ee800410f95f6b9695bd926d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:20.985106   843 consensus_queue.cc:1048] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:21.072453   713 raft_consensus.cc:1275] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:21.073827   793 consensus_queue.cc:1048] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:21.084789   566 raft_consensus.cc:1275] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:21.086493   873 consensus_queue.cc:1048] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:23.046551   431 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:00:23.047605   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:23.066107   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
mode: GRACEFUL
new_leader_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
 from {username='slave'} at 127.0.0.1:35770
I20260709 13:00:23.066702   640 raft_consensus.cc:606] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 2 LEADER]: Received request to transfer leadership to TS e4f3f8d22f334cfeb9dc7316c2771759
I20260709 13:00:23.079504   562 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2"
dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
mode: GRACEFUL
new_leader_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
 from {username='slave'} at 127.0.0.1:33650
I20260709 13:00:23.080174   562 raft_consensus.cc:606] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Received request to transfer leadership to TS e4f3f8d22f334cfeb9dc7316c2771759
I20260709 13:00:23.081442   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:00:23.081959   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:23.352792   885 raft_consensus.cc:993] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea: : Instructing follower e4f3f8d22f334cfeb9dc7316c2771759 to start an election
I20260709 13:00:23.353302   884 raft_consensus.cc:1081] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 2 LEADER]: Signalling peer e4f3f8d22f334cfeb9dc7316c2771759 to start an election
I20260709 13:00:23.354866   713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725"
dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
 from {username='slave'} at 127.0.0.1:37442
I20260709 13:00:23.355458   713 raft_consensus.cc:493] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:23.355862   713 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:23.361927   713 raft_consensus.cc:515] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:23.363826   713 leader_election.cc:290] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 3 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:23.365188   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:23.365218   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:23.365854   562 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:23.366011   640 raft_consensus.cc:3055] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:23.366433   640 raft_consensus.cc:740] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:23.367319   640 consensus_queue.cc:260] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:23.368374   640 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:23.371207   562 raft_consensus.cc:2468] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 3.
I20260709 13:00:23.372380   680 leader_election.cc:304] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:23.373129   883 raft_consensus.cc:2804] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:23.373236   640 raft_consensus.cc:2468] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 3.
I20260709 13:00:23.374782   883 raft_consensus.cc:697] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 LEADER]: Becoming Leader. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:23.375696   883 consensus_queue.cc:237] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:23.383102   459 catalog_manager.cc:5697] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 reported cstate change: term changed from 2 to 3, leader changed from 75b161200bff4cc8a1789a7247d37fea (127.0.107.194) to e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:23.621886   836 raft_consensus.cc:993] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6: : Instructing follower e4f3f8d22f334cfeb9dc7316c2771759 to start an election
I20260709 13:00:23.622357   895 raft_consensus.cc:1081] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Signalling peer e4f3f8d22f334cfeb9dc7316c2771759 to start an election
I20260709 13:00:23.623927   713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2"
dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
 from {username='slave'} at 127.0.0.1:37446
I20260709 13:00:23.624369   713 raft_consensus.cc:493] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:23.624643   713 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:23.630146   713 raft_consensus.cc:515] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:23.632027   713 leader_election.cc:290] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 3 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:23.633112   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:23.633149   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:23.633631   562 raft_consensus.cc:3055] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:23.633841   640 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:23.633930   562 raft_consensus.cc:740] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:23.634794   562 consensus_queue.cc:260] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:23.635718   562 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:23.638530   640 raft_consensus.cc:2468] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 3.
I20260709 13:00:23.639732   562 raft_consensus.cc:2468] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 3.
I20260709 13:00:23.639711   679 leader_election.cc:304] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:23.640542   883 raft_consensus.cc:2804] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:23.641693   883 raft_consensus.cc:697] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 LEADER]: Becoming Leader. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:23.642491   883 consensus_queue.cc:237] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:23.649834   459 catalog_manager.cc:5697] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 reported cstate change: term changed from 2 to 3, leader changed from 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193) to e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:23.799432   562 raft_consensus.cc:1275] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Refusing update from remote peer e4f3f8d22f334cfeb9dc7316c2771759: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:23.800977   883 consensus_queue.cc:1048] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:23.813445   640 raft_consensus.cc:1275] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Refusing update from remote peer e4f3f8d22f334cfeb9dc7316c2771759: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:23.815064   896 consensus_queue.cc:1048] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:24.040917   562 raft_consensus.cc:1275] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Refusing update from remote peer e4f3f8d22f334cfeb9dc7316c2771759: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:24.042447   896 consensus_queue.cc:1048] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:24.051349   640 raft_consensus.cc:1275] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Refusing update from remote peer e4f3f8d22f334cfeb9dc7316c2771759: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:24.053429   889 consensus_queue.cc:1048] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:24.082849   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:24.093495   562 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec5a7dee7d3b4b2cb847171ad1e80e98"
dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
mode: GRACEFUL
new_leader_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:60674
I20260709 13:00:24.094009   562 raft_consensus.cc:606] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Received request to transfer leadership to TS 75b161200bff4cc8a1789a7247d37fea
I20260709 13:00:24.099784   562 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d"
dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
mode: GRACEFUL
new_leader_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:60674
I20260709 13:00:24.100276   562 raft_consensus.cc:606] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Received request to transfer leadership to TS 75b161200bff4cc8a1789a7247d37fea
I20260709 13:00:24.105526   713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2"
dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
mode: GRACEFUL
new_leader_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:37456
I20260709 13:00:24.106161   713 raft_consensus.cc:606] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 LEADER]: Received request to transfer leadership to TS 75b161200bff4cc8a1789a7247d37fea
I20260709 13:00:24.112071   562 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f"
dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
mode: GRACEFUL
new_leader_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:60674
I20260709 13:00:24.112668   562 raft_consensus.cc:606] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Received request to transfer leadership to TS 75b161200bff4cc8a1789a7247d37fea
I20260709 13:00:24.118057   713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725"
dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
mode: GRACEFUL
new_leader_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:37456
I20260709 13:00:24.118631   713 raft_consensus.cc:606] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 LEADER]: Received request to transfer leadership to TS 75b161200bff4cc8a1789a7247d37fea
I20260709 13:00:24.175535   895 raft_consensus.cc:993] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6: : Instructing follower 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:24.176038   895 raft_consensus.cc:1081] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Signalling peer 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:24.177841   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:54548
I20260709 13:00:24.178375   640 raft_consensus.cc:493] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:24.178718   640 raft_consensus.cc:3060] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.183230   640 raft_consensus.cc:515] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.185909   640 leader_election.cc:290] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 3 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:24.186352   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:24.186507   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:24.187017   562 raft_consensus.cc:3055] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:24.187129   713 raft_consensus.cc:3060] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.187398   562 raft_consensus.cc:740] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:24.188171   562 consensus_queue.cc:260] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.188602   905 raft_consensus.cc:993] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6: : Instructing follower 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:24.189133   895 raft_consensus.cc:1081] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Signalling peer 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:24.189239   562 raft_consensus.cc:3060] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.190954   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec5a7dee7d3b4b2cb847171ad1e80e98"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:54548
I20260709 13:00:24.191533   641 raft_consensus.cc:493] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:24.191881   641 raft_consensus.cc:3060] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.193742   713 raft_consensus.cc:2468] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 3.
I20260709 13:00:24.194787   605 leader_election.cc:304] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:24.195571   884 raft_consensus.cc:2804] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:24.195499   562 raft_consensus.cc:2468] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 3.
I20260709 13:00:24.197146   884 raft_consensus.cc:697] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 3 LEADER]: Becoming Leader. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:24.197975   884 consensus_queue.cc:237] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.198609   641 raft_consensus.cc:515] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.201558   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec5a7dee7d3b4b2cb847171ad1e80e98" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:24.202101   562 raft_consensus.cc:3055] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:24.202400   562 raft_consensus.cc:740] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:24.202251   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec5a7dee7d3b4b2cb847171ad1e80e98" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:24.202970   713 raft_consensus.cc:3060] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.203403   562 consensus_queue.cc:260] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.205021   562 raft_consensus.cc:3060] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.209388   713 raft_consensus.cc:2468] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 3.
I20260709 13:00:24.209774   896 raft_consensus.cc:993] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759: : Instructing follower 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:24.210234   896 raft_consensus.cc:1081] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 LEADER]: Signalling peer 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:24.212374   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:54536
I20260709 13:00:24.213037   639 raft_consensus.cc:493] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:24.213514   639 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.214074   605 leader_election.cc:304] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:24.214587   562 raft_consensus.cc:2468] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 3.
I20260709 13:00:24.214848   906 raft_consensus.cc:2804] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:24.215109   641 leader_election.cc:290] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 3 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:24.220091   639 raft_consensus.cc:515] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.220870   906 raft_consensus.cc:697] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 3 LEADER]: Becoming Leader. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:24.221930   906 consensus_queue.cc:237] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.223520   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:24.223728   713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:24.224105   562 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.224332   713 raft_consensus.cc:3055] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:24.224767   713 raft_consensus.cc:740] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:24.224099   459 catalog_manager.cc:5697] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea reported cstate change: term changed from 2 to 3, leader changed from 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193) to 75b161200bff4cc8a1789a7247d37fea (127.0.107.194). New cstate: current_term: 3 leader_uuid: "75b161200bff4cc8a1789a7247d37fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.225659   713 consensus_queue.cc:260] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.227807   713 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.226761   639 leader_election.cc:290] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 4 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:24.233482   562 raft_consensus.cc:2468] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 4.
I20260709 13:00:24.234876   606 leader_election.cc:304] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea; no voters: 
I20260709 13:00:24.235814   908 raft_consensus.cc:2804] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:24.236344   908 raft_consensus.cc:697] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 4 LEADER]: Becoming Leader. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:24.237306   908 consensus_queue.cc:237] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.239099   713 raft_consensus.cc:2468] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 4.
I20260709 13:00:24.249450   459 catalog_manager.cc:5697] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea reported cstate change: term changed from 2 to 3, leader changed from 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193) to 75b161200bff4cc8a1789a7247d37fea (127.0.107.194). New cstate: current_term: 3 leader_uuid: "75b161200bff4cc8a1789a7247d37fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.250635   459 catalog_manager.cc:5697] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea reported cstate change: term changed from 3 to 4, leader changed from e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195) to 75b161200bff4cc8a1789a7247d37fea (127.0.107.194). New cstate: current_term: 4 leader_uuid: "75b161200bff4cc8a1789a7247d37fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.586037   562 raft_consensus.cc:1275] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:24.586154   905 raft_consensus.cc:993] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6: : Instructing follower 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:24.586717   895 raft_consensus.cc:1081] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Signalling peer 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:24.587479   906 consensus_queue.cc:1048] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:24.588493   639 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:54548
I20260709 13:00:24.589259   639 raft_consensus.cc:493] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:24.589672   639 raft_consensus.cc:3060] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.598670   889 raft_consensus.cc:993] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759: : Instructing follower 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:24.599222   889 raft_consensus.cc:1081] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 LEADER]: Signalling peer 75b161200bff4cc8a1789a7247d37fea to start an election
I20260709 13:00:24.601111   641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
 from {username='slave'} at 127.0.0.1:54536
I20260709 13:00:24.601766   641 raft_consensus.cc:493] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:24.602286   641 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.601797   713 raft_consensus.cc:1275] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:24.606382   884 consensus_queue.cc:1048] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:24.610549   641 raft_consensus.cc:515] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.610978   639 raft_consensus.cc:515] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.613121   641 leader_election.cc:290] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 4 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:24.614810   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:24.615361   562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:24.615594   566 raft_consensus.cc:3055] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:24.615926   562 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.615993   566 raft_consensus.cc:740] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:24.617002   566 consensus_queue.cc:260] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.618191   566 raft_consensus.cc:3060] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.614835   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:24.619728   715 raft_consensus.cc:3055] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:24.620064   715 raft_consensus.cc:740] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:24.620954   715 consensus_queue.cc:260] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.622267   715 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.623333   563 raft_consensus.cc:1275] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:24.625315   884 consensus_queue.cc:1048] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:24.630654   715 raft_consensus.cc:2468] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 4.
I20260709 13:00:24.632169   605 leader_election.cc:304] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:24.633214   884 raft_consensus.cc:2804] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:24.634370   562 raft_consensus.cc:2468] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 4.
I20260709 13:00:24.634869   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f" candidate_uuid: "75b161200bff4cc8a1789a7247d37fea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:24.635612   714 raft_consensus.cc:3060] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.636284   884 raft_consensus.cc:697] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 4 LEADER]: Becoming Leader. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:24.637316   884 consensus_queue.cc:237] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.639676   639 leader_election.cc:290] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 3 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:24.642196   566 raft_consensus.cc:2468] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 3.
I20260709 13:00:24.643971   606 leader_election.cc:304] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea; no voters: 
I20260709 13:00:24.645360   714 raft_consensus.cc:2468] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b161200bff4cc8a1789a7247d37fea in term 3.
I20260709 13:00:24.653271   908 raft_consensus.cc:2804] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:24.653934   908 raft_consensus.cc:697] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 3 LEADER]: Becoming Leader. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:24.656205   908 consensus_queue.cc:237] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:24.665946   460 catalog_manager.cc:5697] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea reported cstate change: term changed from 3 to 4, leader changed from e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195) to 75b161200bff4cc8a1789a7247d37fea (127.0.107.194). New cstate: current_term: 4 leader_uuid: "75b161200bff4cc8a1789a7247d37fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.668901   713 raft_consensus.cc:1275] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:24.670763   908 consensus_queue.cc:1048] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:24.683856   463 catalog_manager.cc:5697] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea reported cstate change: term changed from 2 to 3, leader changed from 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193) to 75b161200bff4cc8a1789a7247d37fea (127.0.107.194). New cstate: current_term: 3 leader_uuid: "75b161200bff4cc8a1789a7247d37fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:24.698601   713 raft_consensus.cc:1275] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:24.700245   908 consensus_queue.cc:1048] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:24.707890   563 raft_consensus.cc:1275] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 4 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:24.710408   908 consensus_queue.cc:1048] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:25.042569   563 raft_consensus.cc:1275] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:25.042572   713 raft_consensus.cc:1275] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:25.043370   566 raft_consensus.cc:1275] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 4 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:25.044997   919 consensus_queue.cc:1048] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:25.045357   908 consensus_queue.cc:1048] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:25.046141   884 consensus_queue.cc:1048] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:25.089139   713 raft_consensus.cc:1275] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 FOLLOWER]: Refusing update from remote peer 75b161200bff4cc8a1789a7247d37fea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:25.090646   884 consensus_queue.cc:1048] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:27.132953   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:27.140422   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
mode: GRACEFUL
new_leader_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
 from {username='slave'} at 127.0.0.1:35770
I20260709 13:00:27.141176   639 raft_consensus.cc:606] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 4 LEADER]: Received request to transfer leadership to TS e4f3f8d22f334cfeb9dc7316c2771759
I20260709 13:00:27.145287   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2865aa77674948499e8d5e89dd117de7"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
mode: GRACEFUL
new_leader_uuid: "49c3856ee800410f95f6b9695bd926d6"
 from {username='slave'} at 127.0.0.1:35770
I20260709 13:00:27.145844   639 raft_consensus.cc:606] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 2 LEADER]: Received request to transfer leadership to TS 49c3856ee800410f95f6b9695bd926d6
I20260709 13:00:27.148751   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
mode: GRACEFUL
new_leader_uuid: "49c3856ee800410f95f6b9695bd926d6"
 from {username='slave'} at 127.0.0.1:35770
I20260709 13:00:27.149427   639 raft_consensus.cc:606] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 4 LEADER]: Received request to transfer leadership to TS 49c3856ee800410f95f6b9695bd926d6
I20260709 13:00:27.152278   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
mode: GRACEFUL
new_leader_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
 from {username='slave'} at 127.0.0.1:35770
I20260709 13:00:27.152880   639 raft_consensus.cc:606] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 3 LEADER]: Received request to transfer leadership to TS e4f3f8d22f334cfeb9dc7316c2771759
I20260709 13:00:27.155529   639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d"
dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
mode: GRACEFUL
new_leader_uuid: "49c3856ee800410f95f6b9695bd926d6"
 from {username='slave'} at 127.0.0.1:35770
I20260709 13:00:27.156049   639 raft_consensus.cc:606] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 3 LEADER]: Received request to transfer leadership to TS 49c3856ee800410f95f6b9695bd926d6
I20260709 13:00:27.157331   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:00:27.158263   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:27.190794   908 raft_consensus.cc:993] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea: : Instructing follower 49c3856ee800410f95f6b9695bd926d6 to start an election
I20260709 13:00:27.191356   928 raft_consensus.cc:1081] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 3 LEADER]: Signalling peer 49c3856ee800410f95f6b9695bd926d6 to start an election
I20260709 13:00:27.193022   566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d"
dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
 from {username='slave'} at 127.0.0.1:60664
I20260709 13:00:27.193703   566 raft_consensus.cc:493] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:27.194084   566 raft_consensus.cc:3060] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.222081   908 raft_consensus.cc:993] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea: : Instructing follower 49c3856ee800410f95f6b9695bd926d6 to start an election
I20260709 13:00:27.222610   928 raft_consensus.cc:1081] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 4 LEADER]: Signalling peer 49c3856ee800410f95f6b9695bd926d6 to start an election
I20260709 13:00:27.224323   563 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2"
dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
 from {username='slave'} at 127.0.0.1:60664
I20260709 13:00:27.224854   563 raft_consensus.cc:493] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:27.225172   563 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.301389   566 raft_consensus.cc:515] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.304214   566 leader_election.cc:290] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 4 election: Requested vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:27.306211   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:27.306279   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f32ed3eafdb743e0b5527a057250e80d" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:27.307214   639 raft_consensus.cc:3055] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:27.307181   714 raft_consensus.cc:3060] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.307710   639 raft_consensus.cc:740] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:27.308621   639 consensus_queue.cc:260] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.310014   639 raft_consensus.cc:3060] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.402418   639 raft_consensus.cc:2468] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 4.
I20260709 13:00:27.402446   563 raft_consensus.cc:515] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.402457   714 raft_consensus.cc:2468] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 4.
I20260709 13:00:27.404621   530 leader_election.cc:304] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea; no voters: 
I20260709 13:00:27.404990   563 leader_election.cc:290] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 5 election: Requested vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:27.406417   929 raft_consensus.cc:2804] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:27.407490   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:27.408020   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b999efdb1de432fa180c502b4b2e0e2" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:27.408298   639 raft_consensus.cc:3055] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:27.408627   639 raft_consensus.cc:740] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:27.408560   929 raft_consensus.cc:697] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 4 LEADER]: Becoming Leader. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:27.408602   714 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.409883   929 consensus_queue.cc:237] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.409763   639 consensus_queue.cc:260] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.411152   639 raft_consensus.cc:3060] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.420569   463 catalog_manager.cc:5697] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 reported cstate change: term changed from 3 to 4, leader changed from 75b161200bff4cc8a1789a7247d37fea (127.0.107.194) to 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "49c3856ee800410f95f6b9695bd926d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:27.451218   908 raft_consensus.cc:993] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea: : Instructing follower e4f3f8d22f334cfeb9dc7316c2771759 to start an election
I20260709 13:00:27.451797   908 raft_consensus.cc:1081] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 4 LEADER]: Signalling peer e4f3f8d22f334cfeb9dc7316c2771759 to start an election
I20260709 13:00:27.453637   713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725"
dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
 from {username='slave'} at 127.0.0.1:37442
I20260709 13:00:27.454228   713 raft_consensus.cc:493] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:27.454511   713 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.471488   928 raft_consensus.cc:993] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea: : Instructing follower 49c3856ee800410f95f6b9695bd926d6 to start an election
I20260709 13:00:27.472019   928 raft_consensus.cc:1081] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 2 LEADER]: Signalling peer 49c3856ee800410f95f6b9695bd926d6 to start an election
I20260709 13:00:27.473629   566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2865aa77674948499e8d5e89dd117de7"
dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
 from {username='slave'} at 127.0.0.1:60664
I20260709 13:00:27.474073   566 raft_consensus.cc:493] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:27.474392   566 raft_consensus.cc:3060] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.503257   639 raft_consensus.cc:2468] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 5.
I20260709 13:00:27.503283   714 raft_consensus.cc:2468] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 5.
I20260709 13:00:27.504455   566 raft_consensus.cc:515] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.504778   530 leader_election.cc:304] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:27.505650   929 raft_consensus.cc:2804] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:27.506189   929 raft_consensus.cc:697] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 5 LEADER]: Becoming Leader. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:27.507408   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2865aa77674948499e8d5e89dd117de7" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:27.508042   639 raft_consensus.cc:3055] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:27.507421   929 consensus_queue.cc:237] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.508375   639 raft_consensus.cc:740] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:27.509136   639 consensus_queue.cc:260] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.510025   639 raft_consensus.cc:3060] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.512584   566 leader_election.cc:290] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 3 election: Requested vote from peers 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969), e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195:40989)
I20260709 13:00:27.513881   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2865aa77674948499e8d5e89dd117de7" candidate_uuid: "49c3856ee800410f95f6b9695bd926d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
I20260709 13:00:27.514439   714 raft_consensus.cc:3060] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.521703   463 catalog_manager.cc:5697] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 reported cstate change: term changed from 4 to 5, leader changed from 75b161200bff4cc8a1789a7247d37fea (127.0.107.194) to 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "49c3856ee800410f95f6b9695bd926d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:27.602789   936 raft_consensus.cc:993] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea: : Instructing follower e4f3f8d22f334cfeb9dc7316c2771759 to start an election
I20260709 13:00:27.603276   928 raft_consensus.cc:1081] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 3 LEADER]: Signalling peer e4f3f8d22f334cfeb9dc7316c2771759 to start an election
I20260709 13:00:27.604175   639 raft_consensus.cc:2468] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 3.
I20260709 13:00:27.605170   715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f"
dest_uuid: "e4f3f8d22f334cfeb9dc7316c2771759"
 from {username='slave'} at 127.0.0.1:37442
I20260709 13:00:27.605422   530 leader_election.cc:304] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, 75b161200bff4cc8a1789a7247d37fea; no voters: 
I20260709 13:00:27.605821   715 raft_consensus.cc:493] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:27.606257   715 raft_consensus.cc:3060] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.606554   929 raft_consensus.cc:2804] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:27.607333   929 raft_consensus.cc:697] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 3 LEADER]: Becoming Leader. State: Replica: 49c3856ee800410f95f6b9695bd926d6, State: Running, Role: LEADER
I20260709 13:00:27.608417   929 consensus_queue.cc:237] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.609452   714 raft_consensus.cc:2468] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c3856ee800410f95f6b9695bd926d6 in term 3.
I20260709 13:00:27.604252   713 raft_consensus.cc:515] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.613408   713 leader_election.cc:290] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 5 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:27.613765   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:27.613907   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "175f13d32fad40c8a7cb768f55aca725" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:27.614954   566 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.615180   639 raft_consensus.cc:3055] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:27.615662   639 raft_consensus.cc:740] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:27.616746   639 consensus_queue.cc:260] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.618444   639 raft_consensus.cc:3060] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.625675   463 catalog_manager.cc:5697] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 reported cstate change: term changed from 2 to 3, leader changed from 75b161200bff4cc8a1789a7247d37fea (127.0.107.194) to 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "49c3856ee800410f95f6b9695bd926d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:27.696696   566 raft_consensus.cc:2468] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 5.
I20260709 13:00:27.696837   639 raft_consensus.cc:2468] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 5.
I20260709 13:00:27.696732   715 raft_consensus.cc:515] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.698470   679 leader_election.cc:304] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75b161200bff4cc8a1789a7247d37fea, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:27.699920   896 raft_consensus.cc:2804] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:27.700739   715 leader_election.cc:290] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 4 election: Requested vote from peers 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271), 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969)
I20260709 13:00:27.700978   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "49c3856ee800410f95f6b9695bd926d6"
I20260709 13:00:27.701710   566 raft_consensus.cc:3060] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.703166   896 raft_consensus.cc:697] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 5 LEADER]: Becoming Leader. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:27.704329   896 consensus_queue.cc:237] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.704578   639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f43d3599a4018921d0f12ea30d25f" candidate_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "75b161200bff4cc8a1789a7247d37fea"
I20260709 13:00:27.705183   639 raft_consensus.cc:3055] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:27.705533   639 raft_consensus.cc:740] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 75b161200bff4cc8a1789a7247d37fea, State: Running, Role: LEADER
I20260709 13:00:27.706401   639 consensus_queue.cc:260] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.707791   639 raft_consensus.cc:3060] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.709267   566 raft_consensus.cc:2468] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 4.
I20260709 13:00:27.711064   680 leader_election.cc:304] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c3856ee800410f95f6b9695bd926d6, e4f3f8d22f334cfeb9dc7316c2771759; no voters: 
I20260709 13:00:27.711920   896 raft_consensus.cc:2804] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:27.712462   896 raft_consensus.cc:697] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 LEADER]: Becoming Leader. State: Replica: e4f3f8d22f334cfeb9dc7316c2771759, State: Running, Role: LEADER
I20260709 13:00:27.713327   896 consensus_queue.cc:237] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } }
I20260709 13:00:27.713934   463 catalog_manager.cc:5697] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 reported cstate change: term changed from 4 to 5, leader changed from 75b161200bff4cc8a1789a7247d37fea (127.0.107.194) to e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:27.714744   639 raft_consensus.cc:2468] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f3f8d22f334cfeb9dc7316c2771759 in term 4.
I20260709 13:00:27.726534   463 catalog_manager.cc:5697] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 reported cstate change: term changed from 3 to 4, leader changed from 75b161200bff4cc8a1789a7247d37fea (127.0.107.194) to e4f3f8d22f334cfeb9dc7316c2771759 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:27.892938   639 raft_consensus.cc:1275] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 4 FOLLOWER]: Refusing update from remote peer 49c3856ee800410f95f6b9695bd926d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:27.894546   929 consensus_queue.cc:1048] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:27.898087   715 raft_consensus.cc:1275] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 FOLLOWER]: Refusing update from remote peer 49c3856ee800410f95f6b9695bd926d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:27.900223   929 consensus_queue.cc:1048] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:28.016685   715 raft_consensus.cc:1275] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 5 FOLLOWER]: Refusing update from remote peer 49c3856ee800410f95f6b9695bd926d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:28.018396   930 consensus_queue.cc:1048] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:28.027431   639 raft_consensus.cc:1275] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 5 FOLLOWER]: Refusing update from remote peer 49c3856ee800410f95f6b9695bd926d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:28.029325   929 consensus_queue.cc:1048] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:28.094259   715 raft_consensus.cc:1275] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Refusing update from remote peer 49c3856ee800410f95f6b9695bd926d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:28.095755   929 consensus_queue.cc:1048] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f3f8d22f334cfeb9dc7316c2771759" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:28.108353   639 raft_consensus.cc:1275] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Refusing update from remote peer 49c3856ee800410f95f6b9695bd926d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:28.109756   948 consensus_queue.cc:1048] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:28.117300   641 raft_consensus.cc:1275] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 4 FOLLOWER]: Refusing update from remote peer e4f3f8d22f334cfeb9dc7316c2771759: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:28.118965   896 consensus_queue.cc:1048] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:28.128628   566 raft_consensus.cc:1275] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 4 FOLLOWER]: Refusing update from remote peer e4f3f8d22f334cfeb9dc7316c2771759: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:28.130076   896 consensus_queue.cc:1048] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:28.159616   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:28.163697   566 raft_consensus.cc:1275] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 5 FOLLOWER]: Refusing update from remote peer e4f3f8d22f334cfeb9dc7316c2771759: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:28.165328   896 consensus_queue.cc:1048] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c3856ee800410f95f6b9695bd926d6" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:28.168140   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:00:28.179268   639 raft_consensus.cc:1275] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 5 FOLLOWER]: Refusing update from remote peer e4f3f8d22f334cfeb9dc7316c2771759: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:00:28.180905   680 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:00:28.181180   896 consensus_queue.cc:1048] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b161200bff4cc8a1789a7247d37fea" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:00:28.197439   680 consensus_peers.cc:597] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 -> Peer 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271): Couldn't send request to peer 49c3856ee800410f95f6b9695bd926d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:28.221668   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:28.222581   431 tablet_replica.cc:333] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6: stopping tablet replica
I20260709 13:00:28.223425   431 raft_consensus.cc:2243] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.224419   431 raft_consensus.cc:2272] T 2865aa77674948499e8d5e89dd117de7 P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.227232   431 tablet_replica.cc:333] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6: stopping tablet replica
I20260709 13:00:28.227837   431 raft_consensus.cc:2243] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.228276   431 pending_rounds.cc:70] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6: Trying to abort 1 pending ops.
I20260709 13:00:28.228508   431 pending_rounds.cc:77] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6: Aborting op as it isn't in flight: id { term: 5 index: 5 } timestamp: 7305633905485705216 op_type: NO_OP noop_request { }
I20260709 13:00:28.228899   431 raft_consensus.cc:2889] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:00:28.229192   431 raft_consensus.cc:2272] T 175f13d32fad40c8a7cb768f55aca725 P 49c3856ee800410f95f6b9695bd926d6 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.232348   431 tablet_replica.cc:333] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6: stopping tablet replica
I20260709 13:00:28.233073   431 raft_consensus.cc:2243] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.233521   431 raft_consensus.cc:2272] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.235366   431 tablet_replica.cc:333] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6: stopping tablet replica
I20260709 13:00:28.235888   431 raft_consensus.cc:2243] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.236578   431 raft_consensus.cc:2272] T f32ed3eafdb743e0b5527a057250e80d P 49c3856ee800410f95f6b9695bd926d6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.238639   431 tablet_replica.cc:333] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6: stopping tablet replica
I20260709 13:00:28.239260   431 raft_consensus.cc:2243] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.239697   431 raft_consensus.cc:2272] T a80f43d3599a4018921d0f12ea30d25f P 49c3856ee800410f95f6b9695bd926d6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.241894   431 tablet_replica.cc:333] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6: stopping tablet replica
I20260709 13:00:28.242434   431 raft_consensus.cc:2243] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.243242   431 raft_consensus.cc:2272] T 2b999efdb1de432fa180c502b4b2e0e2 P 49c3856ee800410f95f6b9695bd926d6 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.245146   431 tablet_replica.cc:333] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6: stopping tablet replica
I20260709 13:00:28.245649   431 raft_consensus.cc:2243] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.246064   431 raft_consensus.cc:2272] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 49c3856ee800410f95f6b9695bd926d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.247794   431 tablet_replica.cc:333] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6: stopping tablet replica
I20260709 13:00:28.248265   431 raft_consensus.cc:2243] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.248673   431 raft_consensus.cc:2272] T acad783b77864a29a0bfcfdd201e12f6 P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.250293   431 tablet_replica.cc:333] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6: stopping tablet replica
I20260709 13:00:28.250718   431 raft_consensus.cc:2243] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.251138   431 raft_consensus.cc:2272] T 725a4a0982b1482eacea62cdebb7cb3e P 49c3856ee800410f95f6b9695bd926d6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:00:28.376415   680 consensus_peers.cc:597] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 -> Peer 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271): Couldn't send request to peer 49c3856ee800410f95f6b9695bd926d6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44271: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:28.390352   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:00:28.391745   606 consensus_peers.cc:597] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea -> Peer 49c3856ee800410f95f6b9695bd926d6 (127.0.107.193:44271): Couldn't send request to peer 49c3856ee800410f95f6b9695bd926d6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:44271: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:28.410692   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 13:00:28.443526   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:28.444316   431 tablet_replica.cc:333] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea: stopping tablet replica
I20260709 13:00:28.445235   431 raft_consensus.cc:2243] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.445847   431 raft_consensus.cc:2272] T 175f13d32fad40c8a7cb768f55aca725 P 75b161200bff4cc8a1789a7247d37fea [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.448271   431 tablet_replica.cc:333] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea: stopping tablet replica
I20260709 13:00:28.448814   431 raft_consensus.cc:2243] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.449256   431 raft_consensus.cc:2272] T 9f1df71364ad4b3dbdd84404e9bf9b44 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.451051   431 tablet_replica.cc:333] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea: stopping tablet replica
I20260709 13:00:28.451511   431 raft_consensus.cc:2243] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.452459   431 raft_consensus.cc:2272] T 725a4a0982b1482eacea62cdebb7cb3e P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.454474   431 tablet_replica.cc:333] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea: stopping tablet replica
I20260709 13:00:28.455125   431 raft_consensus.cc:2243] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.455528   431 raft_consensus.cc:2272] T 2865aa77674948499e8d5e89dd117de7 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.457901   431 tablet_replica.cc:333] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea: stopping tablet replica
I20260709 13:00:28.458541   431 raft_consensus.cc:2243] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.459105   431 raft_consensus.cc:2272] T 2b999efdb1de432fa180c502b4b2e0e2 P 75b161200bff4cc8a1789a7247d37fea [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.461573   431 tablet_replica.cc:333] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea: stopping tablet replica
I20260709 13:00:28.462163   431 raft_consensus.cc:2243] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.463117   431 raft_consensus.cc:2272] T acad783b77864a29a0bfcfdd201e12f6 P 75b161200bff4cc8a1789a7247d37fea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.464987   431 tablet_replica.cc:333] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea: stopping tablet replica
I20260709 13:00:28.465533   431 raft_consensus.cc:2243] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.466327   431 raft_consensus.cc:2272] T ec5a7dee7d3b4b2cb847171ad1e80e98 P 75b161200bff4cc8a1789a7247d37fea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.468575   431 tablet_replica.cc:333] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea: stopping tablet replica
I20260709 13:00:28.469105   431 raft_consensus.cc:2243] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.469571   431 raft_consensus.cc:2272] T a80f43d3599a4018921d0f12ea30d25f P 75b161200bff4cc8a1789a7247d37fea [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.471684   431 tablet_replica.cc:333] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea: stopping tablet replica
I20260709 13:00:28.472194   431 raft_consensus.cc:2243] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.472615   431 raft_consensus.cc:2272] T f32ed3eafdb743e0b5527a057250e80d P 75b161200bff4cc8a1789a7247d37fea [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.502841   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:00:28.520485   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:00:28.549558   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:00:28.550179   679 consensus_peers.cc:597] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 -> Peer 75b161200bff4cc8a1789a7247d37fea (127.0.107.194:40969): Couldn't send request to peer 75b161200bff4cc8a1789a7247d37fea. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40969: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:28.550475   431 tablet_replica.cc:333] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759: stopping tablet replica
I20260709 13:00:28.551484   431 raft_consensus.cc:2243] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.552515   431 raft_consensus.cc:2272] T 175f13d32fad40c8a7cb768f55aca725 P e4f3f8d22f334cfeb9dc7316c2771759 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.555533   431 tablet_replica.cc:333] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759: stopping tablet replica
I20260709 13:00:28.556308   431 raft_consensus.cc:2243] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.556874   431 raft_consensus.cc:2272] T 2865aa77674948499e8d5e89dd117de7 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.559651   431 tablet_replica.cc:333] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759: stopping tablet replica
I20260709 13:00:28.560491   431 raft_consensus.cc:2243] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.561560   431 raft_consensus.cc:2272] T a80f43d3599a4018921d0f12ea30d25f P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.564252   431 tablet_replica.cc:333] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759: stopping tablet replica
I20260709 13:00:28.565086   431 raft_consensus.cc:2243] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.565650   431 raft_consensus.cc:2272] T f32ed3eafdb743e0b5527a057250e80d P e4f3f8d22f334cfeb9dc7316c2771759 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.568410   431 tablet_replica.cc:333] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759: stopping tablet replica
I20260709 13:00:28.568912   431 raft_consensus.cc:2243] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.569335   431 raft_consensus.cc:2272] T ec5a7dee7d3b4b2cb847171ad1e80e98 P e4f3f8d22f334cfeb9dc7316c2771759 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.571239   431 tablet_replica.cc:333] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759: stopping tablet replica
I20260709 13:00:28.571830   431 raft_consensus.cc:2243] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.572253   431 raft_consensus.cc:2272] T 2b999efdb1de432fa180c502b4b2e0e2 P e4f3f8d22f334cfeb9dc7316c2771759 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.574672   431 tablet_replica.cc:333] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759: stopping tablet replica
I20260709 13:00:28.575304   431 raft_consensus.cc:2243] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.575726   431 raft_consensus.cc:2272] T acad783b77864a29a0bfcfdd201e12f6 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.577632   431 tablet_replica.cc:333] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759: stopping tablet replica
I20260709 13:00:28.578095   431 raft_consensus.cc:2243] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:28.578598   431 raft_consensus.cc:2272] T 725a4a0982b1482eacea62cdebb7cb3e P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.580345   431 tablet_replica.cc:333] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759: stopping tablet replica
I20260709 13:00:28.580802   431 raft_consensus.cc:2243] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.581545   431 raft_consensus.cc:2272] T 9f1df71364ad4b3dbdd84404e9bf9b44 P e4f3f8d22f334cfeb9dc7316c2771759 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.611039   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:00:28.628705   431 master.cc:562] Master@127.0.107.254:46723 shutting down...
I20260709 13:00:28.700093   519 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 13:00:28.792992   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:00:28.793527   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:28.793824   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P af712a29c564424d9045cda87c4aacb7: stopping tablet replica
I20260709 13:00:28.830904   431 master.cc:584] Master@127.0.107.254:46723 shutdown complete.
49c3856ee800410f95f6b9695bd926d6  4
75b161200bff4cc8a1789a7247d37fea  4
e4f3f8d22f334cfeb9dc7316c2771759  1
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (14055 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:00:28.875447   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:43429
I20260709 13:00:28.876793   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:28.883802   961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:28.883906   964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:28.884837   962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:28.885720   431 server_base.cc:1061] running on GCE node
I20260709 13:00:28.886619   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:28.886853   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:28.887209   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602028887072 us; error 0 us; skew 500 ppm
I20260709 13:00:28.887914   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:28.890618   431 webserver.cc:533] Webserver started at http://127.0.107.254:37905/ using document root <none> and password file <none>
I20260709 13:00:28.891495   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:28.891731   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:28.892045   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:28.893241   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/master-0-root/instance:
uuid: "0dfb1a14974243a78631cbd3c496bbe6"
format_stamp: "Formatted at 2026-07-09 13:00:28 on dist-test-slave-cq4t"
I20260709 13:00:28.898067   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:00:28.901917   969 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:28.902848   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:00:28.903301   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/master-0-root
uuid: "0dfb1a14974243a78631cbd3c496bbe6"
format_stamp: "Formatted at 2026-07-09 13:00:28 on dist-test-slave-cq4t"
I20260709 13:00:28.903681   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:28.925983   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:28.927402   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:28.970844   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:43429
I20260709 13:00:28.971067  1020 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:43429 every 8 connection(s)
I20260709 13:00:28.975001  1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:28.984704  1021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6: Bootstrap starting.
I20260709 13:00:28.989686  1021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:28.994547  1021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6: No bootstrap required, opened a new log
I20260709 13:00:28.996793  1021 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dfb1a14974243a78631cbd3c496bbe6" member_type: VOTER }
I20260709 13:00:28.997185  1021 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:28.997470  1021 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dfb1a14974243a78631cbd3c496bbe6, State: Initialized, Role: FOLLOWER
I20260709 13:00:28.998095  1021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dfb1a14974243a78631cbd3c496bbe6" member_type: VOTER }
I20260709 13:00:28.998562  1021 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:00:28.998767  1021 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:00:28.999068  1021 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:29.003360  1021 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dfb1a14974243a78631cbd3c496bbe6" member_type: VOTER }
I20260709 13:00:29.003926  1021 leader_election.cc:304] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dfb1a14974243a78631cbd3c496bbe6; no voters: 
I20260709 13:00:29.005141  1021 leader_election.cc:290] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:00:29.005669  1024 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:29.007206  1024 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [term 1 LEADER]: Becoming Leader. State: Replica: 0dfb1a14974243a78631cbd3c496bbe6, State: Running, Role: LEADER
I20260709 13:00:29.008034  1024 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dfb1a14974243a78631cbd3c496bbe6" member_type: VOTER }
I20260709 13:00:29.008440  1021 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:00:29.013367  1026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dfb1a14974243a78631cbd3c496bbe6. Latest consensus state: current_term: 1 leader_uuid: "0dfb1a14974243a78631cbd3c496bbe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dfb1a14974243a78631cbd3c496bbe6" member_type: VOTER } }
I20260709 13:00:29.013686  1025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dfb1a14974243a78631cbd3c496bbe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dfb1a14974243a78631cbd3c496bbe6" member_type: VOTER } }
I20260709 13:00:29.014400  1026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:29.014432  1025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:29.019692  1032 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:00:29.028455  1032 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:00:29.029532   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:00:29.038322  1032 catalog_manager.cc:1383] Generated new cluster ID: a81f1b092d0744789f5636a7e8715401
I20260709 13:00:29.038651  1032 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:00:29.095439  1032 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:00:29.096735  1032 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:00:29.109961  1032 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6: Generated new TSK 0
I20260709 13:00:29.110755  1032 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:00:29.160624   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:29.167626  1042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:29.168996  1043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:29.171603  1045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:29.171681   431 server_base.cc:1061] running on GCE node
I20260709 13:00:29.172922   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:29.173136   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:29.173344   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602029173321 us; error 0 us; skew 500 ppm
I20260709 13:00:29.173985   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:29.176841   431 webserver.cc:533] Webserver started at http://127.0.107.193:33965/ using document root <none> and password file <none>
I20260709 13:00:29.177392   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:29.177572   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:29.177909   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:29.179072   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-0-root/instance:
uuid: "a175f71f75a8459a9b6222a90dbd98bd"
format_stamp: "Formatted at 2026-07-09 13:00:29 on dist-test-slave-cq4t"
I20260709 13:00:29.184384   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:00:29.188037  1050 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.189050   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:00:29.189424   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-0-root
uuid: "a175f71f75a8459a9b6222a90dbd98bd"
format_stamp: "Formatted at 2026-07-09 13:00:29 on dist-test-slave-cq4t"
I20260709 13:00:29.189721   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:29.201393   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:29.202777   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:29.204439   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:29.207065   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:29.207278   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.207592   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:29.207754   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.253773   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:46021
I20260709 13:00:29.253968  1112 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:46021 every 8 connection(s)
I20260709 13:00:29.259130   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:29.267115  1117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:29.267573  1118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:29.273928   431 server_base.cc:1061] running on GCE node
W20260709 13:00:29.274603  1120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:29.275802  1113 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43429
I20260709 13:00:29.275857   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:29.276244   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:29.276315  1113 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:29.276455   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602029276438 us; error 0 us; skew 500 ppm
I20260709 13:00:29.277145   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:29.277208  1113 heartbeater.cc:507] Master 127.0.107.254:43429 requested a full tablet report, sending...
I20260709 13:00:29.279542   986 ts_manager.cc:194] Registered new tserver with Master: a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021)
I20260709 13:00:29.280627   431 webserver.cc:533] Webserver started at http://127.0.107.194:40621/ using document root <none> and password file <none>
I20260709 13:00:29.281215   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:29.281458   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:29.281458   986 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46494
I20260709 13:00:29.282006   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:29.283401   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-1-root/instance:
uuid: "b26d86b24f1c4b45b1535569d9afba25"
format_stamp: "Formatted at 2026-07-09 13:00:29 on dist-test-slave-cq4t"
I20260709 13:00:29.288328   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:00:29.292399  1125 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.293416   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:00:29.293795   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-1-root
uuid: "b26d86b24f1c4b45b1535569d9afba25"
format_stamp: "Formatted at 2026-07-09 13:00:29 on dist-test-slave-cq4t"
I20260709 13:00:29.294219   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:29.306756   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:29.308248   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:29.309995   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:29.312538   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:29.312808   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.313107   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:29.313305   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.359359   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:33499
I20260709 13:00:29.359419  1187 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:33499 every 8 connection(s)
I20260709 13:00:29.364671   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:29.373312  1191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:29.375074  1192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:29.379498   431 server_base.cc:1061] running on GCE node
I20260709 13:00:29.379442  1188 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43429
W20260709 13:00:29.379717  1194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:29.380129  1188 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:29.381235  1188 heartbeater.cc:507] Master 127.0.107.254:43429 requested a full tablet report, sending...
I20260709 13:00:29.382870   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:29.383168   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:29.383320   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602029383308 us; error 0 us; skew 500 ppm
I20260709 13:00:29.383919   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:29.383697   986 ts_manager.cc:194] Registered new tserver with Master: b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499)
I20260709 13:00:29.385270   986 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46496
I20260709 13:00:29.386890   431 webserver.cc:533] Webserver started at http://127.0.107.195:41109/ using document root <none> and password file <none>
I20260709 13:00:29.387591   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:29.387844   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:29.388092   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:29.389158   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-2-root/instance:
uuid: "96b84c522d8744e8833c0d47ed810b9d"
format_stamp: "Formatted at 2026-07-09 13:00:29 on dist-test-slave-cq4t"
I20260709 13:00:29.393642   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.006s
I20260709 13:00:29.397434  1199 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.398444   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:00:29.398789   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-2-root
uuid: "96b84c522d8744e8833c0d47ed810b9d"
format_stamp: "Formatted at 2026-07-09 13:00:29 on dist-test-slave-cq4t"
I20260709 13:00:29.399120   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602014632082-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:29.415748   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:29.416981   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:29.418567   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:29.421139   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:29.421339   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.421630   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:29.421787   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:29.466320   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:38145
I20260709 13:00:29.466426  1261 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:38145 every 8 connection(s)
I20260709 13:00:29.482410  1262 heartbeater.cc:344] Connected to a master server at 127.0.107.254:43429
I20260709 13:00:29.482853  1262 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:29.483777  1262 heartbeater.cc:507] Master 127.0.107.254:43429 requested a full tablet report, sending...
I20260709 13:00:29.486105   986 ts_manager.cc:194] Registered new tserver with Master: 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:29.486398   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016339926s
I20260709 13:00:29.487560   986 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46498
I20260709 13:00:29.515460   986 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46504:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:00:29.518477   986 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:00:29.592941  1152 tablet_service.cc:1511] Processing CreateTablet for tablet 580e548b77894ff5a453639555207804 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:00:29.594559  1152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 580e548b77894ff5a453639555207804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.596268  1153 tablet_service.cc:1511] Processing CreateTablet for tablet 51814b87cc5f4a4fa6d60f0a14241261 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:00:29.598217  1151 tablet_service.cc:1511] Processing CreateTablet for tablet d6f62839830a4b37b0d53590fa702935 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:00:29.602164  1227 tablet_service.cc:1511] Processing CreateTablet for tablet 51814b87cc5f4a4fa6d60f0a14241261 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:00:29.604238  1226 tablet_service.cc:1511] Processing CreateTablet for tablet 580e548b77894ff5a453639555207804 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:00:29.605808  1226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 580e548b77894ff5a453639555207804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.608309  1225 tablet_service.cc:1511] Processing CreateTablet for tablet d6f62839830a4b37b0d53590fa702935 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:00:29.609937  1225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f62839830a4b37b0d53590fa702935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.610085  1223 tablet_service.cc:1511] Processing CreateTablet for tablet fdcf061d7a3a40119e65674f41587517 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:00:29.608304  1224 tablet_service.cc:1511] Processing CreateTablet for tablet aac0c1b05d20415bb64a5b8de8579483 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:00:29.611644  1223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdcf061d7a3a40119e65674f41587517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.612481  1224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aac0c1b05d20415bb64a5b8de8579483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.616817  1222 tablet_service.cc:1511] Processing CreateTablet for tablet 92e9932b04aa48418d0db4088c1a76d0 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:00:29.618333  1222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92e9932b04aa48418d0db4088c1a76d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.620062  1150 tablet_service.cc:1511] Processing CreateTablet for tablet aac0c1b05d20415bb64a5b8de8579483 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:00:29.620479  1153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51814b87cc5f4a4fa6d60f0a14241261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.621575  1151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f62839830a4b37b0d53590fa702935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.624409  1148 tablet_service.cc:1511] Processing CreateTablet for tablet 92e9932b04aa48418d0db4088c1a76d0 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:00:29.625886  1148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92e9932b04aa48418d0db4088c1a76d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.622588  1149 tablet_service.cc:1511] Processing CreateTablet for tablet fdcf061d7a3a40119e65674f41587517 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:00:29.626932  1147 tablet_service.cc:1511] Processing CreateTablet for tablet 48471c61e2f84d0d9398f9b5b2d71a99 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:00:29.633504  1221 tablet_service.cc:1511] Processing CreateTablet for tablet 48471c61e2f84d0d9398f9b5b2d71a99 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:00:29.635476  1147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48471c61e2f84d0d9398f9b5b2d71a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.637991  1221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48471c61e2f84d0d9398f9b5b2d71a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.638963  1227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51814b87cc5f4a4fa6d60f0a14241261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.639566  1149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdcf061d7a3a40119e65674f41587517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.639330  1220 tablet_service.cc:1511] Processing CreateTablet for tablet f92324f9111f4363bcd60d1fc1fc1fd4 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:00:29.640913  1220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f92324f9111f4363bcd60d1fc1fc1fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.641398  1150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aac0c1b05d20415bb64a5b8de8579483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.639577  1146 tablet_service.cc:1511] Processing CreateTablet for tablet f92324f9111f4363bcd60d1fc1fc1fd4 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:00:29.642869  1146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f92324f9111f4363bcd60d1fc1fc1fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.673557  1077 tablet_service.cc:1511] Processing CreateTablet for tablet 580e548b77894ff5a453639555207804 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:00:29.675477  1077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 580e548b77894ff5a453639555207804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.675164  1078 tablet_service.cc:1511] Processing CreateTablet for tablet 51814b87cc5f4a4fa6d60f0a14241261 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:00:29.676904  1078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51814b87cc5f4a4fa6d60f0a14241261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.673623  1074 tablet_service.cc:1511] Processing CreateTablet for tablet fdcf061d7a3a40119e65674f41587517 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:00:29.678359  1074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdcf061d7a3a40119e65674f41587517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.673549  1219 tablet_service.cc:1511] Processing CreateTablet for tablet db84a39f3b824926adcd9d990bbcc2b6 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:00:29.679692  1145 tablet_service.cc:1511] Processing CreateTablet for tablet db84a39f3b824926adcd9d990bbcc2b6 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:00:29.680449  1219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db84a39f3b824926adcd9d990bbcc2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.681131  1145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db84a39f3b824926adcd9d990bbcc2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.673830  1076 tablet_service.cc:1511] Processing CreateTablet for tablet d6f62839830a4b37b0d53590fa702935 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:00:29.683885  1076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f62839830a4b37b0d53590fa702935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.683732  1070 tablet_service.cc:1511] Processing CreateTablet for tablet db84a39f3b824926adcd9d990bbcc2b6 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:00:29.685618  1071 tablet_service.cc:1511] Processing CreateTablet for tablet f92324f9111f4363bcd60d1fc1fc1fd4 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:00:29.691406  1075 tablet_service.cc:1511] Processing CreateTablet for tablet aac0c1b05d20415bb64a5b8de8579483 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:00:29.691766  1073 tablet_service.cc:1511] Processing CreateTablet for tablet 92e9932b04aa48418d0db4088c1a76d0 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:00:29.691896  1072 tablet_service.cc:1511] Processing CreateTablet for tablet 48471c61e2f84d0d9398f9b5b2d71a99 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:00:29.693657  1277 tablet_bootstrap.cc:492] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d: Bootstrap starting.
I20260709 13:00:29.687196  1071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f92324f9111f4363bcd60d1fc1fc1fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.696463  1072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48471c61e2f84d0d9398f9b5b2d71a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.697659  1069 tablet_service.cc:1511] Processing CreateTablet for tablet 17d075eea46e470481e6bd91b9c78da5 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:00:29.697980  1144 tablet_service.cc:1511] Processing CreateTablet for tablet 17d075eea46e470481e6bd91b9c78da5 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:00:29.699366  1069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17d075eea46e470481e6bd91b9c78da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.700709  1075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aac0c1b05d20415bb64a5b8de8579483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.699371  1144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17d075eea46e470481e6bd91b9c78da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.703248  1073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92e9932b04aa48418d0db4088c1a76d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.704593  1070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db84a39f3b824926adcd9d990bbcc2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.728624  1223 tablet_service.cc:1511] Processing CreateTablet for tablet 17d075eea46e470481e6bd91b9c78da5 (DEFAULT_TABLE table=test-workload [id=4f308e1b335948d5bac5e5a982314f92]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:00:29.730021  1223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17d075eea46e470481e6bd91b9c78da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:29.731698  1277 tablet_bootstrap.cc:654] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:29.804512  1277 tablet_bootstrap.cc:492] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d: No bootstrap required, opened a new log
I20260709 13:00:29.805015  1277 ts_tablet_manager.cc:1403] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d: Time spent bootstrapping tablet: real 0.112s	user 0.021s	sys 0.020s
I20260709 13:00:29.808041  1277 raft_consensus.cc:359] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.808781  1277 raft_consensus.cc:385] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:29.809147  1277 raft_consensus.cc:740] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Initialized, Role: FOLLOWER
I20260709 13:00:29.810174  1277 consensus_queue.cc:260] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.837529  1281 tablet_bootstrap.cc:492] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd: Bootstrap starting.
I20260709 13:00:29.846365  1279 tablet_bootstrap.cc:492] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25: Bootstrap starting.
I20260709 13:00:29.846567  1262 heartbeater.cc:499] Master 127.0.107.254:43429 was elected leader, sending a full tablet report...
I20260709 13:00:29.847157  1277 ts_tablet_manager.cc:1434] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d: Time spent starting tablet: real 0.042s	user 0.014s	sys 0.025s
I20260709 13:00:29.848171  1277 tablet_bootstrap.cc:492] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d: Bootstrap starting.
I20260709 13:00:29.848457  1281 tablet_bootstrap.cc:654] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:29.854297  1279 tablet_bootstrap.cc:654] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:29.854868  1277 tablet_bootstrap.cc:654] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:29.862460  1281 tablet_bootstrap.cc:492] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd: No bootstrap required, opened a new log
I20260709 13:00:29.862959  1281 ts_tablet_manager.cc:1403] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.004s
I20260709 13:00:29.865885  1281 raft_consensus.cc:359] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.866668  1281 raft_consensus.cc:385] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:29.866990  1281 raft_consensus.cc:740] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Initialized, Role: FOLLOWER
I20260709 13:00:29.867972  1281 consensus_queue.cc:260] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.869998  1279 tablet_bootstrap.cc:492] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25: No bootstrap required, opened a new log
I20260709 13:00:29.870523  1279 ts_tablet_manager.cc:1403] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.000s
I20260709 13:00:29.871032  1113 heartbeater.cc:499] Master 127.0.107.254:43429 was elected leader, sending a full tablet report...
I20260709 13:00:29.871917  1281 ts_tablet_manager.cc:1434] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:00:29.873090  1281 tablet_bootstrap.cc:492] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd: Bootstrap starting.
I20260709 13:00:29.873814  1279 raft_consensus.cc:359] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.874574  1279 raft_consensus.cc:385] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:29.874953  1277 tablet_bootstrap.cc:492] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d: No bootstrap required, opened a new log
I20260709 13:00:29.875531  1277 ts_tablet_manager.cc:1403] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d: Time spent bootstrapping tablet: real 0.028s	user 0.004s	sys 0.010s
I20260709 13:00:29.878875  1277 raft_consensus.cc:359] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.879231  1280 raft_consensus.cc:493] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.874852  1279 raft_consensus.cc:740] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Initialized, Role: FOLLOWER
I20260709 13:00:29.879818  1280 raft_consensus.cc:515] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.880718  1279 consensus_queue.cc:260] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.884948  1280 leader_election.cc:290] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499)
I20260709 13:00:29.886235  1281 tablet_bootstrap.cc:654] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:29.888806  1277 raft_consensus.cc:385] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:29.889217  1277 raft_consensus.cc:740] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Initialized, Role: FOLLOWER
I20260709 13:00:29.890254  1277 consensus_queue.cc:260] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.903360  1277 ts_tablet_manager.cc:1434] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.004s
I20260709 13:00:29.904775  1277 tablet_bootstrap.cc:492] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d: Bootstrap starting.
I20260709 13:00:29.913427  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdcf061d7a3a40119e65674f41587517" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd" is_pre_election: true
W20260709 13:00:29.915776  1200 leader_election.cc:343] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:29.928408  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdcf061d7a3a40119e65674f41587517" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25" is_pre_election: true
I20260709 13:00:29.930042  1277 tablet_bootstrap.cc:654] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:29.930163  1200 leader_election.cc:343] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:29.930624  1200 leader_election.cc:304] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d; no voters: a175f71f75a8459a9b6222a90dbd98bd, b26d86b24f1c4b45b1535569d9afba25
I20260709 13:00:29.931474  1280 raft_consensus.cc:2749] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:29.937601  1281 tablet_bootstrap.cc:492] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd: No bootstrap required, opened a new log
I20260709 13:00:29.938323  1281 ts_tablet_manager.cc:1403] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd: Time spent bootstrapping tablet: real 0.065s	user 0.030s	sys 0.021s
I20260709 13:00:29.938235  1285 raft_consensus.cc:493] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.939147  1285 raft_consensus.cc:515] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.942565  1281 raft_consensus.cc:359] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.943955  1188 heartbeater.cc:499] Master 127.0.107.254:43429 was elected leader, sending a full tablet report...
I20260709 13:00:29.944319  1281 raft_consensus.cc:385] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:29.944576  1279 ts_tablet_manager.cc:1434] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25: Time spent starting tablet: real 0.074s	user 0.034s	sys 0.035s
I20260709 13:00:29.944670  1281 raft_consensus.cc:740] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Initialized, Role: FOLLOWER
I20260709 13:00:29.945964  1279 tablet_bootstrap.cc:492] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25: Bootstrap starting.
I20260709 13:00:29.945540  1281 consensus_queue.cc:260] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.951349  1285 leader_election.cc:290] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:29.958062  1280 raft_consensus.cc:493] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.958630  1280 raft_consensus.cc:515] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.960852  1280 leader_election.cc:290] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499)
I20260709 13:00:29.963253  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580e548b77894ff5a453639555207804" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd" is_pre_election: true
I20260709 13:00:29.964600  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580e548b77894ff5a453639555207804" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25" is_pre_election: true
W20260709 13:00:29.966805  1200 leader_election.cc:343] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:29.967563  1200 leader_election.cc:343] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:29.968166  1279 tablet_bootstrap.cc:654] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:29.968138  1200 leader_election.cc:304] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d; no voters: a175f71f75a8459a9b6222a90dbd98bd, b26d86b24f1c4b45b1535569d9afba25
I20260709 13:00:29.970189  1280 raft_consensus.cc:2749] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:29.975955  1277 tablet_bootstrap.cc:492] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d: No bootstrap required, opened a new log
I20260709 13:00:29.976532  1277 ts_tablet_manager.cc:1403] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d: Time spent bootstrapping tablet: real 0.072s	user 0.007s	sys 0.011s
I20260709 13:00:29.979511  1277 raft_consensus.cc:359] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.980518  1277 raft_consensus.cc:385] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:29.980839  1277 raft_consensus.cc:740] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Initialized, Role: FOLLOWER
I20260709 13:00:29.981575  1277 consensus_queue.cc:260] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:29.983211  1281 ts_tablet_manager.cc:1434] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd: Time spent starting tablet: real 0.044s	user 0.016s	sys 0.020s
I20260709 13:00:29.983847  1277 ts_tablet_manager.cc:1434] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:29.984237  1281 tablet_bootstrap.cc:492] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd: Bootstrap starting.
I20260709 13:00:29.984833  1277 tablet_bootstrap.cc:492] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d: Bootstrap starting.
I20260709 13:00:29.992238  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f62839830a4b37b0d53590fa702935" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25" is_pre_election: true
I20260709 13:00:29.992784  1277 tablet_bootstrap.cc:654] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:29.993870  1051 leader_election.cc:343] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:29.997548  1279 tablet_bootstrap.cc:492] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25: No bootstrap required, opened a new log
I20260709 13:00:29.998076  1279 ts_tablet_manager.cc:1403] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25: Time spent bootstrapping tablet: real 0.052s	user 0.005s	sys 0.009s
I20260709 13:00:30.002053  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f62839830a4b37b0d53590fa702935" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d" is_pre_election: true
I20260709 13:00:30.001209  1279 raft_consensus.cc:359] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:30.003593  1279 raft_consensus.cc:385] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.003731  1281 tablet_bootstrap.cc:654] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.003952  1279 raft_consensus.cc:740] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Initialized, Role: FOLLOWER
W20260709 13:00:30.003964  1052 leader_election.cc:343] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:30.004556  1052 leader_election.cc:304] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a175f71f75a8459a9b6222a90dbd98bd; no voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25
I20260709 13:00:30.004904  1279 consensus_queue.cc:260] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:30.005772  1291 raft_consensus.cc:2749] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:30.008103  1279 ts_tablet_manager.cc:1434] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 13:00:30.009086  1279 tablet_bootstrap.cc:492] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25: Bootstrap starting.
I20260709 13:00:30.009840  1281 tablet_bootstrap.cc:492] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd: No bootstrap required, opened a new log
I20260709 13:00:30.010262  1281 ts_tablet_manager.cc:1403] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.004s
I20260709 13:00:30.012845  1277 tablet_bootstrap.cc:492] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d: No bootstrap required, opened a new log
I20260709 13:00:30.013391  1277 ts_tablet_manager.cc:1403] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.004s
I20260709 13:00:30.012976  1281 raft_consensus.cc:359] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.013778  1281 raft_consensus.cc:385] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.014099  1281 raft_consensus.cc:740] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.014823  1281 consensus_queue.cc:260] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.015668  1279 tablet_bootstrap.cc:654] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.017050  1281 ts_tablet_manager.cc:1434] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:00:30.017978  1281 tablet_bootstrap.cc:492] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd: Bootstrap starting.
W20260709 13:00:30.021059  1034 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:30.021659  1277 raft_consensus.cc:359] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.022544  1277 raft_consensus.cc:385] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.022850  1277 raft_consensus.cc:740] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.023757  1277 consensus_queue.cc:260] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.025029  1281 tablet_bootstrap.cc:654] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.026091  1277 ts_tablet_manager.cc:1434] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.000s
I20260709 13:00:30.027148  1277 tablet_bootstrap.cc:492] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d: Bootstrap starting.
I20260709 13:00:30.035951  1277 tablet_bootstrap.cc:654] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.039809  1286 raft_consensus.cc:493] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.040477  1286 raft_consensus.cc:515] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:30.043205  1279 tablet_bootstrap.cc:492] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25: No bootstrap required, opened a new log
I20260709 13:00:30.043730  1279 ts_tablet_manager.cc:1403] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.007s
I20260709 13:00:30.045253  1277 tablet_bootstrap.cc:492] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d: No bootstrap required, opened a new log
I20260709 13:00:30.045751  1277 ts_tablet_manager.cc:1403] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.000s
I20260709 13:00:30.046819  1279 raft_consensus.cc:359] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.047657  1279 raft_consensus.cc:385] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.047963  1279 raft_consensus.cc:740] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.048599  1277 raft_consensus.cc:359] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.048695  1279 consensus_queue.cc:260] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.049448  1277 raft_consensus.cc:385] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.049546  1286 leader_election.cc:290] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145), a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021)
I20260709 13:00:30.049765  1277 raft_consensus.cc:740] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.050519  1277 consensus_queue.cc:260] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.052667  1277 ts_tablet_manager.cc:1434] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:00:30.053571  1277 tablet_bootstrap.cc:492] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d: Bootstrap starting.
I20260709 13:00:30.056516  1281 tablet_bootstrap.cc:492] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd: No bootstrap required, opened a new log
I20260709 13:00:30.056984  1281 ts_tablet_manager.cc:1403] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd: Time spent bootstrapping tablet: real 0.039s	user 0.007s	sys 0.015s
I20260709 13:00:30.059922  1281 raft_consensus.cc:359] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.060668  1281 raft_consensus.cc:385] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.060989  1281 raft_consensus.cc:740] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.061708  1281 consensus_queue.cc:260] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.064699  1281 ts_tablet_manager.cc:1434] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:00:30.066154  1281 tablet_bootstrap.cc:492] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd: Bootstrap starting.
I20260709 13:00:30.066896  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d" is_pre_election: true
W20260709 13:00:30.069409  1127 leader_election.cc:343] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:30.072546  1281 tablet_bootstrap.cc:654] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.080017  1277 tablet_bootstrap.cc:654] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.083750  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd" is_pre_election: true
I20260709 13:00:30.085430  1281 tablet_bootstrap.cc:492] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd: No bootstrap required, opened a new log
I20260709 13:00:30.085896  1281 ts_tablet_manager.cc:1403] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.003s
W20260709 13:00:30.085758  1126 leader_election.cc:343] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:30.086488  1126 leader_election.cc:304] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b26d86b24f1c4b45b1535569d9afba25; no voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:30.087556  1286 raft_consensus.cc:2749] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:30.089360  1281 raft_consensus.cc:359] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.089993  1281 raft_consensus.cc:385] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.090243  1281 raft_consensus.cc:740] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.092015  1279 ts_tablet_manager.cc:1434] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25: Time spent starting tablet: real 0.048s	user 0.025s	sys 0.020s
I20260709 13:00:30.091653  1281 consensus_queue.cc:260] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.092217  1277 tablet_bootstrap.cc:492] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d: No bootstrap required, opened a new log
I20260709 13:00:30.092686  1277 ts_tablet_manager.cc:1403] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.000s
I20260709 13:00:30.092960  1279 tablet_bootstrap.cc:492] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25: Bootstrap starting.
I20260709 13:00:30.095608  1277 raft_consensus.cc:359] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.096320  1277 raft_consensus.cc:385] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.096367  1291 raft_consensus.cc:493] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.096733  1277 raft_consensus.cc:740] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.096881  1291 raft_consensus.cc:515] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.097499  1277 consensus_queue.cc:260] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.099138  1291 leader_election.cc:290] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.099491  1279 tablet_bootstrap.cc:654] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.099875  1277 ts_tablet_manager.cc:1434] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:30.101665  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db84a39f3b824926adcd9d990bbcc2b6" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25" is_pre_election: true
I20260709 13:00:30.102171  1277 tablet_bootstrap.cc:492] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d: Bootstrap starting.
W20260709 13:00:30.104409  1051 leader_election.cc:343] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:30.101598  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db84a39f3b824926adcd9d990bbcc2b6" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d" is_pre_election: true
I20260709 13:00:30.106344  1237 raft_consensus.cc:2468] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 0.
I20260709 13:00:30.107627  1052 leader_election.cc:304] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: b26d86b24f1c4b45b1535569d9afba25
I20260709 13:00:30.108860  1277 tablet_bootstrap.cc:654] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.108747  1291 raft_consensus.cc:2804] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:30.109368  1291 raft_consensus.cc:493] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.109746  1291 raft_consensus.cc:3060] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.111342  1281 ts_tablet_manager.cc:1434] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:00:30.112972  1281 tablet_bootstrap.cc:492] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd: Bootstrap starting.
I20260709 13:00:30.116201  1279 tablet_bootstrap.cc:492] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25: No bootstrap required, opened a new log
I20260709 13:00:30.116761  1279 ts_tablet_manager.cc:1403] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.000s
I20260709 13:00:30.119632  1281 tablet_bootstrap.cc:654] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.119740  1279 raft_consensus.cc:359] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.120613  1279 raft_consensus.cc:385] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.120985  1279 raft_consensus.cc:740] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.121917  1279 consensus_queue.cc:260] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.125905  1285 raft_consensus.cc:493] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.126590  1285 raft_consensus.cc:515] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.131942  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17d075eea46e470481e6bd91b9c78da5" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25" is_pre_election: true
I20260709 13:00:30.132843  1163 raft_consensus.cc:2468] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 0.
I20260709 13:00:30.127687  1291 raft_consensus.cc:515] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.134588  1051 leader_election.cc:304] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a175f71f75a8459a9b6222a90dbd98bd, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:30.138127  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17d075eea46e470481e6bd91b9c78da5" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d" is_pre_election: true
W20260709 13:00:30.140074  1052 leader_election.cc:343] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:30.141132  1301 raft_consensus.cc:493] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.141767  1301 raft_consensus.cc:515] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.145531  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdcf061d7a3a40119e65674f41587517" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd" is_pre_election: true
I20260709 13:00:30.147055  1291 leader_election.cc:290] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 election: Requested vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
W20260709 13:00:30.147301  1126 leader_election.cc:343] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:30.147898  1291 raft_consensus.cc:2804] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:30.148373  1291 raft_consensus.cc:493] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.148806  1291 raft_consensus.cc:3060] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.149022  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db84a39f3b824926adcd9d990bbcc2b6" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:30.149824  1237 raft_consensus.cc:3060] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.153172  1236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdcf061d7a3a40119e65674f41587517" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d" is_pre_election: true
I20260709 13:00:30.153661  1277 tablet_bootstrap.cc:492] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d: No bootstrap required, opened a new log
I20260709 13:00:30.153893  1236 raft_consensus.cc:2468] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 0.
I20260709 13:00:30.154135  1277 ts_tablet_manager.cc:1403] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d: Time spent bootstrapping tablet: real 0.052s	user 0.018s	sys 0.032s
I20260709 13:00:30.155020  1285 leader_election.cc:290] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.155520  1301 leader_election.cc:290] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.155375  1127 leader_election.cc:304] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:30.156855  1301 raft_consensus.cc:2804] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:30.157265  1301 raft_consensus.cc:493] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.157117  1277 raft_consensus.cc:359] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.157756  1301 raft_consensus.cc:3060] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.157910  1277 raft_consensus.cc:385] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.158361  1277 raft_consensus.cc:740] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.158582  1291 raft_consensus.cc:515] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.159355  1277 consensus_queue.cc:260] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.161123  1291 leader_election.cc:290] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 election: Requested vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.161810  1236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17d075eea46e470481e6bd91b9c78da5" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:30.163633  1277 ts_tablet_manager.cc:1434] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
W20260709 13:00:30.163560  1052 leader_election.cc:343] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:30.164932  1277 tablet_bootstrap.cc:492] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d: Bootstrap starting.
I20260709 13:00:30.166846  1279 ts_tablet_manager.cc:1434] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 13:00:30.167289  1281 tablet_bootstrap.cc:492] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd: No bootstrap required, opened a new log
I20260709 13:00:30.168012  1281 ts_tablet_manager.cc:1403] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd: Time spent bootstrapping tablet: real 0.055s	user 0.013s	sys 0.028s
I20260709 13:00:30.169170  1237 raft_consensus.cc:2468] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 1.
I20260709 13:00:30.170769  1052 leader_election.cc:304] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: 
I20260709 13:00:30.171571  1277 tablet_bootstrap.cc:654] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.171854  1279 tablet_bootstrap.cc:492] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25: Bootstrap starting.
I20260709 13:00:30.172322  1301 raft_consensus.cc:515] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.175727  1291 raft_consensus.cc:2804] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:30.176430  1291 raft_consensus.cc:697] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 1 LEADER]: Becoming Leader. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:30.177536  1291 consensus_queue.cc:237] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.174396  1281 raft_consensus.cc:359] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.180271  1281 raft_consensus.cc:385] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.180537  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdcf061d7a3a40119e65674f41587517" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:30.181093  1301 leader_election.cc:290] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 election: Requested vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.181167  1237 raft_consensus.cc:3060] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.180722  1281 raft_consensus.cc:740] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.182942  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdcf061d7a3a40119e65674f41587517" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:30.182771  1281 consensus_queue.cc:260] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.184453  1277 tablet_bootstrap.cc:492] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d: No bootstrap required, opened a new log
W20260709 13:00:30.184398  1126 leader_election.cc:343] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:30.185063  1277 ts_tablet_manager.cc:1403] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.010s
I20260709 13:00:30.185675  1281 ts_tablet_manager.cc:1434] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:00:30.187044  1281 tablet_bootstrap.cc:492] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd: Bootstrap starting.
W20260709 13:00:30.188282  1051 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:33499, user_credentials={real_user=slave}} blocked reactor thread for 53849.7us
I20260709 13:00:30.188340  1277 raft_consensus.cc:359] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.189204  1277 raft_consensus.cc:385] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.189539  1277 raft_consensus.cc:740] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.190119  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db84a39f3b824926adcd9d990bbcc2b6" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:30.190442  1277 consensus_queue.cc:260] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.192674  1277 ts_tablet_manager.cc:1434] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:00:30.192631  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17d075eea46e470481e6bd91b9c78da5" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:30.201435  1163 raft_consensus.cc:3060] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.202134  1277 tablet_bootstrap.cc:492] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d: Bootstrap starting.
I20260709 13:00:30.205860   985 catalog_manager.cc:5697] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd reported cstate change: term changed from 0 to 1, leader changed from <none> to a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.208846  1163 raft_consensus.cc:2468] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 1.
I20260709 13:00:30.209244  1277 tablet_bootstrap.cc:654] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:30.193598  1051 leader_election.cc:343] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:30.210654  1051 leader_election.cc:304] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a175f71f75a8459a9b6222a90dbd98bd, b26d86b24f1c4b45b1535569d9afba25; no voters: 96b84c522d8744e8833c0d47ed810b9d
I20260709 13:00:30.211539  1291 raft_consensus.cc:2804] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:30.212239  1291 raft_consensus.cc:697] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 1 LEADER]: Becoming Leader. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:30.199249  1279 tablet_bootstrap.cc:654] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.213096  1291 consensus_queue.cc:237] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.219098  1281 tablet_bootstrap.cc:654] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.221714  1237 raft_consensus.cc:2468] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 1.
I20260709 13:00:30.226490  1277 tablet_bootstrap.cc:492] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d: No bootstrap required, opened a new log
I20260709 13:00:30.227051  1277 ts_tablet_manager.cc:1403] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.006s
I20260709 13:00:30.226655  1127 leader_election.cc:304] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:30.227950  1301 raft_consensus.cc:2804] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:30.228475  1301 raft_consensus.cc:697] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Becoming Leader. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:30.229344  1301 consensus_queue.cc:237] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.229777  1279 tablet_bootstrap.cc:492] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25: No bootstrap required, opened a new log
I20260709 13:00:30.229835  1277 raft_consensus.cc:359] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.232852  1279 ts_tablet_manager.cc:1403] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25: Time spent bootstrapping tablet: real 0.061s	user 0.010s	sys 0.012s
I20260709 13:00:30.233055  1277 raft_consensus.cc:385] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.233353  1277 raft_consensus.cc:740] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.234033  1277 consensus_queue.cc:260] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.235994  1277 ts_tablet_manager.cc:1434] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:00:30.236416  1279 raft_consensus.cc:359] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.236974  1280 raft_consensus.cc:493] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.237111  1279 raft_consensus.cc:385] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.237632  1279 raft_consensus.cc:740] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.237808  1280 raft_consensus.cc:515] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.238377  1279 consensus_queue.cc:260] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.240082  1277 tablet_bootstrap.cc:492] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d: Bootstrap starting.
I20260709 13:00:30.240521  1279 ts_tablet_manager.cc:1434] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:00:30.241472  1279 tablet_bootstrap.cc:492] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25: Bootstrap starting.
I20260709 13:00:30.244827  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e9932b04aa48418d0db4088c1a76d0" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd" is_pre_election: true
I20260709 13:00:30.245266  1280 leader_election.cc:290] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499)
I20260709 13:00:30.245555  1088 raft_consensus.cc:2468] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 0.
I20260709 13:00:30.246585  1277 tablet_bootstrap.cc:654] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.247160  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e9932b04aa48418d0db4088c1a76d0" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25" is_pre_election: true
I20260709 13:00:30.247843  1163 raft_consensus.cc:2468] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 0.
I20260709 13:00:30.249526  1200 leader_election.cc:304] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: 
I20260709 13:00:30.251288  1280 raft_consensus.cc:2804] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:30.251766  1280 raft_consensus.cc:493] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.252198  1280 raft_consensus.cc:3060] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.258391  1277 tablet_bootstrap.cc:492] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d: No bootstrap required, opened a new log
I20260709 13:00:30.258988  1277 ts_tablet_manager.cc:1403] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:00:30.260113  1280 raft_consensus.cc:515] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.261650  1277 raft_consensus.cc:359] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:30.262492  1277 raft_consensus.cc:385] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.263079  1286 raft_consensus.cc:493] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.263597  1286 raft_consensus.cc:515] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.265801  1286 leader_election.cc:290] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.263005  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e9932b04aa48418d0db4088c1a76d0" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:30.266639  1087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48471c61e2f84d0d9398f9b5b2d71a99" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd" is_pre_election: true
I20260709 13:00:30.266937  1088 raft_consensus.cc:3060] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.267357  1087 raft_consensus.cc:2468] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 0.
I20260709 13:00:30.267588  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48471c61e2f84d0d9398f9b5b2d71a99" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d" is_pre_election: true
I20260709 13:00:30.268513  1237 raft_consensus.cc:2468] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 0.
I20260709 13:00:30.268707  1126 leader_election.cc:304] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a175f71f75a8459a9b6222a90dbd98bd, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:30.269620  1286 raft_consensus.cc:2804] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:30.269989  1286 raft_consensus.cc:493] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.270341  1286 raft_consensus.cc:3060] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.274679  1088 raft_consensus.cc:2468] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 1.
I20260709 13:00:30.276396  1200 leader_election.cc:304] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: 
I20260709 13:00:30.277555  1286 raft_consensus.cc:515] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.262831  1277 raft_consensus.cc:740] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.280193  1277 consensus_queue.cc:260] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:30.281028  1279 tablet_bootstrap.cc:654] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.283995  1281 tablet_bootstrap.cc:492] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd: No bootstrap required, opened a new log
I20260709 13:00:30.284730  1281 ts_tablet_manager.cc:1403] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd: Time spent bootstrapping tablet: real 0.098s	user 0.013s	sys 0.005s
I20260709 13:00:30.284109  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48471c61e2f84d0d9398f9b5b2d71a99" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:30.287111  1088 raft_consensus.cc:3060] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.289773  1286 leader_election.cc:290] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 election: Requested vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.289950  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48471c61e2f84d0d9398f9b5b2d71a99" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:30.290576  1237 raft_consensus.cc:3060] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.292590  1277 ts_tablet_manager.cc:1434] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:00:30.291488  1320 raft_consensus.cc:2804] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:30.287976  1280 leader_election.cc:290] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 election: Requested vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499)
I20260709 13:00:30.294952  1279 tablet_bootstrap.cc:492] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25: No bootstrap required, opened a new log
I20260709 13:00:30.295938  1279 ts_tablet_manager.cc:1403] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.005s
I20260709 13:00:30.304445  1237 raft_consensus.cc:2468] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 1.
I20260709 13:00:30.305780  1127 leader_election.cc:304] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:30.305696   983 catalog_manager.cc:5697] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 reported cstate change: term changed from 0 to 1, leader changed from <none> to b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "b26d86b24f1c4b45b1535569d9afba25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.306741  1281 raft_consensus.cc:359] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.307523  1281 raft_consensus.cc:385] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.307652  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e9932b04aa48418d0db4088c1a76d0" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:30.308301  1163 raft_consensus.cc:3060] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.309099  1279 raft_consensus.cc:359] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.309805  1279 raft_consensus.cc:385] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.310088  1279 raft_consensus.cc:740] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.311054  1320 raft_consensus.cc:697] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 1 LEADER]: Becoming Leader. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Running, Role: LEADER
I20260709 13:00:30.310909  1279 consensus_queue.cc:260] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.311954  1286 raft_consensus.cc:2804] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:30.311995  1320 consensus_queue.cc:237] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.314244  1279 ts_tablet_manager.cc:1434] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:00:30.315261  1279 tablet_bootstrap.cc:492] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25: Bootstrap starting.
I20260709 13:00:30.315757  1163 raft_consensus.cc:2468] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 1.
I20260709 13:00:30.307834  1281 raft_consensus.cc:740] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.317610  1088 raft_consensus.cc:2468] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 1.
I20260709 13:00:30.320036  1281 consensus_queue.cc:260] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.321964  1319 raft_consensus.cc:493] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.322597  1319 raft_consensus.cc:515] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.326895  1291 raft_consensus.cc:493] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.327457  1291 raft_consensus.cc:515] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.329034  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd" is_pre_election: true
I20260709 13:00:30.329026  1279 tablet_bootstrap.cc:654] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:30.331063  1200 leader_election.cc:343] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:30.332571  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25" is_pre_election: true
W20260709 13:00:30.334453  1200 leader_election.cc:343] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:30.335038  1291 leader_election.cc:290] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.335145  1200 leader_election.cc:304] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d; no voters: a175f71f75a8459a9b6222a90dbd98bd, b26d86b24f1c4b45b1535569d9afba25
I20260709 13:00:30.336454  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdcf061d7a3a40119e65674f41587517" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d" is_pre_election: true
I20260709 13:00:30.337253  1319 leader_election.cc:290] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499)
I20260709 13:00:30.337592  1237 raft_consensus.cc:2393] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a175f71f75a8459a9b6222a90dbd98bd in current term 1: Already voted for candidate b26d86b24f1c4b45b1535569d9afba25 in this term.
I20260709 13:00:30.336642  1280 raft_consensus.cc:2749] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:30.338984  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdcf061d7a3a40119e65674f41587517" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25" is_pre_election: true
I20260709 13:00:30.341174  1279 tablet_bootstrap.cc:492] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25: No bootstrap required, opened a new log
I20260709 13:00:30.341686  1279 ts_tablet_manager.cc:1403] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.002s
I20260709 13:00:30.342407  1051 leader_election.cc:304] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a175f71f75a8459a9b6222a90dbd98bd; no voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25
I20260709 13:00:30.343001  1281 ts_tablet_manager.cc:1434] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd: Time spent starting tablet: real 0.058s	user 0.004s	sys 0.009s
I20260709 13:00:30.343593  1291 raft_consensus.cc:3060] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.344774  1281 tablet_bootstrap.cc:492] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd: Bootstrap starting.
I20260709 13:00:30.344643  1279 raft_consensus.cc:359] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.345410  1279 raft_consensus.cc:385] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.345718  1279 raft_consensus.cc:740] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.352751  1279 consensus_queue.cc:260] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.352463   982 catalog_manager.cc:5697] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd reported cstate change: term changed from 0 to 1, leader changed from <none> to a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.354594  1281 tablet_bootstrap.cc:654] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.356777  1279 ts_tablet_manager.cc:1434] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:00:30.357813  1279 tablet_bootstrap.cc:492] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25: Bootstrap starting.
I20260709 13:00:30.360227  1307 raft_consensus.cc:493] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.360769  1307 raft_consensus.cc:515] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.365146  1307 leader_election.cc:290] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.371057  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aac0c1b05d20415bb64a5b8de8579483" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d" is_pre_election: true
I20260709 13:00:30.371774  1237 raft_consensus.cc:2468] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 0.
I20260709 13:00:30.372145  1279 tablet_bootstrap.cc:654] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.373291  1052 leader_election.cc:304] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: 
I20260709 13:00:30.375341  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aac0c1b05d20415bb64a5b8de8579483" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25" is_pre_election: true
I20260709 13:00:30.377040  1307 raft_consensus.cc:2804] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:30.377583  1307 raft_consensus.cc:493] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.378036  1307 raft_consensus.cc:3060] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:30.379776  1051 leader_election.cc:343] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:30.381891  1279 tablet_bootstrap.cc:492] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25: No bootstrap required, opened a new log
I20260709 13:00:30.382354  1279 ts_tablet_manager.cc:1403] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.002s
I20260709 13:00:30.385218  1279 raft_consensus.cc:359] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.385932  1279 raft_consensus.cc:385] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.386196  1279 raft_consensus.cc:740] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.392729  1279 consensus_queue.cc:260] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.397189  1279 ts_tablet_manager.cc:1434] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.002s
I20260709 13:00:30.398159  1279 tablet_bootstrap.cc:492] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25: Bootstrap starting.
I20260709 13:00:30.408128  1307 raft_consensus.cc:515] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.409893  1291 raft_consensus.cc:2749] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:30.408627  1279 tablet_bootstrap.cc:654] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.421527  1279 tablet_bootstrap.cc:492] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25: No bootstrap required, opened a new log
I20260709 13:00:30.422118  1279 ts_tablet_manager.cc:1403] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.001s
I20260709 13:00:30.424817  1279 raft_consensus.cc:359] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.425524  1279 raft_consensus.cc:385] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.425443  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aac0c1b05d20415bb64a5b8de8579483" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:30.425891  1279 raft_consensus.cc:740] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.426568  1286 raft_consensus.cc:697] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Becoming Leader. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:30.427624  1286 consensus_queue.cc:237] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.429318  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aac0c1b05d20415bb64a5b8de8579483" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:30.429909  1237 raft_consensus.cc:3060] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.432758  1279 consensus_queue.cc:260] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.431773   983 catalog_manager.cc:5697] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195). New cstate: current_term: 1 leader_uuid: "96b84c522d8744e8833c0d47ed810b9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:30.435026  1301 raft_consensus.cc:493] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.435525  1301 raft_consensus.cc:515] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.439162  1236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f62839830a4b37b0d53590fa702935" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d" is_pre_election: true
I20260709 13:00:30.439283  1301 leader_election.cc:290] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.439937  1162 raft_consensus.cc:3060] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.442548  1279 ts_tablet_manager.cc:1434] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 13:00:30.439945  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f62839830a4b37b0d53590fa702935" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd" is_pre_election: true
I20260709 13:00:30.443621  1279 tablet_bootstrap.cc:492] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25: Bootstrap starting.
I20260709 13:00:30.443572  1088 raft_consensus.cc:2468] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 0.
I20260709 13:00:30.445540  1126 leader_election.cc:304] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a175f71f75a8459a9b6222a90dbd98bd, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:30.439896  1236 raft_consensus.cc:2468] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 0.
I20260709 13:00:30.451284  1286 raft_consensus.cc:2804] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:30.452004  1286 raft_consensus.cc:493] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.452613  1286 raft_consensus.cc:3060] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.455227   982 catalog_manager.cc:5697] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 reported cstate change: term changed from 0 to 1, leader changed from <none> to b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "b26d86b24f1c4b45b1535569d9afba25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.460274  1279 tablet_bootstrap.cc:654] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.460808  1307 leader_election.cc:290] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 election: Requested vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.466001  1237 raft_consensus.cc:2468] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 1.
I20260709 13:00:30.467797  1052 leader_election.cc:304] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: 
I20260709 13:00:30.469755  1281 tablet_bootstrap.cc:492] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd: No bootstrap required, opened a new log
I20260709 13:00:30.470372  1281 ts_tablet_manager.cc:1403] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd: Time spent bootstrapping tablet: real 0.126s	user 0.048s	sys 0.069s
I20260709 13:00:30.471082  1286 raft_consensus.cc:515] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.471946  1307 raft_consensus.cc:2804] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:30.472479  1307 raft_consensus.cc:697] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 1 LEADER]: Becoming Leader. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:30.473187  1281 raft_consensus.cc:359] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:30.473549  1286 leader_election.cc:290] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 election: Requested vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.473913  1281 raft_consensus.cc:385] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.474259  1281 raft_consensus.cc:740] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.474570  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f62839830a4b37b0d53590fa702935" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:30.475363  1088 raft_consensus.cc:3060] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.474962  1307 consensus_queue.cc:237] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.475235  1281 consensus_queue.cc:260] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:30.478856  1281 ts_tablet_manager.cc:1434] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.002s
I20260709 13:00:30.479830  1320 raft_consensus.cc:493] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.480501  1281 tablet_bootstrap.cc:492] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd: Bootstrap starting.
I20260709 13:00:30.480413  1320 raft_consensus.cc:515] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:30.475839  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f62839830a4b37b0d53590fa702935" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:30.482028  1237 raft_consensus.cc:3060] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.482769  1320 leader_election.cc:290] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021)
I20260709 13:00:30.483618  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25" is_pre_election: true
I20260709 13:00:30.484058  1087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd" is_pre_election: true
I20260709 13:00:30.484354  1163 raft_consensus.cc:2468] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 0.
I20260709 13:00:30.477344  1162 raft_consensus.cc:2468] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 1.
I20260709 13:00:30.484812  1087 raft_consensus.cc:2468] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 0.
I20260709 13:00:30.485769  1279 tablet_bootstrap.cc:492] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25: No bootstrap required, opened a new log
I20260709 13:00:30.486043  1200 leader_election.cc:304] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:30.487741  1320 raft_consensus.cc:2804] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:30.488166  1320 raft_consensus.cc:493] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.488577  1320 raft_consensus.cc:3060] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.490494  1279 ts_tablet_manager.cc:1403] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25: Time spent bootstrapping tablet: real 0.047s	user 0.012s	sys 0.000s
I20260709 13:00:30.493731  1088 raft_consensus.cc:2468] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 1.
I20260709 13:00:30.495555  1126 leader_election.cc:304] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a175f71f75a8459a9b6222a90dbd98bd, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:30.497447  1279 raft_consensus.cc:359] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.498170  1279 raft_consensus.cc:385] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.497816  1286 raft_consensus.cc:2804] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:30.498819  1286 raft_consensus.cc:697] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Becoming Leader. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:30.499769  1286 consensus_queue.cc:237] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.500867  1237 raft_consensus.cc:2468] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 1.
I20260709 13:00:30.502622  1279 raft_consensus.cc:740] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.503485  1279 consensus_queue.cc:260] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.505857  1279 ts_tablet_manager.cc:1434] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:00:30.509480  1319 raft_consensus.cc:493] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.508678  1323 raft_consensus.cc:493] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.511198  1281 tablet_bootstrap.cc:654] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.511327  1323 raft_consensus.cc:515] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.515550  1323 leader_election.cc:290] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.516880  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d" is_pre_election: true
I20260709 13:00:30.517632  1237 raft_consensus.cc:2468] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 0.
I20260709 13:00:30.518891  1127 leader_election.cc:304] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:30.519913  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd" is_pre_election: true
W20260709 13:00:30.522089  1126 leader_election.cc:343] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:30.518446  1320 raft_consensus.cc:515] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:30.519870  1323 raft_consensus.cc:2804] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:30.524314  1323 raft_consensus.cc:493] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.524837  1323 raft_consensus.cc:3060] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.510250  1319 raft_consensus.cc:515] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.533497  1319 leader_election.cc:290] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499)
I20260709 13:00:30.534343   982 catalog_manager.cc:5697] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd reported cstate change: term changed from 0 to 1, leader changed from <none> to a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.535583  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17d075eea46e470481e6bd91b9c78da5" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd" is_pre_election: true
I20260709 13:00:30.539489  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:30.540390  1162 raft_consensus.cc:3060] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.540299  1320 leader_election.cc:290] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 election: Requested vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021)
I20260709 13:00:30.540510  1087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:30.541299  1087 raft_consensus.cc:3060] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.543475  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17d075eea46e470481e6bd91b9c78da5" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25" is_pre_election: true
I20260709 13:00:30.546464  1163 raft_consensus.cc:2393] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96b84c522d8744e8833c0d47ed810b9d in current term 1: Already voted for candidate a175f71f75a8459a9b6222a90dbd98bd in this term.
I20260709 13:00:30.548152  1200 leader_election.cc:304] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d; no voters: a175f71f75a8459a9b6222a90dbd98bd, b26d86b24f1c4b45b1535569d9afba25
I20260709 13:00:30.552953  1323 raft_consensus.cc:515] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.557176  1323 leader_election.cc:290] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 election: Requested vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.558001  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:30.558658  1237 raft_consensus.cc:3060] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.559688  1281 tablet_bootstrap.cc:492] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd: No bootstrap required, opened a new log
I20260709 13:00:30.560158  1281 ts_tablet_manager.cc:1403] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd: Time spent bootstrapping tablet: real 0.080s	user 0.031s	sys 0.024s
I20260709 13:00:30.561395  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
W20260709 13:00:30.562812  1126 leader_election.cc:343] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:30.562944  1281 raft_consensus.cc:359] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.563660  1281 raft_consensus.cc:385] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.563956  1281 raft_consensus.cc:740] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.564248  1320 raft_consensus.cc:3060] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.564879  1281 consensus_queue.cc:260] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.569118  1281 ts_tablet_manager.cc:1434] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:00:30.570154  1281 tablet_bootstrap.cc:492] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd: Bootstrap starting.
I20260709 13:00:30.569298   985 catalog_manager.cc:5697] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 reported cstate change: term changed from 0 to 1, leader changed from <none> to b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "b26d86b24f1c4b45b1535569d9afba25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.570266  1162 raft_consensus.cc:2468] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 1.
I20260709 13:00:30.571731  1200 leader_election.cc:304] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:30.572991  1319 raft_consensus.cc:2804] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:30.573529  1319 raft_consensus.cc:697] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 1 LEADER]: Becoming Leader. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Running, Role: LEADER
I20260709 13:00:30.574687  1319 consensus_queue.cc:237] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:30.575486  1237 raft_consensus.cc:2468] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 1.
I20260709 13:00:30.577955  1281 tablet_bootstrap.cc:654] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:30.579337  1127 leader_election.cc:304] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:30.583287  1286 raft_consensus.cc:2804] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:30.587129  1286 raft_consensus.cc:697] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Becoming Leader. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:30.589550  1286 consensus_queue.cc:237] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.585615  1087 raft_consensus.cc:2468] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 1.
I20260709 13:00:30.598161  1323 raft_consensus.cc:493] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.598883  1323 raft_consensus.cc:515] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.602454  1162 raft_consensus.cc:3060] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.612149  1323 leader_election.cc:290] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.612890  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580e548b77894ff5a453639555207804" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d" is_pre_election: true
I20260709 13:00:30.613875  1237 raft_consensus.cc:2468] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 0.
I20260709 13:00:30.616029  1127 leader_election.cc:304] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:30.610607   982 catalog_manager.cc:5697] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195). New cstate: current_term: 1 leader_uuid: "96b84c522d8744e8833c0d47ed810b9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.619185  1323 raft_consensus.cc:2804] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:30.619740  1323 raft_consensus.cc:493] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.620345  1323 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.626418  1087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580e548b77894ff5a453639555207804" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd" is_pre_election: true
I20260709 13:00:30.627177  1087 raft_consensus.cc:2468] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 0.
I20260709 13:00:30.630128  1320 raft_consensus.cc:2749] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:30.631120  1281 tablet_bootstrap.cc:492] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd: No bootstrap required, opened a new log
I20260709 13:00:30.631573  1281 ts_tablet_manager.cc:1403] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd: Time spent bootstrapping tablet: real 0.062s	user 0.026s	sys 0.025s
I20260709 13:00:30.634285  1301 raft_consensus.cc:493] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.634307  1281 raft_consensus.cc:359] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.635250  1281 raft_consensus.cc:385] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:30.635643  1281 raft_consensus.cc:740] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Initialized, Role: FOLLOWER
I20260709 13:00:30.635144  1301 raft_consensus.cc:515] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.635414  1307 consensus_queue.cc:1048] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:30.636396   982 catalog_manager.cc:5697] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 reported cstate change: term changed from 0 to 1, leader changed from <none> to b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "b26d86b24f1c4b45b1535569d9afba25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.637132  1281 consensus_queue.cc:260] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.639351  1281 ts_tablet_manager.cc:1434] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:00:30.641137  1087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db84a39f3b824926adcd9d990bbcc2b6" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd" is_pre_election: true
I20260709 13:00:30.641916  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db84a39f3b824926adcd9d990bbcc2b6" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d" is_pre_election: true
I20260709 13:00:30.642612  1237 raft_consensus.cc:2468] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 1.
I20260709 13:00:30.633064  1323 raft_consensus.cc:515] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.644836  1127 leader_election.cc:304] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:30.645160  1323 leader_election.cc:290] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 election: Requested vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.645871  1286 raft_consensus.cc:2804] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:30.646176  1286 raft_consensus.cc:493] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a175f71f75a8459a9b6222a90dbd98bd)
I20260709 13:00:30.646440  1286 raft_consensus.cc:3060] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:30.647135  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580e548b77894ff5a453639555207804" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:30.647768  1237 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.648253  1087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580e548b77894ff5a453639555207804" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:30.648849  1087 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:30.656307  1237 raft_consensus.cc:2468] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 1.
I20260709 13:00:30.656924  1087 raft_consensus.cc:2468] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 1.
I20260709 13:00:30.657604  1127 leader_election.cc:304] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:30.658948  1323 raft_consensus.cc:2804] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:30.660359  1286 raft_consensus.cc:515] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.664593  1087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db84a39f3b824926adcd9d990bbcc2b6" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:30.669806  1162 raft_consensus.cc:1240] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Rejecting Update request from peer a175f71f75a8459a9b6222a90dbd98bd for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:30.672573  1285 consensus_queue.cc:1048] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:30.665938  1286 leader_election.cc:290] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 2 election: Requested vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.661586  1301 leader_election.cc:290] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.675020  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db84a39f3b824926adcd9d990bbcc2b6" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:30.675798  1237 raft_consensus.cc:3060] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:30.681361  1285 consensus_queue.cc:1059] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:30.681663  1323 raft_consensus.cc:697] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Becoming Leader. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:30.683857  1323 consensus_queue.cc:237] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.687290  1237 raft_consensus.cc:2468] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 2.
I20260709 13:00:30.688162  1285 raft_consensus.cc:3055] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:30.688375  1236 raft_consensus.cc:1240] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Rejecting Update request from peer a175f71f75a8459a9b6222a90dbd98bd for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:00:30.688616  1285 raft_consensus.cc:740] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:30.689755  1285 consensus_queue.cc:260] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.690963  1285 raft_consensus.cc:3060] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:30.693404  1127 leader_election.cc:304] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:30.694231  1323 raft_consensus.cc:2804] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:30.694808  1323 raft_consensus.cc:697] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 2 LEADER]: Becoming Leader. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:30.695691  1323 consensus_queue.cc:237] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.695920   985 catalog_manager.cc:5697] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 reported cstate change: term changed from 0 to 1, leader changed from <none> to b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "b26d86b24f1c4b45b1535569d9afba25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.710000   985 catalog_manager.cc:5697] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 reported cstate change: term changed from 1 to 2, leader changed from a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193) to b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "b26d86b24f1c4b45b1535569d9afba25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.744132  1285 consensus_queue.cc:1048] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:30.756167  1285 consensus_queue.cc:1048] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:30.770641  1323 consensus_queue.cc:1048] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:30.779347  1323 consensus_queue.cc:1048] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:30.821091  1237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261"
dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
mode: GRACEFUL
new_leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:48550
I20260709 13:00:30.821725  1237 raft_consensus.cc:606] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 1 LEADER]: Received request to transfer leadership to TS a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:30.836362  1162 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "580e548b77894ff5a453639555207804"
dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
mode: GRACEFUL
new_leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:41060
I20260709 13:00:30.836992  1162 raft_consensus.cc:606] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Received request to transfer leadership to TS a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:30.843549  1162 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6f62839830a4b37b0d53590fa702935"
dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
mode: GRACEFUL
new_leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:41060
I20260709 13:00:30.844135  1162 raft_consensus.cc:606] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Received request to transfer leadership to TS a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:30.849407  1320 consensus_queue.cc:1048] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:30.853852  1163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fdcf061d7a3a40119e65674f41587517"
dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
mode: GRACEFUL
new_leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:41060
I20260709 13:00:30.854439  1163 raft_consensus.cc:606] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Received request to transfer leadership to TS a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:30.861173  1320 consensus_queue.cc:1048] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:30.862848  1237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92e9932b04aa48418d0db4088c1a76d0"
dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
mode: GRACEFUL
new_leader_uuid: "b26d86b24f1c4b45b1535569d9afba25"
 from {username='slave'} at 127.0.0.1:48550
I20260709 13:00:30.863500  1237 raft_consensus.cc:606] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 1 LEADER]: Received request to transfer leadership to TS b26d86b24f1c4b45b1535569d9afba25
I20260709 13:00:30.865192  1320 raft_consensus.cc:993] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d: : Instructing follower b26d86b24f1c4b45b1535569d9afba25 to start an election
I20260709 13:00:30.865597  1320 raft_consensus.cc:1081] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 1 LEADER]: Signalling peer b26d86b24f1c4b45b1535569d9afba25 to start an election
I20260709 13:00:30.869714  1162 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92e9932b04aa48418d0db4088c1a76d0"
dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
 from {username='slave'} at 127.0.0.1:41044
I20260709 13:00:30.870405  1162 raft_consensus.cc:493] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:30.870855  1162 raft_consensus.cc:3060] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:30.876999  1162 raft_consensus.cc:515] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.880645  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e9932b04aa48418d0db4088c1a76d0" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:30.880035  1087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e9932b04aa48418d0db4088c1a76d0" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:30.881316  1237 raft_consensus.cc:3055] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:30.881667  1162 leader_election.cc:290] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 2 election: Requested vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:30.881718  1237 raft_consensus.cc:740] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Running, Role: LEADER
I20260709 13:00:30.882701  1237 consensus_queue.cc:260] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.883379  1087 raft_consensus.cc:3060] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:30.884020  1237 raft_consensus.cc:3060] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:30.890441  1237 raft_consensus.cc:2468] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 2.
I20260709 13:00:30.890491  1087 raft_consensus.cc:2468] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 2.
I20260709 13:00:30.892302  1126 leader_election.cc:304] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a175f71f75a8459a9b6222a90dbd98bd, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:30.893311  1323 raft_consensus.cc:2804] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:30.893944  1323 raft_consensus.cc:697] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 2 LEADER]: Becoming Leader. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:30.894970  1323 consensus_queue.cc:237] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:30.898326  1285 consensus_queue.cc:1048] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:30.905032  1087 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17d075eea46e470481e6bd91b9c78da5"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
mode: GRACEFUL
new_leader_uuid: "96b84c522d8744e8833c0d47ed810b9d"
 from {username='slave'} at 127.0.0.1:40356
I20260709 13:00:30.905010   982 catalog_manager.cc:5697] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 reported cstate change: term changed from 1 to 2, leader changed from 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195) to b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "b26d86b24f1c4b45b1535569d9afba25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.905912  1087 raft_consensus.cc:606] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 1 LEADER]: Received request to transfer leadership to TS 96b84c522d8744e8833c0d47ed810b9d
I20260709 13:00:30.906409  1301 consensus_queue.cc:1048] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:30.922712  1286 consensus_queue.cc:1048] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:30.928018  1307 consensus_queue.cc:1048] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:31.012315  1280 consensus_queue.cc:1048] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:31.024403  1280 consensus_queue.cc:1048] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:31.035045  1319 raft_consensus.cc:993] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d: : Instructing follower a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:31.035548  1351 raft_consensus.cc:1081] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 1 LEADER]: Signalling peer a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:31.037175  1087 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:40342
I20260709 13:00:31.037721  1087 raft_consensus.cc:493] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:31.038120  1087 raft_consensus.cc:3060] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.043215  1087 raft_consensus.cc:515] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:31.045382  1087 leader_election.cc:290] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 2 election: Requested vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:31.046140  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:31.046451  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:31.046809  1162 raft_consensus.cc:3060] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.047127  1237 raft_consensus.cc:3055] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:31.047470  1237 raft_consensus.cc:740] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Running, Role: LEADER
I20260709 13:00:31.048290  1237 consensus_queue.cc:260] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:31.051221  1237 raft_consensus.cc:3060] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.052459  1162 raft_consensus.cc:2468] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 2.
I20260709 13:00:31.053725  1051 leader_election.cc:304] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a175f71f75a8459a9b6222a90dbd98bd, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:31.054558  1307 raft_consensus.cc:2804] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:31.055037  1307 raft_consensus.cc:697] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Becoming Leader. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:31.055866  1307 consensus_queue.cc:237] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:31.057544  1237 raft_consensus.cc:2468] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 2.
I20260709 13:00:31.062680   983 catalog_manager.cc:5697] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd reported cstate change: term changed from 1 to 2, leader changed from 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195) to a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:31.091856  1286 consensus_queue.cc:1048] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:31.103740  1301 consensus_queue.cc:1048] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:31.112424  1286 raft_consensus.cc:993] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25: : Instructing follower a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:31.112979  1301 raft_consensus.cc:1081] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Signalling peer a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:31.114620  1087 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6f62839830a4b37b0d53590fa702935"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:40352
I20260709 13:00:31.115252  1087 raft_consensus.cc:493] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:31.115625  1087 raft_consensus.cc:3060] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.120960  1087 raft_consensus.cc:515] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:31.122993  1087 leader_election.cc:290] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 2 election: Requested vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:31.124086  1237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f62839830a4b37b0d53590fa702935" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:31.124112  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f62839830a4b37b0d53590fa702935" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:31.124714  1307 raft_consensus.cc:493] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.125006  1237 raft_consensus.cc:3060] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.125100  1162 raft_consensus.cc:3055] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:31.125610  1162 raft_consensus.cc:740] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:31.125202  1307 raft_consensus.cc:515] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:31.126428  1162 consensus_queue.cc:260] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:31.127719  1162 raft_consensus.cc:3060] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.130409  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b26d86b24f1c4b45b1535569d9afba25" is_pre_election: true
I20260709 13:00:31.130596  1307 leader_election.cc:290] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:31.132902  1236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b84c522d8744e8833c0d47ed810b9d" is_pre_election: true
I20260709 13:00:31.133692  1236 raft_consensus.cc:2393] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a175f71f75a8459a9b6222a90dbd98bd in current term 1: Already voted for candidate b26d86b24f1c4b45b1535569d9afba25 in this term.
I20260709 13:00:31.134078  1237 raft_consensus.cc:2468] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 2.
I20260709 13:00:31.135345  1052 leader_election.cc:304] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a175f71f75a8459a9b6222a90dbd98bd; no voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25
I20260709 13:00:31.136230  1162 raft_consensus.cc:2468] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 2.
I20260709 13:00:31.136468  1307 raft_consensus.cc:3060] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:31.136526  1052 leader_election.cc:304] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: 
I20260709 13:00:31.138161  1291 raft_consensus.cc:2804] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:31.138671  1291 raft_consensus.cc:697] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Becoming Leader. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:31.139245  1087 raft_consensus.cc:1275] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Refusing update from remote peer b26d86b24f1c4b45b1535569d9afba25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:31.139496  1291 consensus_queue.cc:237] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:31.136731  1301 consensus_queue.cc:1048] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:31.141530  1301 consensus_queue.cc:1048] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.147544  1339 consensus_queue.cc:1048] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:31.149746  1285 raft_consensus.cc:993] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd: : Instructing follower 96b84c522d8744e8833c0d47ed810b9d to start an election
I20260709 13:00:31.150270  1291 raft_consensus.cc:1081] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 1 LEADER]: Signalling peer 96b84c522d8744e8833c0d47ed810b9d to start an election
I20260709 13:00:31.152175  1235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17d075eea46e470481e6bd91b9c78da5"
dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
 from {username='slave'} at 127.0.0.1:48532
I20260709 13:00:31.152782  1235 raft_consensus.cc:493] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:31.153143  1235 raft_consensus.cc:3060] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.154794  1234 raft_consensus.cc:1275] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Refusing update from remote peer b26d86b24f1c4b45b1535569d9afba25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:31.156455  1323 consensus_queue.cc:1048] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:31.170382  1235 raft_consensus.cc:515] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:31.172279   985 catalog_manager.cc:5697] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd reported cstate change: term changed from 1 to 2, leader changed from b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194) to a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:31.173451  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17d075eea46e470481e6bd91b9c78da5" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:31.174017  1088 raft_consensus.cc:3055] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:31.174319  1088 raft_consensus.cc:740] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:31.172977  1323 consensus_queue.cc:1048] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:31.175120  1088 consensus_queue.cc:260] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:31.176304  1088 raft_consensus.cc:3060] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.184999  1307 raft_consensus.cc:2749] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:31.187071  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17d075eea46e470481e6bd91b9c78da5" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:31.187744  1162 raft_consensus.cc:3060] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.182160  1235 leader_election.cc:290] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 2 election: Requested vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499)
I20260709 13:00:31.190325  1088 raft_consensus.cc:2468] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 2.
I20260709 13:00:31.191582  1286 consensus_queue.cc:1048] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:31.191550  1200 leader_election.cc:304] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: 
I20260709 13:00:31.194227  1280 raft_consensus.cc:2804] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:31.194749  1280 raft_consensus.cc:697] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [term 2 LEADER]: Becoming Leader. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Running, Role: LEADER
I20260709 13:00:31.195628  1280 consensus_queue.cc:237] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:31.199216  1301 raft_consensus.cc:993] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25: : Instructing follower a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:31.199734  1301 raft_consensus.cc:1081] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Signalling peer a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:31.202438  1088 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "580e548b77894ff5a453639555207804"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:40352
I20260709 13:00:31.203500  1088 raft_consensus.cc:493] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:31.204957  1088 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.205075  1162 raft_consensus.cc:2468] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 2.
I20260709 13:00:31.212656  1088 raft_consensus.cc:515] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:31.214895  1088 leader_election.cc:290] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 2 election: Requested vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:31.216806  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580e548b77894ff5a453639555207804" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:31.217279  1233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580e548b77894ff5a453639555207804" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:31.217523  1162 raft_consensus.cc:3055] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:31.217932  1233 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.217072   984 catalog_manager.cc:5697] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d reported cstate change: term changed from 1 to 2, leader changed from a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193) to 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195). New cstate: current_term: 2 leader_uuid: "96b84c522d8744e8833c0d47ed810b9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:31.217919  1162 raft_consensus.cc:740] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:31.219895  1162 consensus_queue.cc:260] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:31.221268  1162 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.225492  1233 raft_consensus.cc:2468] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 2.
I20260709 13:00:31.226815  1052 leader_election.cc:304] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: 
I20260709 13:00:31.227679  1307 raft_consensus.cc:2804] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:31.228225  1307 raft_consensus.cc:697] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Becoming Leader. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:31.228487  1162 raft_consensus.cc:2468] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 2.
I20260709 13:00:31.229184  1307 consensus_queue.cc:237] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:31.236825   985 catalog_manager.cc:5697] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd reported cstate change: term changed from 1 to 2, leader changed from b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194) to a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:31.297837  1088 raft_consensus.cc:1275] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Refusing update from remote peer b26d86b24f1c4b45b1535569d9afba25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:31.299271  1339 consensus_queue.cc:1048] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.308825  1233 raft_consensus.cc:1275] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Refusing update from remote peer b26d86b24f1c4b45b1535569d9afba25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:31.311048  1323 consensus_queue.cc:1048] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.395136  1339 raft_consensus.cc:993] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25: : Instructing follower a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:31.395629  1323 raft_consensus.cc:1081] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Signalling peer a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:31.397241  1088 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fdcf061d7a3a40119e65674f41587517"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:40352
I20260709 13:00:31.397759  1088 raft_consensus.cc:493] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:31.398061  1088 raft_consensus.cc:3060] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.403573  1088 raft_consensus.cc:515] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:31.406301  1088 leader_election.cc:290] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 2 election: Requested vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:31.407192  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdcf061d7a3a40119e65674f41587517" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:31.407846  1233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdcf061d7a3a40119e65674f41587517" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:31.407867  1162 raft_consensus.cc:3055] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:31.408432  1162 raft_consensus.cc:740] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:31.408481  1233 raft_consensus.cc:3060] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.409336  1162 consensus_queue.cc:260] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:31.410548  1162 raft_consensus.cc:3060] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.414033  1233 raft_consensus.cc:2468] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 2.
I20260709 13:00:31.415314  1052 leader_election.cc:304] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: 
I20260709 13:00:31.416167  1307 raft_consensus.cc:2804] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:31.416597  1162 raft_consensus.cc:2468] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 2.
I20260709 13:00:31.416611  1307 raft_consensus.cc:697] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Becoming Leader. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:31.417603  1307 consensus_queue.cc:237] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:31.424283   984 catalog_manager.cc:5697] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd reported cstate change: term changed from 1 to 2, leader changed from b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194) to a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:31.535743  1233 raft_consensus.cc:1275] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:31.537519  1307 consensus_queue.cc:1048] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.549883  1162 raft_consensus.cc:1275] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:31.551649  1307 consensus_queue.cc:1048] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.623978  1233 raft_consensus.cc:1275] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:31.625351  1307 consensus_queue.cc:1048] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.633692  1162 raft_consensus.cc:1275] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:31.635476  1088 raft_consensus.cc:1275] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Refusing update from remote peer 96b84c522d8744e8833c0d47ed810b9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:31.636062  1291 consensus_queue.cc:1048] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.638456  1280 consensus_queue.cc:1048] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:31.649511  1162 raft_consensus.cc:1275] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Refusing update from remote peer 96b84c522d8744e8833c0d47ed810b9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:31.650979  1280 consensus_queue.cc:1048] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.696614  1163 raft_consensus.cc:1275] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:31.697890  1291 consensus_queue.cc:1048] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.710714  1233 raft_consensus.cc:1275] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:31.712237  1368 consensus_queue.cc:1048] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:31.829205  1163 raft_consensus.cc:1275] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:31.830761  1368 consensus_queue.cc:1048] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:31.839653  1233 raft_consensus.cc:1275] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:31.841570  1368 consensus_queue.cc:1048] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:33.959571   431 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:00:33.960566   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:33.982612  1086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
mode: GRACEFUL
new_leader_uuid: "96b84c522d8744e8833c0d47ed810b9d"
 from {username='slave'} at 127.0.0.1:40258
I20260709 13:00:33.983199  1086 raft_consensus.cc:606] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Received request to transfer leadership to TS 96b84c522d8744e8833c0d47ed810b9d
I20260709 13:00:33.987433  1086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "580e548b77894ff5a453639555207804"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
mode: GRACEFUL
new_leader_uuid: "96b84c522d8744e8833c0d47ed810b9d"
 from {username='slave'} at 127.0.0.1:40258
I20260709 13:00:33.988235  1086 raft_consensus.cc:606] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Received request to transfer leadership to TS 96b84c522d8744e8833c0d47ed810b9d
I20260709 13:00:33.990028   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:00:33.990527   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:34.200016  1375 raft_consensus.cc:993] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd: : Instructing follower 96b84c522d8744e8833c0d47ed810b9d to start an election
I20260709 13:00:34.200544  1368 raft_consensus.cc:1081] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Signalling peer 96b84c522d8744e8833c0d47ed810b9d to start an election
I20260709 13:00:34.203055  1233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261"
dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
 from {username='slave'} at 127.0.0.1:48532
I20260709 13:00:34.203775  1233 raft_consensus.cc:493] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:34.204701  1233 raft_consensus.cc:3060] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.211513  1233 raft_consensus.cc:515] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:34.214078  1233 leader_election.cc:290] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 3 election: Requested vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021)
I20260709 13:00:34.215391  1086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:34.215368  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:34.216395  1163 raft_consensus.cc:3060] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.216395  1086 raft_consensus.cc:3055] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:34.217079  1086 raft_consensus.cc:740] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:34.218061  1086 consensus_queue.cc:260] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:34.219645  1086 raft_consensus.cc:3060] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.226399  1163 raft_consensus.cc:2468] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 3.
I20260709 13:00:34.227836  1200 leader_election.cc:304] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:34.228739  1371 raft_consensus.cc:2804] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:34.229390  1086 raft_consensus.cc:2468] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 3.
I20260709 13:00:34.230585  1371 raft_consensus.cc:697] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 3 LEADER]: Becoming Leader. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Running, Role: LEADER
I20260709 13:00:34.231519  1371 consensus_queue.cc:237] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:34.242309   984 catalog_manager.cc:5697] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d reported cstate change: term changed from 2 to 3, leader changed from a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193) to 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195). New cstate: current_term: 3 leader_uuid: "96b84c522d8744e8833c0d47ed810b9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:34.416216  1375 raft_consensus.cc:993] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd: : Instructing follower 96b84c522d8744e8833c0d47ed810b9d to start an election
I20260709 13:00:34.416766  1368 raft_consensus.cc:1081] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Signalling peer 96b84c522d8744e8833c0d47ed810b9d to start an election
I20260709 13:00:34.418576  1233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "580e548b77894ff5a453639555207804"
dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
 from {username='slave'} at 127.0.0.1:48532
I20260709 13:00:34.419853  1233 raft_consensus.cc:493] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:34.420332  1233 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.428262  1233 raft_consensus.cc:515] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:34.431142  1233 leader_election.cc:290] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 3 election: Requested vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499)
I20260709 13:00:34.432343  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580e548b77894ff5a453639555207804" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:34.432240  1086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580e548b77894ff5a453639555207804" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:34.433048  1163 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.433337  1086 raft_consensus.cc:3055] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:34.433732  1086 raft_consensus.cc:740] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:34.434917  1086 consensus_queue.cc:260] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:34.436216  1086 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.441267  1163 raft_consensus.cc:2468] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 3.
I20260709 13:00:34.441865  1086 raft_consensus.cc:2468] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 3.
I20260709 13:00:34.443045  1200 leader_election.cc:304] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:34.444070  1371 raft_consensus.cc:2804] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:34.444656  1371 raft_consensus.cc:697] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 3 LEADER]: Becoming Leader. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Running, Role: LEADER
I20260709 13:00:34.445761  1371 consensus_queue.cc:237] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:34.456086   984 catalog_manager.cc:5697] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d reported cstate change: term changed from 2 to 3, leader changed from a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193) to 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195). New cstate: current_term: 3 leader_uuid: "96b84c522d8744e8833c0d47ed810b9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:34.641708  1163 raft_consensus.cc:1275] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Refusing update from remote peer 96b84c522d8744e8833c0d47ed810b9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:34.644037  1371 consensus_queue.cc:1048] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:34.656986  1086 raft_consensus.cc:1275] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Refusing update from remote peer 96b84c522d8744e8833c0d47ed810b9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:34.658851  1376 consensus_queue.cc:1048] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:34.992614   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:34.991837  1086 raft_consensus.cc:1275] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Refusing update from remote peer 96b84c522d8744e8833c0d47ed810b9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:34.996404  1371 consensus_queue.cc:1048] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:00:35.019475  1163 raft_consensus.cc:1275] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Refusing update from remote peer 96b84c522d8744e8833c0d47ed810b9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:35.021978  1371 consensus_queue.cc:1048] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:35.022680  1233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261"
dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
mode: GRACEFUL
new_leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:48550
I20260709 13:00:35.023743  1233 raft_consensus.cc:606] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 3 LEADER]: Received request to transfer leadership to TS a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:35.033825  1235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "580e548b77894ff5a453639555207804"
dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
mode: GRACEFUL
new_leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:48550
I20260709 13:00:35.034686  1235 raft_consensus.cc:606] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 3 LEADER]: Received request to transfer leadership to TS a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:35.062323  1163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92e9932b04aa48418d0db4088c1a76d0"
dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
mode: GRACEFUL
new_leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:41060
I20260709 13:00:35.063086  1163 raft_consensus.cc:606] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 2 LEADER]: Received request to transfer leadership to TS a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:35.071185  1376 raft_consensus.cc:993] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d: : Instructing follower a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:35.071157  1163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48471c61e2f84d0d9398f9b5b2d71a99"
dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
mode: GRACEFUL
new_leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:41060
I20260709 13:00:35.072027  1376 raft_consensus.cc:1081] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 3 LEADER]: Signalling peer a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:35.072172  1163 raft_consensus.cc:606] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Received request to transfer leadership to TS a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:35.074610  1086 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:40342
I20260709 13:00:35.075664  1086 raft_consensus.cc:493] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:35.076319  1086 raft_consensus.cc:3060] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.081311  1163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4"
dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
mode: GRACEFUL
new_leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:41060
I20260709 13:00:35.082187  1163 raft_consensus.cc:606] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Received request to transfer leadership to TS a175f71f75a8459a9b6222a90dbd98bd
I20260709 13:00:35.086956  1086 raft_consensus.cc:515] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:35.091166  1163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:35.092444  1163 raft_consensus.cc:3060] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.092761  1086 leader_election.cc:290] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 4 election: Requested vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:35.094148  1235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51814b87cc5f4a4fa6d60f0a14241261" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:35.094880  1388 raft_consensus.cc:993] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25: : Instructing follower a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:35.095036  1235 raft_consensus.cc:3055] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:35.095628  1235 raft_consensus.cc:740] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Running, Role: LEADER
I20260709 13:00:35.095798  1392 raft_consensus.cc:1081] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Signalling peer a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:35.097229  1235 consensus_queue.cc:260] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:35.099354  1086 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48471c61e2f84d0d9398f9b5b2d71a99"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:40352
I20260709 13:00:35.100700  1086 raft_consensus.cc:493] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:35.100660  1235 raft_consensus.cc:3060] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.101328  1086 raft_consensus.cc:3060] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:35.104367  1163 raft_consensus.cc:2468] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 4.
I20260709 13:00:35.106186  1051 leader_election.cc:304] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a175f71f75a8459a9b6222a90dbd98bd, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:35.108058  1375 raft_consensus.cc:2804] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:35.108891  1375 raft_consensus.cc:697] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 4 LEADER]: Becoming Leader. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:35.110437  1375 consensus_queue.cc:237] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } }
I20260709 13:00:35.123114  1086 raft_consensus.cc:515] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:35.125077  1235 raft_consensus.cc:2468] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 4.
I20260709 13:00:35.123901   984 catalog_manager.cc:5697] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd reported cstate change: term changed from 3 to 4, leader changed from 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195) to a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193). New cstate: current_term: 4 leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:35.127862  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48471c61e2f84d0d9398f9b5b2d71a99" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:35.128511  1162 raft_consensus.cc:3055] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:35.128613  1086 leader_election.cc:290] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 2 election: Requested vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:35.129076  1162 raft_consensus.cc:740] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:35.130254  1235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48471c61e2f84d0d9398f9b5b2d71a99" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:35.130260  1162 consensus_queue.cc:260] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:35.131543  1235 raft_consensus.cc:3060] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:35.132726  1162 raft_consensus.cc:3060] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:35.140942  1235 raft_consensus.cc:2468] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 2.
I20260709 13:00:35.142400  1162 raft_consensus.cc:2468] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 2.
I20260709 13:00:35.142680  1052 leader_election.cc:304] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: 
I20260709 13:00:35.143982  1375 raft_consensus.cc:2804] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:35.144551  1375 raft_consensus.cc:697] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Becoming Leader. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:35.146004  1375 consensus_queue.cc:237] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:35.156479   984 catalog_manager.cc:5697] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd reported cstate change: term changed from 1 to 2, leader changed from b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194) to a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:35.244830  1323 raft_consensus.cc:993] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25: : Instructing follower a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:35.245333  1323 raft_consensus.cc:1081] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Signalling peer a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:35.247807  1086 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:40352
I20260709 13:00:35.248354  1086 raft_consensus.cc:493] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:35.249135  1086 raft_consensus.cc:3060] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:35.255085  1086 raft_consensus.cc:515] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:35.257297  1086 leader_election.cc:290] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 2 election: Requested vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:35.258358  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:35.258391  1235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:35.259052  1162 raft_consensus.cc:3055] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:35.259290  1235 raft_consensus.cc:3060] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:35.259461  1162 raft_consensus.cc:740] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:35.260855  1162 consensus_queue.cc:260] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:35.261865  1162 raft_consensus.cc:3060] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:35.265669  1235 raft_consensus.cc:2468] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 2.
I20260709 13:00:35.267227  1162 raft_consensus.cc:2468] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 2.
I20260709 13:00:35.267382  1052 leader_election.cc:304] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: 
I20260709 13:00:35.268473  1375 raft_consensus.cc:2804] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:35.269016  1375 raft_consensus.cc:697] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Becoming Leader. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:35.269795  1375 consensus_queue.cc:237] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:35.278246   984 catalog_manager.cc:5697] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd reported cstate change: term changed from 1 to 2, leader changed from b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194) to a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:35.436317  1392 raft_consensus.cc:993] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25: : Instructing follower a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:35.436908  1392 raft_consensus.cc:1081] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 2 LEADER]: Signalling peer a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:35.439263  1086 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92e9932b04aa48418d0db4088c1a76d0"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:40352
I20260709 13:00:35.439824  1086 raft_consensus.cc:493] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:35.440145  1086 raft_consensus.cc:3060] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:35.446552  1086 raft_consensus.cc:515] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:35.449530  1086 leader_election.cc:290] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 3 election: Requested vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:35.449926  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e9932b04aa48418d0db4088c1a76d0" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:35.450598  1162 raft_consensus.cc:3055] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:35.450660  1235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92e9932b04aa48418d0db4088c1a76d0" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:35.451073  1162 raft_consensus.cc:740] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:35.451303  1235 raft_consensus.cc:3060] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:35.452013  1162 consensus_queue.cc:260] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:35.453516  1162 raft_consensus.cc:3060] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:35.460780  1235 raft_consensus.cc:2468] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 3.
I20260709 13:00:35.461793  1162 raft_consensus.cc:2468] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 3.
I20260709 13:00:35.462235  1052 leader_election.cc:304] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: 
I20260709 13:00:35.463753  1375 raft_consensus.cc:2804] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:35.464334  1375 raft_consensus.cc:697] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 3 LEADER]: Becoming Leader. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:35.465484  1375 consensus_queue.cc:237] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:35.473621   984 catalog_manager.cc:5697] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd reported cstate change: term changed from 2 to 3, leader changed from b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194) to a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193). New cstate: current_term: 3 leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:35.548561  1162 raft_consensus.cc:1275] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 4 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:35.550047  1375 consensus_queue.cc:1048] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:35.562458  1235 raft_consensus.cc:1275] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 4 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:35.564930  1375 consensus_queue.cc:1048] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:35.626335  1371 raft_consensus.cc:993] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d: : Instructing follower a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:35.627415  1376 raft_consensus.cc:1081] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 3 LEADER]: Signalling peer a175f71f75a8459a9b6222a90dbd98bd to start an election
I20260709 13:00:35.629858  1086 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "580e548b77894ff5a453639555207804"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
 from {username='slave'} at 127.0.0.1:40342
I20260709 13:00:35.630419  1086 raft_consensus.cc:493] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:35.630765  1086 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.638259  1086 raft_consensus.cc:515] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:35.641544  1086 leader_election.cc:290] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 4 election: Requested vote from peers b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:35.642665  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580e548b77894ff5a453639555207804" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:35.643471  1162 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.643321  1235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580e548b77894ff5a453639555207804" candidate_uuid: "a175f71f75a8459a9b6222a90dbd98bd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:35.644622  1233 raft_consensus.cc:1275] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:35.645196  1235 raft_consensus.cc:3055] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:35.645571  1235 raft_consensus.cc:740] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Running, Role: LEADER
I20260709 13:00:35.646401  1235 consensus_queue.cc:260] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:35.646562  1375 consensus_queue.cc:1048] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:35.647990  1235 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.651991  1162 raft_consensus.cc:2468] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 4.
I20260709 13:00:35.653987  1051 leader_election.cc:304] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a175f71f75a8459a9b6222a90dbd98bd, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:35.655262  1375 raft_consensus.cc:2804] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:35.655827  1375 raft_consensus.cc:697] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 4 LEADER]: Becoming Leader. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:35.656773  1375 consensus_queue.cc:237] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:35.669831  1162 raft_consensus.cc:1275] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:35.660602  1235 raft_consensus.cc:2468] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a175f71f75a8459a9b6222a90dbd98bd in term 4.
I20260709 13:00:35.672978  1404 consensus_queue.cc:1048] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:35.676092   982 catalog_manager.cc:5697] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd reported cstate change: term changed from 3 to 4, leader changed from 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195) to a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193). New cstate: current_term: 4 leader_uuid: "a175f71f75a8459a9b6222a90dbd98bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:35.793514  1235 raft_consensus.cc:1275] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:35.795184  1368 consensus_queue.cc:1048] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:35.806674  1162 raft_consensus.cc:1275] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:35.812983  1404 consensus_queue.cc:1048] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:35.991120  1235 raft_consensus.cc:1275] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:35.992690  1404 consensus_queue.cc:1048] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:36.004194  1162 raft_consensus.cc:1275] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:36.008519  1405 consensus_queue.cc:1048] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:36.075157  1235 raft_consensus.cc:1275] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 4 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:36.076709  1406 consensus_queue.cc:1048] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:36.088955  1162 raft_consensus.cc:1275] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 4 FOLLOWER]: Refusing update from remote peer a175f71f75a8459a9b6222a90dbd98bd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:36.091696  1406 consensus_queue.cc:1048] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:38.109851   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:38.120409  1088 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48471c61e2f84d0d9398f9b5b2d71a99"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
mode: GRACEFUL
new_leader_uuid: "96b84c522d8744e8833c0d47ed810b9d"
 from {username='slave'} at 127.0.0.1:40258
I20260709 13:00:38.121034  1088 raft_consensus.cc:606] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Received request to transfer leadership to TS 96b84c522d8744e8833c0d47ed810b9d
I20260709 13:00:38.124796  1088 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "580e548b77894ff5a453639555207804"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
mode: GRACEFUL
new_leader_uuid: "b26d86b24f1c4b45b1535569d9afba25"
 from {username='slave'} at 127.0.0.1:40258
I20260709 13:00:38.125346  1088 raft_consensus.cc:606] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 4 LEADER]: Received request to transfer leadership to TS b26d86b24f1c4b45b1535569d9afba25
I20260709 13:00:38.128768  1088 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aac0c1b05d20415bb64a5b8de8579483"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
mode: GRACEFUL
new_leader_uuid: "b26d86b24f1c4b45b1535569d9afba25"
 from {username='slave'} at 127.0.0.1:40258
I20260709 13:00:38.129503  1088 raft_consensus.cc:606] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 1 LEADER]: Received request to transfer leadership to TS b26d86b24f1c4b45b1535569d9afba25
I20260709 13:00:38.133638  1088 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4"
dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
mode: GRACEFUL
new_leader_uuid: "96b84c522d8744e8833c0d47ed810b9d"
 from {username='slave'} at 127.0.0.1:40258
I20260709 13:00:38.134256  1088 raft_consensus.cc:606] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Received request to transfer leadership to TS 96b84c522d8744e8833c0d47ed810b9d
I20260709 13:00:38.135614   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:00:38.136364   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:38.232494  1405 raft_consensus.cc:993] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd: : Instructing follower b26d86b24f1c4b45b1535569d9afba25 to start an election
I20260709 13:00:38.233012  1405 raft_consensus.cc:1081] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 1 LEADER]: Signalling peer b26d86b24f1c4b45b1535569d9afba25 to start an election
I20260709 13:00:38.235062  1163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aac0c1b05d20415bb64a5b8de8579483"
dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
 from {username='slave'} at 127.0.0.1:41046
I20260709 13:00:38.235774  1163 raft_consensus.cc:493] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:38.236452  1163 raft_consensus.cc:3060] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:38.243443  1163 raft_consensus.cc:515] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:38.245978  1163 leader_election.cc:290] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 2 election: Requested vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:38.247102  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aac0c1b05d20415bb64a5b8de8579483" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:38.247978  1088 raft_consensus.cc:3055] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:38.248353  1235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aac0c1b05d20415bb64a5b8de8579483" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:38.248744  1088 raft_consensus.cc:740] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:38.249346  1235 raft_consensus.cc:3060] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:38.250099  1088 consensus_queue.cc:260] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:38.252314  1088 raft_consensus.cc:3060] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:38.272656  1088 raft_consensus.cc:2468] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 2.
I20260709 13:00:38.272656  1235 raft_consensus.cc:2468] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 2.
I20260709 13:00:38.274765  1127 leader_election.cc:304] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:38.275661  1392 raft_consensus.cc:2804] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:38.277026  1392 raft_consensus.cc:697] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [term 2 LEADER]: Becoming Leader. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:38.278291  1392 consensus_queue.cc:237] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:38.288674   984 catalog_manager.cc:5697] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 reported cstate change: term changed from 1 to 2, leader changed from a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193) to b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "b26d86b24f1c4b45b1535569d9afba25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:38.379022  1368 raft_consensus.cc:993] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd: : Instructing follower b26d86b24f1c4b45b1535569d9afba25 to start an election
I20260709 13:00:38.379498  1368 raft_consensus.cc:1081] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 4 LEADER]: Signalling peer b26d86b24f1c4b45b1535569d9afba25 to start an election
I20260709 13:00:38.381480  1163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "580e548b77894ff5a453639555207804"
dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
 from {username='slave'} at 127.0.0.1:41046
I20260709 13:00:38.382179  1163 raft_consensus.cc:493] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:38.382694  1163 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.383425  1368 raft_consensus.cc:993] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd: : Instructing follower 96b84c522d8744e8833c0d47ed810b9d to start an election
I20260709 13:00:38.383921  1368 raft_consensus.cc:1081] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Signalling peer 96b84c522d8744e8833c0d47ed810b9d to start an election
I20260709 13:00:38.385674  1235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48471c61e2f84d0d9398f9b5b2d71a99"
dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
 from {username='slave'} at 127.0.0.1:48532
I20260709 13:00:38.386291  1235 raft_consensus.cc:493] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:38.386663  1235 raft_consensus.cc:3060] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:38.399178  1163 raft_consensus.cc:515] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:38.399194  1235 raft_consensus.cc:515] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:38.401960  1368 raft_consensus.cc:993] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd: : Instructing follower 96b84c522d8744e8833c0d47ed810b9d to start an election
I20260709 13:00:38.404646  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48471c61e2f84d0d9398f9b5b2d71a99" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:38.405342  1162 raft_consensus.cc:3060] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:38.405596  1368 raft_consensus.cc:1081] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Signalling peer 96b84c522d8744e8833c0d47ed810b9d to start an election
I20260709 13:00:38.405704  1233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580e548b77894ff5a453639555207804" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
I20260709 13:00:38.403692  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580e548b77894ff5a453639555207804" candidate_uuid: "b26d86b24f1c4b45b1535569d9afba25" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:38.406373  1233 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.406862  1088 raft_consensus.cc:3055] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:38.407226  1088 raft_consensus.cc:740] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:38.408226  1237 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4"
dest_uuid: "96b84c522d8744e8833c0d47ed810b9d"
 from {username='slave'} at 127.0.0.1:48532
I20260709 13:00:38.408360  1088 consensus_queue.cc:260] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:38.402693  1163 leader_election.cc:290] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 5 election: Requested vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195:38145)
I20260709 13:00:38.403837  1086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48471c61e2f84d0d9398f9b5b2d71a99" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:38.410282  1088 raft_consensus.cc:3060] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.410681  1086 raft_consensus.cc:3055] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:38.409228  1237 raft_consensus.cc:493] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:38.411089  1086 raft_consensus.cc:740] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:38.411243  1237 raft_consensus.cc:3060] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:38.412155  1086 consensus_queue.cc:260] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:38.414239  1086 raft_consensus.cc:3060] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:38.404475  1235 leader_election.cc:290] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 3 election: Requested vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499)
I20260709 13:00:38.444505  1162 raft_consensus.cc:2468] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 3.
I20260709 13:00:38.444530  1233 raft_consensus.cc:2468] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 5.
I20260709 13:00:38.444826  1086 raft_consensus.cc:2468] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 3.
I20260709 13:00:38.444505  1088 raft_consensus.cc:2468] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b26d86b24f1c4b45b1535569d9afba25 in term 5.
I20260709 13:00:38.445847  1237 raft_consensus.cc:515] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:38.447090  1127 leader_election.cc:304] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, b26d86b24f1c4b45b1535569d9afba25; no voters: 
I20260709 13:00:38.447947  1392 raft_consensus.cc:2804] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:38.448480  1392 raft_consensus.cc:697] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 5 LEADER]: Becoming Leader. State: Replica: b26d86b24f1c4b45b1535569d9afba25, State: Running, Role: LEADER
I20260709 13:00:38.448606  1200 leader_election.cc:304] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: 
I20260709 13:00:38.449676  1392 consensus_queue.cc:237] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:38.450191  1376 raft_consensus.cc:2804] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:38.451828  1088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a175f71f75a8459a9b6222a90dbd98bd"
I20260709 13:00:38.452450  1088 raft_consensus.cc:3055] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:38.452785  1088 raft_consensus.cc:740] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a175f71f75a8459a9b6222a90dbd98bd, State: Running, Role: LEADER
I20260709 13:00:38.453394  1376 raft_consensus.cc:697] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 3 LEADER]: Becoming Leader. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Running, Role: LEADER
I20260709 13:00:38.448760  1237 leader_election.cc:290] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 3 election: Requested vote from peers a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021), b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499)
I20260709 13:00:38.454690  1088 consensus_queue.cc:260] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:38.454785  1376 consensus_queue.cc:237] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:38.456624  1162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92324f9111f4363bcd60d1fc1fc1fd4" candidate_uuid: "96b84c522d8744e8833c0d47ed810b9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b26d86b24f1c4b45b1535569d9afba25"
I20260709 13:00:38.456877  1088 raft_consensus.cc:3060] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:38.457755  1162 raft_consensus.cc:3060] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:38.473991   984 catalog_manager.cc:5697] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 reported cstate change: term changed from 4 to 5, leader changed from a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193) to b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "b26d86b24f1c4b45b1535569d9afba25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:38.478369   982 catalog_manager.cc:5697] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d reported cstate change: term changed from 2 to 3, leader changed from a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193) to 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195). New cstate: current_term: 3 leader_uuid: "96b84c522d8744e8833c0d47ed810b9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:38.507457  1088 raft_consensus.cc:2468] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 3.
I20260709 13:00:38.507452  1162 raft_consensus.cc:2468] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b84c522d8744e8833c0d47ed810b9d in term 3.
I20260709 13:00:38.509297  1200 leader_election.cc:304] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96b84c522d8744e8833c0d47ed810b9d, a175f71f75a8459a9b6222a90dbd98bd; no voters: 
I20260709 13:00:38.510455  1376 raft_consensus.cc:2804] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:38.511093  1376 raft_consensus.cc:697] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 3 LEADER]: Becoming Leader. State: Replica: 96b84c522d8744e8833c0d47ed810b9d, State: Running, Role: LEADER
I20260709 13:00:38.512629  1376 consensus_queue.cc:237] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } }
I20260709 13:00:38.523272   982 catalog_manager.cc:5697] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d reported cstate change: term changed from 2 to 3, leader changed from a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193) to 96b84c522d8744e8833c0d47ed810b9d (127.0.107.195). New cstate: current_term: 3 leader_uuid: "96b84c522d8744e8833c0d47ed810b9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:38.811806  1237 raft_consensus.cc:1275] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Refusing update from remote peer b26d86b24f1c4b45b1535569d9afba25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:38.813313  1392 consensus_queue.cc:1048] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:38.828933  1088 raft_consensus.cc:1275] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Refusing update from remote peer b26d86b24f1c4b45b1535569d9afba25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:38.831663  1392 consensus_queue.cc:1048] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:38.905109  1088 raft_consensus.cc:1275] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Refusing update from remote peer 96b84c522d8744e8833c0d47ed810b9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:38.907776  1376 consensus_queue.cc:1048] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:38.920897  1162 raft_consensus.cc:1275] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Refusing update from remote peer 96b84c522d8744e8833c0d47ed810b9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:38.922842  1421 consensus_queue.cc:1048] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:38.953394  1237 raft_consensus.cc:1275] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 5 FOLLOWER]: Refusing update from remote peer b26d86b24f1c4b45b1535569d9afba25: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:38.955842  1392 consensus_queue.cc:1048] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b84c522d8744e8833c0d47ed810b9d" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:38.972498  1088 raft_consensus.cc:1275] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 5 FOLLOWER]: Refusing update from remote peer b26d86b24f1c4b45b1535569d9afba25: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:38.974057  1435 consensus_queue.cc:1048] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:39.058524  1088 raft_consensus.cc:1275] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Refusing update from remote peer 96b84c522d8744e8833c0d47ed810b9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:39.060498  1421 consensus_queue.cc:1048] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a175f71f75a8459a9b6222a90dbd98bd" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:39.072280  1162 raft_consensus.cc:1275] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Refusing update from remote peer 96b84c522d8744e8833c0d47ed810b9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:39.074203  1421 consensus_queue.cc:1048] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b26d86b24f1c4b45b1535569d9afba25" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:39.137394   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:39.143724   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:00:39.189710  1126 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
I20260709 13:00:39.194897   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:00:39.195564  1126 consensus_peers.cc:597] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 -> Peer a175f71f75a8459a9b6222a90dbd98bd (127.0.107.193:46021): Couldn't send request to peer a175f71f75a8459a9b6222a90dbd98bd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:39.196362   431 tablet_replica.cc:333] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd: stopping tablet replica
I20260709 13:00:39.197542   431 raft_consensus.cc:2243] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.198194   431 raft_consensus.cc:2272] T f92324f9111f4363bcd60d1fc1fc1fd4 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.200618   431 tablet_replica.cc:333] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd: stopping tablet replica
I20260709 13:00:39.201314   431 raft_consensus.cc:2243] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.201790   431 raft_consensus.cc:2272] T 580e548b77894ff5a453639555207804 P a175f71f75a8459a9b6222a90dbd98bd [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.205727   431 tablet_replica.cc:333] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd: stopping tablet replica
I20260709 13:00:39.206784   431 raft_consensus.cc:2243] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:00:39.207792   431 raft_consensus.cc:2272] T 51814b87cc5f4a4fa6d60f0a14241261 P a175f71f75a8459a9b6222a90dbd98bd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.210613   431 tablet_replica.cc:333] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd: stopping tablet replica
I20260709 13:00:39.211762   431 raft_consensus.cc:2243] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.212502   431 raft_consensus.cc:2272] T 48471c61e2f84d0d9398f9b5b2d71a99 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.215366   431 tablet_replica.cc:333] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd: stopping tablet replica
I20260709 13:00:39.216413   431 raft_consensus.cc:2243] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.217104   431 raft_consensus.cc:2272] T db84a39f3b824926adcd9d990bbcc2b6 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.219125   431 tablet_replica.cc:333] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd: stopping tablet replica
I20260709 13:00:39.219631   431 raft_consensus.cc:2243] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.220283   431 raft_consensus.cc:2272] T 17d075eea46e470481e6bd91b9c78da5 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.222844   431 tablet_replica.cc:333] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd: stopping tablet replica
I20260709 13:00:39.223647   431 raft_consensus.cc:2243] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:00:39.224565   431 raft_consensus.cc:2272] T 92e9932b04aa48418d0db4088c1a76d0 P a175f71f75a8459a9b6222a90dbd98bd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.227468   431 tablet_replica.cc:333] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd: stopping tablet replica
I20260709 13:00:39.228430   431 raft_consensus.cc:2243] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.229200   431 raft_consensus.cc:2272] T aac0c1b05d20415bb64a5b8de8579483 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.231653   431 tablet_replica.cc:333] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd: stopping tablet replica
I20260709 13:00:39.232425   431 raft_consensus.cc:2243] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:39.233326   431 raft_consensus.cc:2272] T fdcf061d7a3a40119e65674f41587517 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.235524   431 tablet_replica.cc:333] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd: stopping tablet replica
I20260709 13:00:39.236099   431 raft_consensus.cc:2243] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:39.236913   431 raft_consensus.cc:2272] T d6f62839830a4b37b0d53590fa702935 P a175f71f75a8459a9b6222a90dbd98bd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.274649   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:00:39.300326   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 13:00:39.349602   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:39.350344   431 tablet_replica.cc:333] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25: stopping tablet replica
I20260709 13:00:39.351032   431 raft_consensus.cc:2243] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:39.351791   431 raft_consensus.cc:2272] T aac0c1b05d20415bb64a5b8de8579483 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.354671   431 tablet_replica.cc:333] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25: stopping tablet replica
I20260709 13:00:39.356046   431 raft_consensus.cc:2243] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:00:39.356930   431 raft_consensus.cc:2272] T 580e548b77894ff5a453639555207804 P b26d86b24f1c4b45b1535569d9afba25 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.360020   431 tablet_replica.cc:333] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25: stopping tablet replica
I20260709 13:00:39.360705   431 raft_consensus.cc:2243] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.361311   431 raft_consensus.cc:2272] T f92324f9111f4363bcd60d1fc1fc1fd4 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.364017   431 tablet_replica.cc:333] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25: stopping tablet replica
I20260709 13:00:39.364611   431 raft_consensus.cc:2243] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.365077   431 raft_consensus.cc:2272] T 92e9932b04aa48418d0db4088c1a76d0 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.366847   431 tablet_replica.cc:333] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25: stopping tablet replica
I20260709 13:00:39.367537   431 raft_consensus.cc:2243] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.368037   431 raft_consensus.cc:2272] T d6f62839830a4b37b0d53590fa702935 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.370421   431 tablet_replica.cc:333] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25: stopping tablet replica
I20260709 13:00:39.371014   431 raft_consensus.cc:2243] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.371490   431 raft_consensus.cc:2272] T fdcf061d7a3a40119e65674f41587517 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.373636   431 tablet_replica.cc:333] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25: stopping tablet replica
I20260709 13:00:39.374231   431 raft_consensus.cc:2243] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:39.375286   431 raft_consensus.cc:2272] T db84a39f3b824926adcd9d990bbcc2b6 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.377254   431 tablet_replica.cc:333] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25: stopping tablet replica
I20260709 13:00:39.377974   431 raft_consensus.cc:2243] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.378440   431 raft_consensus.cc:2272] T 17d075eea46e470481e6bd91b9c78da5 P b26d86b24f1c4b45b1535569d9afba25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.380477   431 tablet_replica.cc:333] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25: stopping tablet replica
I20260709 13:00:39.380999   431 raft_consensus.cc:2243] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.381472   431 raft_consensus.cc:2272] T 51814b87cc5f4a4fa6d60f0a14241261 P b26d86b24f1c4b45b1535569d9afba25 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.383204   431 tablet_replica.cc:333] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25: stopping tablet replica
I20260709 13:00:39.383901   431 raft_consensus.cc:2243] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.384469   431 raft_consensus.cc:2272] T 48471c61e2f84d0d9398f9b5b2d71a99 P b26d86b24f1c4b45b1535569d9afba25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.419049   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:00:39.439170  1200 consensus_peers.cc:597] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d -> Peer b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499): Couldn't send request to peer b26d86b24f1c4b45b1535569d9afba25. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33499: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:39.449057   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:00:39.485826  1200 consensus_peers.cc:597] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d -> Peer b26d86b24f1c4b45b1535569d9afba25 (127.0.107.194:33499): Couldn't send request to peer b26d86b24f1c4b45b1535569d9afba25. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33499: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:39.493525   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:00:39.494611   431 tablet_replica.cc:333] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d: stopping tablet replica
I20260709 13:00:39.496076   431 raft_consensus.cc:2243] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.496794   431 raft_consensus.cc:2272] T aac0c1b05d20415bb64a5b8de8579483 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.499828   431 tablet_replica.cc:333] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d: stopping tablet replica
I20260709 13:00:39.500447   431 raft_consensus.cc:2243] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:00:39.501405   431 raft_consensus.cc:2272] T 17d075eea46e470481e6bd91b9c78da5 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.503681   431 tablet_replica.cc:333] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d: stopping tablet replica
I20260709 13:00:39.504307   431 raft_consensus.cc:2243] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.504892   431 raft_consensus.cc:2272] T db84a39f3b824926adcd9d990bbcc2b6 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.507295   431 tablet_replica.cc:333] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d: stopping tablet replica
I20260709 13:00:39.507949   431 raft_consensus.cc:2243] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.508448   431 raft_consensus.cc:2272] T 51814b87cc5f4a4fa6d60f0a14241261 P 96b84c522d8744e8833c0d47ed810b9d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.510315   431 tablet_replica.cc:333] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d: stopping tablet replica
I20260709 13:00:39.511335   431 raft_consensus.cc:2243] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:00:39.512351   431 raft_consensus.cc:2272] T 48471c61e2f84d0d9398f9b5b2d71a99 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.515844   431 tablet_replica.cc:333] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d: stopping tablet replica
I20260709 13:00:39.516609   431 raft_consensus.cc:2243] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.517120   431 raft_consensus.cc:2272] T 580e548b77894ff5a453639555207804 P 96b84c522d8744e8833c0d47ed810b9d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.519999   431 tablet_replica.cc:333] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d: stopping tablet replica
I20260709 13:00:39.521005   431 raft_consensus.cc:2243] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:00:39.522192   431 raft_consensus.cc:2272] T f92324f9111f4363bcd60d1fc1fc1fd4 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.524462   431 tablet_replica.cc:333] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d: stopping tablet replica
I20260709 13:00:39.525655   431 raft_consensus.cc:2243] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.526149   431 raft_consensus.cc:2272] T 92e9932b04aa48418d0db4088c1a76d0 P 96b84c522d8744e8833c0d47ed810b9d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.528698   431 tablet_replica.cc:333] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d: stopping tablet replica
I20260709 13:00:39.529706   431 raft_consensus.cc:2243] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.530220   431 raft_consensus.cc:2272] T d6f62839830a4b37b0d53590fa702935 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.532534   431 tablet_replica.cc:333] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d: stopping tablet replica
I20260709 13:00:39.533113   431 raft_consensus.cc:2243] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:39.533825   431 raft_consensus.cc:2272] T fdcf061d7a3a40119e65674f41587517 P 96b84c522d8744e8833c0d47ed810b9d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.570281   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:00:39.596254   431 master.cc:562] Master@127.0.107.254:43429 shutting down...
I20260709 13:00:39.629616   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:00:39.630354   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:39.630750   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0dfb1a14974243a78631cbd3c496bbe6: stopping tablet replica
I20260709 13:00:39.654754   431 master.cc:584] Master@127.0.107.254:43429 shutdown complete.
96b84c522d8744e8833c0d47ed810b9d  1
a175f71f75a8459a9b6222a90dbd98bd  5
b26d86b24f1c4b45b1535569d9afba25  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10835 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:00:39.712260   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:37201
I20260709 13:00:39.714514   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:39.723254  1442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:39.723325  1443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:39.723738  1445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:39.727912   431 server_base.cc:1061] running on GCE node
I20260709 13:00:39.729482   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:39.729736   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:39.729924   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602039729909 us; error 0 us; skew 500 ppm
I20260709 13:00:39.730670   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:39.734058   431 webserver.cc:533] Webserver started at http://127.0.107.254:36323/ using document root <none> and password file <none>
I20260709 13:00:39.735002   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:39.735219   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:39.735970   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:39.737241   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/master-0-root/instance:
uuid: "6ffeec09a12e4bd6997354c8ecdff02d"
format_stamp: "Formatted at 2026-07-09 13:00:39 on dist-test-slave-cq4t"
I20260709 13:00:39.743813   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:00:39.748597  1450 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:39.749562   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:00:39.749840   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/master-0-root
uuid: "6ffeec09a12e4bd6997354c8ecdff02d"
format_stamp: "Formatted at 2026-07-09 13:00:39 on dist-test-slave-cq4t"
I20260709 13:00:39.750149   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:39.787464   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:39.789220   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:39.849814  1501 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:37201 every 8 connection(s)
I20260709 13:00:39.850436   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:37201
I20260709 13:00:39.855939  1502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:39.868130  1502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Bootstrap starting.
I20260709 13:00:39.875555  1502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:39.881726  1502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: No bootstrap required, opened a new log
I20260709 13:00:39.885053  1502 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ffeec09a12e4bd6997354c8ecdff02d" member_type: VOTER }
I20260709 13:00:39.885689  1502 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:39.886055  1502 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ffeec09a12e4bd6997354c8ecdff02d, State: Initialized, Role: FOLLOWER
I20260709 13:00:39.886864  1502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ffeec09a12e4bd6997354c8ecdff02d" member_type: VOTER }
I20260709 13:00:39.887600  1502 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:00:39.888031  1502 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:00:39.888427  1502 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:39.911669  1502 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ffeec09a12e4bd6997354c8ecdff02d" member_type: VOTER }
I20260709 13:00:39.912942  1502 leader_election.cc:304] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ffeec09a12e4bd6997354c8ecdff02d; no voters: 
I20260709 13:00:39.914649  1502 leader_election.cc:290] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:00:39.915570  1505 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:39.917246  1505 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [term 1 LEADER]: Becoming Leader. State: Replica: 6ffeec09a12e4bd6997354c8ecdff02d, State: Running, Role: LEADER
I20260709 13:00:39.918257  1505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ffeec09a12e4bd6997354c8ecdff02d" member_type: VOTER }
I20260709 13:00:39.919073  1502 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:00:39.923578  1507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ffeec09a12e4bd6997354c8ecdff02d. Latest consensus state: current_term: 1 leader_uuid: "6ffeec09a12e4bd6997354c8ecdff02d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ffeec09a12e4bd6997354c8ecdff02d" member_type: VOTER } }
I20260709 13:00:39.924139  1507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:39.926136  1506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ffeec09a12e4bd6997354c8ecdff02d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ffeec09a12e4bd6997354c8ecdff02d" member_type: VOTER } }
I20260709 13:00:39.926483  1511 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:00:39.926719  1506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [sys.catalog]: This master's current role is: LEADER
I20260709 13:00:39.934584  1511 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:00:39.942391   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:00:39.946859  1511 catalog_manager.cc:1383] Generated new cluster ID: 8d9583ef83f54369857703dd7bc1716e
I20260709 13:00:39.947233  1511 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:00:39.970940  1511 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:00:39.972820  1511 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:00:39.995061  1511 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Generated new TSK 0
I20260709 13:00:39.995687  1511 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:00:40.014241   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:40.023048  1523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:40.024994  1524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:40.029963  1526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:40.033227   431 server_base.cc:1061] running on GCE node
I20260709 13:00:40.034372   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:40.034735   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:40.035167   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602040035138 us; error 0 us; skew 500 ppm
I20260709 13:00:40.035948   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:40.039625   431 webserver.cc:533] Webserver started at http://127.0.107.193:44873/ using document root <none> and password file <none>
I20260709 13:00:40.040243   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:40.040843   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:40.041333   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:40.042992   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/instance:
uuid: "3f64b76180a54492bb015e405caeacec"
format_stamp: "Formatted at 2026-07-09 13:00:40 on dist-test-slave-cq4t"
I20260709 13:00:40.050366   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:00:40.054903  1531 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:40.055975   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:00:40.056610   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root
uuid: "3f64b76180a54492bb015e405caeacec"
format_stamp: "Formatted at 2026-07-09 13:00:40 on dist-test-slave-cq4t"
I20260709 13:00:40.057042   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:40.072521   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:40.074471   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:40.076609   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:40.080483   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:40.080825   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:40.081130   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:40.081339   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:40.145859   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:36547
I20260709 13:00:40.145920  1593 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:36547 every 8 connection(s)
I20260709 13:00:40.152716   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:40.164256  1598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:40.165458  1599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:40.171636  1601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:40.173334   431 server_base.cc:1061] running on GCE node
I20260709 13:00:40.175145   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:40.175470   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:40.175801  1594 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37201
I20260709 13:00:40.175863   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602040175834 us; error 0 us; skew 500 ppm
I20260709 13:00:40.176319  1594 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:40.176667   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:40.177172  1594 heartbeater.cc:507] Master 127.0.107.254:37201 requested a full tablet report, sending...
I20260709 13:00:40.180393   431 webserver.cc:533] Webserver started at http://127.0.107.194:35981/ using document root <none> and password file <none>
I20260709 13:00:40.181164   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:40.181036  1467 ts_manager.cc:194] Registered new tserver with Master: 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
I20260709 13:00:40.181531   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:40.182068   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:40.184113  1467 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45142
I20260709 13:00:40.184201   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/instance:
uuid: "5184c101be204393adf3ac8b31f25ad6"
format_stamp: "Formatted at 2026-07-09 13:00:40 on dist-test-slave-cq4t"
I20260709 13:00:40.190660   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:00:40.195647  1606 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:40.196676   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:00:40.197048   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root
uuid: "5184c101be204393adf3ac8b31f25ad6"
format_stamp: "Formatted at 2026-07-09 13:00:40 on dist-test-slave-cq4t"
I20260709 13:00:40.197373   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:40.223203   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:40.225021   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:40.227706   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:40.231419   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:40.231706   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:40.231977   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:40.232156   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:40.299528   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:40023
I20260709 13:00:40.299613  1668 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:40023 every 8 connection(s)
I20260709 13:00:40.306877   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:40.317744  1672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:40.320551  1669 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37201
I20260709 13:00:40.321419  1669 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:40.322376   431 server_base.cc:1061] running on GCE node
I20260709 13:00:40.322682  1669 heartbeater.cc:507] Master 127.0.107.254:37201 requested a full tablet report, sending...
W20260709 13:00:40.322396  1673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:40.324605  1676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:40.325879   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:40.326093   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:40.326377   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602040326336 us; error 0 us; skew 500 ppm
I20260709 13:00:40.326097  1467 ts_manager.cc:194] Registered new tserver with Master: 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:40.327194   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:40.328447  1467 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45150
I20260709 13:00:40.332755   431 webserver.cc:533] Webserver started at http://127.0.107.195:33647/ using document root <none> and password file <none>
I20260709 13:00:40.333510   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:40.333842   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:40.334149   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:40.335521   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/instance:
uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
format_stamp: "Formatted at 2026-07-09 13:00:40 on dist-test-slave-cq4t"
I20260709 13:00:40.341404   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 13:00:40.346387  1680 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:40.347805   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:00:40.348197   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root
uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
format_stamp: "Formatted at 2026-07-09 13:00:40 on dist-test-slave-cq4t"
I20260709 13:00:40.348721   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:40.368033   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:40.369378   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:40.371146   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:40.374537   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:40.374811   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:40.375113   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:40.375487   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:40.442906   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:33997
I20260709 13:00:40.443095  1742 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:33997 every 8 connection(s)
I20260709 13:00:40.465934  1743 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37201
I20260709 13:00:40.466508  1743 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:40.468019  1743 heartbeater.cc:507] Master 127.0.107.254:37201 requested a full tablet report, sending...
I20260709 13:00:40.470834  1467 ts_manager.cc:194] Registered new tserver with Master: 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:40.471707   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.023736133s
I20260709 13:00:40.472751  1467 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45164
I20260709 13:00:40.508754  1467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45170:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:00:40.518265  1467 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:00:40.788522  1559 tablet_service.cc:1511] Processing CreateTablet for tablet 1b7e7c3881a44475827c9cbca08a2fe1 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:00:40.791081  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b7e7c3881a44475827c9cbca08a2fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.792709  1555 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6815b361554c338159f274bd78c0e9 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:00:40.796535  1556 tablet_service.cc:1511] Processing CreateTablet for tablet f712f9c188e44e588e739161cac39330 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:00:40.801644  1558 tablet_service.cc:1511] Processing CreateTablet for tablet 6711207c14044518a57e7c8914ac0091 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:00:40.799515  1557 tablet_service.cc:1511] Processing CreateTablet for tablet fb294ac919b44e56960745f79e83dfc4 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:00:40.805277  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6815b361554c338159f274bd78c0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.806005  1554 tablet_service.cc:1511] Processing CreateTablet for tablet c5eae102421c4bc4b19781bf4470a0b0 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:00:40.806280  1553 tablet_service.cc:1511] Processing CreateTablet for tablet 9310eb7a2c8447e28c9096c983ddf5b5 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:00:40.807247  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb294ac919b44e56960745f79e83dfc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.809293  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f712f9c188e44e588e739161cac39330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.810847  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6711207c14044518a57e7c8914ac0091. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.810586  1552 tablet_service.cc:1511] Processing CreateTablet for tablet 82743dc3081f4edba18591ca4b0d9c09 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:00:40.812899  1554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5eae102421c4bc4b19781bf4470a0b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.814842  1551 tablet_service.cc:1511] Processing CreateTablet for tablet 147a8e85db5b4380bf845a82736ec5fd (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:00:40.817037  1551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 147a8e85db5b4380bf845a82736ec5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.819633  1553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9310eb7a2c8447e28c9096c983ddf5b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.827267  1552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82743dc3081f4edba18591ca4b0d9c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.830468  1634 tablet_service.cc:1511] Processing CreateTablet for tablet 1b7e7c3881a44475827c9cbca08a2fe1 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:00:40.832460  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b7e7c3881a44475827c9cbca08a2fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.835088  1633 tablet_service.cc:1511] Processing CreateTablet for tablet 6711207c14044518a57e7c8914ac0091 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:00:40.836705  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6711207c14044518a57e7c8914ac0091. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.837891  1708 tablet_service.cc:1511] Processing CreateTablet for tablet 1b7e7c3881a44475827c9cbca08a2fe1 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:00:40.843839  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b7e7c3881a44475827c9cbca08a2fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.845461  1707 tablet_service.cc:1511] Processing CreateTablet for tablet 6711207c14044518a57e7c8914ac0091 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:00:40.848338  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6711207c14044518a57e7c8914ac0091. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.848559  1705 tablet_service.cc:1511] Processing CreateTablet for tablet f712f9c188e44e588e739161cac39330 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:00:40.850093  1705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f712f9c188e44e588e739161cac39330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.850606  1632 tablet_service.cc:1511] Processing CreateTablet for tablet fb294ac919b44e56960745f79e83dfc4 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:00:40.852025  1702 tablet_service.cc:1511] Processing CreateTablet for tablet 9310eb7a2c8447e28c9096c983ddf5b5 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:00:40.852798  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb294ac919b44e56960745f79e83dfc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.853811  1702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9310eb7a2c8447e28c9096c983ddf5b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.852113  1703 tablet_service.cc:1511] Processing CreateTablet for tablet c5eae102421c4bc4b19781bf4470a0b0 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:00:40.854214  1550 tablet_service.cc:1511] Processing CreateTablet for tablet d9ba36df4f014c3a916b0393305ea44a (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:00:40.857264  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9ba36df4f014c3a916b0393305ea44a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.856688  1704 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6815b361554c338159f274bd78c0e9 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:00:40.860198  1706 tablet_service.cc:1511] Processing CreateTablet for tablet fb294ac919b44e56960745f79e83dfc4 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:00:40.864248  1701 tablet_service.cc:1511] Processing CreateTablet for tablet 82743dc3081f4edba18591ca4b0d9c09 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:00:40.868731  1700 tablet_service.cc:1511] Processing CreateTablet for tablet 147a8e85db5b4380bf845a82736ec5fd (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:00:40.875406  1701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82743dc3081f4edba18591ca4b0d9c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.876295  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5eae102421c4bc4b19781bf4470a0b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.880637  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb294ac919b44e56960745f79e83dfc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.883875  1704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6815b361554c338159f274bd78c0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.894016  1631 tablet_service.cc:1511] Processing CreateTablet for tablet f712f9c188e44e588e739161cac39330 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:00:40.895363  1700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 147a8e85db5b4380bf845a82736ec5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.896895  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f712f9c188e44e588e739161cac39330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.902030  1630 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6815b361554c338159f274bd78c0e9 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:00:40.904867  1630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6815b361554c338159f274bd78c0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.910347  1629 tablet_service.cc:1511] Processing CreateTablet for tablet c5eae102421c4bc4b19781bf4470a0b0 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:00:40.915520  1629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5eae102421c4bc4b19781bf4470a0b0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:40.935308  1515 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:40.938059  1628 tablet_service.cc:1511] Processing CreateTablet for tablet 9310eb7a2c8447e28c9096c983ddf5b5 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:00:40.941283  1628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9310eb7a2c8447e28c9096c983ddf5b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.950317  1627 tablet_service.cc:1511] Processing CreateTablet for tablet 82743dc3081f4edba18591ca4b0d9c09 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:00:40.954548  1627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82743dc3081f4edba18591ca4b0d9c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.960386  1699 tablet_service.cc:1511] Processing CreateTablet for tablet d9ba36df4f014c3a916b0393305ea44a (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:00:40.962134  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9ba36df4f014c3a916b0393305ea44a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:40.993633  1626 tablet_service.cc:1511] Processing CreateTablet for tablet 147a8e85db5b4380bf845a82736ec5fd (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:00:40.999742  1626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 147a8e85db5b4380bf845a82736ec5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.006328  1761 tablet_bootstrap.cc:492] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:41.009311  1625 tablet_service.cc:1511] Processing CreateTablet for tablet d9ba36df4f014c3a916b0393305ea44a (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:00:41.013568  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9ba36df4f014c3a916b0393305ea44a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.015585  1707 tablet_service.cc:1511] Processing CreateTablet for tablet 2b9b7b078d0d47d3880d3dec3230b779 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:00:41.016956  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b9b7b078d0d47d3880d3dec3230b779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.017361  1550 tablet_service.cc:1511] Processing CreateTablet for tablet cdf54b70d44944ef9013c4adb0970687 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:00:41.018864  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdf54b70d44944ef9013c4adb0970687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.040634  1558 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe2946ff23d498f98c4627ee968a4e3 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:00:41.042433  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe2946ff23d498f98c4627ee968a4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.051976  1559 tablet_service.cc:1511] Processing CreateTablet for tablet 37b565cd27a64e96addb09389c2c7d5e (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:00:41.053570  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37b565cd27a64e96addb09389c2c7d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.070106  1762 tablet_bootstrap.cc:492] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:41.075955  1555 tablet_service.cc:1511] Processing CreateTablet for tablet 2b9b7b078d0d47d3880d3dec3230b779 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:00:41.080179  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b9b7b078d0d47d3880d3dec3230b779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.082094  1703 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe2946ff23d498f98c4627ee968a4e3 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:00:41.088924  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe2946ff23d498f98c4627ee968a4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.109413  1761 tablet_bootstrap.cc:654] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:41.117532  1763 tablet_bootstrap.cc:492] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:41.120503  1633 tablet_service.cc:1511] Processing CreateTablet for tablet 2b9b7b078d0d47d3880d3dec3230b779 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:00:41.123049  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b9b7b078d0d47d3880d3dec3230b779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.126431  1762 tablet_bootstrap.cc:654] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:41.067108  1707 tablet_service.cc:1511] Processing CreateTablet for tablet cdf54b70d44944ef9013c4adb0970687 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:00:41.129714  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdf54b70d44944ef9013c4adb0970687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.141469  1634 tablet_service.cc:1511] Processing CreateTablet for tablet cdf54b70d44944ef9013c4adb0970687 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:00:41.166242  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdf54b70d44944ef9013c4adb0970687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.187883  1594 heartbeater.cc:499] Master 127.0.107.254:37201 was elected leader, sending a full tablet report...
W20260709 13:00:41.175551  1451 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 55399us
I20260709 13:00:41.207907  1552 tablet_service.cc:1511] Processing CreateTablet for tablet 1e353d5ccbb3478ba62d1d60ecc3ec05 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:00:41.210533  1552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e353d5ccbb3478ba62d1d60ecc3ec05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.212024  1699 tablet_service.cc:1511] Processing CreateTablet for tablet 37b565cd27a64e96addb09389c2c7d5e (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:00:41.213900  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37b565cd27a64e96addb09389c2c7d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.215142  1631 tablet_service.cc:1511] Processing CreateTablet for tablet 344c88e0be9640f583140fb4aab3af58 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:00:41.216796  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 344c88e0be9640f583140fb4aab3af58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.180668  1632 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe2946ff23d498f98c4627ee968a4e3 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:00:41.217970  1702 tablet_service.cc:1511] Processing CreateTablet for tablet 1e353d5ccbb3478ba62d1d60ecc3ec05 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:00:41.219816  1702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e353d5ccbb3478ba62d1d60ecc3ec05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.222033  1763 tablet_bootstrap.cc:654] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:41.222967  1761 tablet_bootstrap.cc:492] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:41.223539  1761 ts_tablet_manager.cc:1403] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.218s	user 0.011s	sys 0.010s
I20260709 13:00:41.198369  1630 tablet_service.cc:1511] Processing CreateTablet for tablet d4bc96896acb4fcf8b5ec5b7206248ca (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:00:41.202811  1626 tablet_service.cc:1511] Processing CreateTablet for tablet 1e353d5ccbb3478ba62d1d60ecc3ec05 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:00:41.231992  1629 tablet_service.cc:1511] Processing CreateTablet for tablet 403bcba338a042c1b5fecfe8ac430922 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:00:41.234902  1551 tablet_service.cc:1511] Processing CreateTablet for tablet c46cf090bf9b4084b648d00f5f0053e2 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:00:41.235284  1761 raft_consensus.cc:359] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:00:41.236183  1761 raft_consensus.cc:385] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.236266  1551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c46cf090bf9b4084b648d00f5f0053e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.236640  1761 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:41.236330  1705 tablet_service.cc:1511] Processing CreateTablet for tablet c46cf090bf9b4084b648d00f5f0053e2 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:00:41.232086  1550 tablet_service.cc:1511] Processing CreateTablet for tablet d4bc96896acb4fcf8b5ec5b7206248ca (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:00:41.219720  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe2946ff23d498f98c4627ee968a4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.189191  1627 tablet_service.cc:1511] Processing CreateTablet for tablet 37b565cd27a64e96addb09389c2c7d5e (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:00:41.246456  1705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c46cf090bf9b4084b648d00f5f0053e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.248886  1630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4bc96896acb4fcf8b5ec5b7206248ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.256498  1628 tablet_service.cc:1511] Processing CreateTablet for tablet c46cf090bf9b4084b648d00f5f0053e2 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:00:41.269488  1554 tablet_service.cc:1511] Processing CreateTablet for tablet 344c88e0be9640f583140fb4aab3af58 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:00:41.269552  1627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37b565cd27a64e96addb09389c2c7d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.269654  1706 tablet_service.cc:1511] Processing CreateTablet for tablet d4bc96896acb4fcf8b5ec5b7206248ca (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:00:41.279531  1708 tablet_service.cc:1511] Processing CreateTablet for tablet 344c88e0be9640f583140fb4aab3af58 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:00:41.247789  1761 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:00:41.282056  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 344c88e0be9640f583140fb4aab3af58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.277294  1762 tablet_bootstrap.cc:492] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:41.273456  1629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 403bcba338a042c1b5fecfe8ac430922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.286392  1557 tablet_service.cc:1511] Processing CreateTablet for tablet 403bcba338a042c1b5fecfe8ac430922 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:00:41.272235  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4bc96896acb4fcf8b5ec5b7206248ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.296020  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 403bcba338a042c1b5fecfe8ac430922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.298663  1701 tablet_service.cc:1511] Processing CreateTablet for tablet 403bcba338a042c1b5fecfe8ac430922 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:00:41.258584  1625 tablet_service.cc:1511] Processing CreateTablet for tablet c2832827f30046328715699fa6ee0c52 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:00:41.301152  1556 tablet_service.cc:1511] Processing CreateTablet for tablet c2832827f30046328715699fa6ee0c52 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:00:41.302541  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2832827f30046328715699fa6ee0c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.307111  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4bc96896acb4fcf8b5ec5b7206248ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.318490  1704 tablet_service.cc:1511] Processing CreateTablet for tablet c2832827f30046328715699fa6ee0c52 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:00:41.320729  1704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2832827f30046328715699fa6ee0c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.332898  1669 heartbeater.cc:499] Master 127.0.107.254:37201 was elected leader, sending a full tablet report...
I20260709 13:00:41.335772  1700 tablet_service.cc:1511] Processing CreateTablet for tablet d5ccce7ae459467a984bd58d73f701ae (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:00:41.336148  1633 tablet_service.cc:1511] Processing CreateTablet for tablet d5ccce7ae459467a984bd58d73f701ae (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:00:41.339763  1700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5ccce7ae459467a984bd58d73f701ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.340304  1762 ts_tablet_manager.cc:1403] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.271s	user 0.018s	sys 0.001s
I20260709 13:00:41.346280  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5ccce7ae459467a984bd58d73f701ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.349602  1554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 344c88e0be9640f583140fb4aab3af58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.363101  1559 tablet_service.cc:1511] Processing CreateTablet for tablet 408b4ec829c54266b41334aac8245405 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:00:41.368183  1701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 403bcba338a042c1b5fecfe8ac430922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.369733  1553 tablet_service.cc:1511] Processing CreateTablet for tablet 520f39ecc282462782487a3651e28b44 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:00:41.333258  1558 tablet_service.cc:1511] Processing CreateTablet for tablet d5ccce7ae459467a984bd58d73f701ae (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:00:41.385144  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 408b4ec829c54266b41334aac8245405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.391381  1553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 520f39ecc282462782487a3651e28b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.393886  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5ccce7ae459467a984bd58d73f701ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.397415  1762 raft_consensus.cc:359] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:41.398607  1762 raft_consensus.cc:385] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.382004  1628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c46cf090bf9b4084b648d00f5f0053e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.412753  1555 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4f9d3ff1a04fcbba1e77b6fa3e6358 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:00:41.459394  1762 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:41.460253  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4f9d3ff1a04fcbba1e77b6fa3e6358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.460562  1762 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:41.463124  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2832827f30046328715699fa6ee0c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.414664  1556 tablet_service.cc:1511] Processing CreateTablet for tablet c129438e9d58493a93890208b25765cf (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:00:41.467272  1626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e353d5ccbb3478ba62d1d60ecc3ec05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.467962  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c129438e9d58493a93890208b25765cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.415396  1761 ts_tablet_manager.cc:1434] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.191s	user 0.014s	sys 0.021s
I20260709 13:00:41.472007  1761 tablet_bootstrap.cc:492] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:41.416617  1707 tablet_service.cc:1511] Processing CreateTablet for tablet 520f39ecc282462782487a3651e28b44 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:00:41.475728  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 520f39ecc282462782487a3651e28b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.476508  1743 heartbeater.cc:499] Master 127.0.107.254:37201 was elected leader, sending a full tablet report...
I20260709 13:00:41.483633  1761 tablet_bootstrap.cc:654] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:41.383457  1767 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.491727  1767 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:00:41.446339  1763 tablet_bootstrap.cc:492] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:41.498461  1763 ts_tablet_manager.cc:1403] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.381s	user 0.029s	sys 0.008s
I20260709 13:00:41.501473  1631 tablet_service.cc:1511] Processing CreateTablet for tablet c129438e9d58493a93890208b25765cf (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:00:41.448307  1703 tablet_service.cc:1511] Processing CreateTablet for tablet 408b4ec829c54266b41334aac8245405 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:00:41.448972  1632 tablet_service.cc:1511] Processing CreateTablet for tablet 408b4ec829c54266b41334aac8245405 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:00:41.508010  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 408b4ec829c54266b41334aac8245405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.506675  1767 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:41.523545  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 408b4ec829c54266b41334aac8245405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.524856  1762 ts_tablet_manager.cc:1434] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.175s	user 0.008s	sys 0.001s
I20260709 13:00:41.526283  1769 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.526944  1769 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:41.537022  1557 tablet_service.cc:1511] Processing CreateTablet for tablet 4c0f61db23f34217bd5089a459cbb914 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:00:41.538715  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0f61db23f34217bd5089a459cbb914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.498824  1699 tablet_service.cc:1511] Processing CreateTablet for tablet c129438e9d58493a93890208b25765cf (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:00:41.552295  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c129438e9d58493a93890208b25765cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.552363  1552 tablet_service.cc:1511] Processing CreateTablet for tablet 213d9f9af66a4ebd8c9a7b0163196a57 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:00:41.554193  1552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 213d9f9af66a4ebd8c9a7b0163196a57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.512452  1763 raft_consensus.cc:359] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:41.567971  1763 raft_consensus.cc:385] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.568527  1763 raft_consensus.cc:740] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:41.569280  1763 consensus_queue.cc:260] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:41.545886  1559 tablet_service.cc:1511] Processing CreateTablet for tablet 8a4760a45328493e87523c2ee8292599 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:00:41.526566  1762 tablet_bootstrap.cc:492] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:41.556814  1708 tablet_service.cc:1511] Processing CreateTablet for tablet 213d9f9af66a4ebd8c9a7b0163196a57 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:00:41.559386  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c129438e9d58493a93890208b25765cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.494529  1702 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4f9d3ff1a04fcbba1e77b6fa3e6358 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:00:41.559412  1704 tablet_service.cc:1511] Processing CreateTablet for tablet 4c0f61db23f34217bd5089a459cbb914 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:00:41.546612  1706 tablet_service.cc:1511] Processing CreateTablet for tablet 8a4760a45328493e87523c2ee8292599 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:00:41.577940  1634 tablet_service.cc:1511] Processing CreateTablet for tablet 520f39ecc282462782487a3651e28b44 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:00:41.601037  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 520f39ecc282462782487a3651e28b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.603724  1705 tablet_service.cc:1511] Processing CreateTablet for tablet 33657ac7590d4739b9bda3f7bb345c7b (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:00:41.605497  1769 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:41.615206  1550 tablet_service.cc:1511] Processing CreateTablet for tablet 33657ac7590d4739b9bda3f7bb345c7b (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:00:41.614513  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a4760a45328493e87523c2ee8292599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.619743  1627 tablet_service.cc:1511] Processing CreateTablet for tablet 8a4760a45328493e87523c2ee8292599 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:00:41.650265  1702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4f9d3ff1a04fcbba1e77b6fa3e6358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.656291  1625 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4f9d3ff1a04fcbba1e77b6fa3e6358 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:00:41.660542  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33657ac7590d4739b9bda3f7bb345c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.662454  1700 tablet_service.cc:1511] Processing CreateTablet for tablet e62e41b6e9a54f26a1c7c1538bdf5334 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:00:41.646862  1762 tablet_bootstrap.cc:654] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:41.675527  1629 tablet_service.cc:1511] Processing CreateTablet for tablet 213d9f9af66a4ebd8c9a7b0163196a57 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:00:41.658805  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4f9d3ff1a04fcbba1e77b6fa3e6358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.681653  1630 tablet_service.cc:1511] Processing CreateTablet for tablet 33657ac7590d4739b9bda3f7bb345c7b (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:00:41.683280  1630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33657ac7590d4739b9bda3f7bb345c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.687817  1627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a4760a45328493e87523c2ee8292599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.692363  1705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33657ac7590d4739b9bda3f7bb345c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.692252  1552 tablet_service.cc:1511] Processing CreateTablet for tablet e62e41b6e9a54f26a1c7c1538bdf5334 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:00:41.693480  1704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0f61db23f34217bd5089a459cbb914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.694111  1552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e62e41b6e9a54f26a1c7c1538bdf5334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.694463  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a4760a45328493e87523c2ee8292599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.697124  1629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 213d9f9af66a4ebd8c9a7b0163196a57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.699549  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 213d9f9af66a4ebd8c9a7b0163196a57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.700243  1700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e62e41b6e9a54f26a1c7c1538bdf5334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.699967  1628 tablet_service.cc:1511] Processing CreateTablet for tablet 4c0f61db23f34217bd5089a459cbb914 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:00:41.701300  1628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0f61db23f34217bd5089a459cbb914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.719288  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:41.719306  1761 tablet_bootstrap.cc:492] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:41.720558  1633 tablet_service.cc:1511] Processing CreateTablet for tablet e62e41b6e9a54f26a1c7c1538bdf5334 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:00:41.722440  1761 ts_tablet_manager.cc:1403] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.251s	user 0.010s	sys 0.015s
W20260709 13:00:41.730839  1533 leader_election.cc:343] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:41.732308  1701 tablet_service.cc:1511] Processing CreateTablet for tablet 8c85446f87564ae6865060bb988ee1b8 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:00:41.734987  1626 tablet_service.cc:1511] Processing CreateTablet for tablet 8c85446f87564ae6865060bb988ee1b8 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:00:41.739285  1556 tablet_service.cc:1511] Processing CreateTablet for tablet 8c85446f87564ae6865060bb988ee1b8 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:00:41.755836  1701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c85446f87564ae6865060bb988ee1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.762995  1556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c85446f87564ae6865060bb988ee1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.764326  1554 tablet_service.cc:1511] Processing CreateTablet for tablet 2792e3e29993414a8d6f55078ae13994 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:00:41.765053  1555 tablet_service.cc:1511] Processing CreateTablet for tablet 92a2b88a149f424591686a115bc1f0d6 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:00:41.773432  1558 tablet_service.cc:1511] Processing CreateTablet for tablet 50d8055bc87c4624a96f13638fe667e6 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:00:41.788956  1553 tablet_service.cc:1511] Processing CreateTablet for tablet d0049d6871cf4f95be599108abda58a5 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:00:41.792171  1761 raft_consensus.cc:359] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:41.793124  1761 raft_consensus.cc:385] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:41.793478  1761 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:41.795313  1632 tablet_service.cc:1511] Processing CreateTablet for tablet 2792e3e29993414a8d6f55078ae13994 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:00:41.795949  1704 tablet_service.cc:1511] Processing CreateTablet for tablet 2792e3e29993414a8d6f55078ae13994 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:00:41.796998  1551 tablet_service.cc:1511] Processing CreateTablet for tablet d59c4d21b6cb46edbfc2b3d50b51812a (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:00:41.797864  1704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2792e3e29993414a8d6f55078ae13994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.811997  1763 ts_tablet_manager.cc:1434] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.313s	user 0.008s	sys 0.033s
I20260709 13:00:41.813025  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:41.815213  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:00:41.816216  1681 leader_election.cc:343] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:41.816263  1554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2792e3e29993414a8d6f55078ae13994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.816891  1706 tablet_service.cc:1511] Processing CreateTablet for tablet 50d8055bc87c4624a96f13638fe667e6 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
W20260709 13:00:41.817802  1532 leader_election.cc:343] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:41.818293  1532 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:41.818598  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50d8055bc87c4624a96f13638fe667e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.813113  1699 tablet_service.cc:1511] Processing CreateTablet for tablet 92a2b88a149f424591686a115bc1f0d6 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:00:41.819314  1558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50d8055bc87c4624a96f13638fe667e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.819615  1707 tablet_service.cc:1511] Processing CreateTablet for tablet d0049d6871cf4f95be599108abda58a5 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:00:41.821734  1626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c85446f87564ae6865060bb988ee1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.823485  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:41.825030  1553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0049d6871cf4f95be599108abda58a5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:41.825479  1682 leader_election.cc:343] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:41.825940  1682 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:41.827713  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2792e3e29993414a8d6f55078ae13994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.831120  1769 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:41.832827  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e62e41b6e9a54f26a1c7c1538bdf5334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.834372  1631 tablet_service.cc:1511] Processing CreateTablet for tablet 92a2b88a149f424591686a115bc1f0d6 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:00:41.835597  1767 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:41.836314  1700 tablet_service.cc:1511] Processing CreateTablet for tablet d59c4d21b6cb46edbfc2b3d50b51812a (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:00:41.837009  1551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d59c4d21b6cb46edbfc2b3d50b51812a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.842835  1554 tablet_service.cc:1511] Processing CreateTablet for tablet 3905c76f9588440c8a836e5786f6667e (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:00:41.844025  1763 tablet_bootstrap.cc:492] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:41.845039  1554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3905c76f9588440c8a836e5786f6667e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.852687  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92a2b88a149f424591686a115bc1f0d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.853299  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92a2b88a149f424591686a115bc1f0d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.845366  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92a2b88a149f424591686a115bc1f0d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.863260  1700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d59c4d21b6cb46edbfc2b3d50b51812a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.862818  1761 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:41.864570  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0049d6871cf4f95be599108abda58a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.884146  1703 tablet_service.cc:1511] Processing CreateTablet for tablet 3905c76f9588440c8a836e5786f6667e (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:00:41.903539  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3905c76f9588440c8a836e5786f6667e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.922825  1762 tablet_bootstrap.cc:492] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:41.922973  1628 tablet_service.cc:1511] Processing CreateTablet for tablet 50d8055bc87c4624a96f13638fe667e6 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:00:41.923497  1762 ts_tablet_manager.cc:1403] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.397s	user 0.011s	sys 0.014s
I20260709 13:00:41.924371  1628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50d8055bc87c4624a96f13638fe667e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.924569  1761 ts_tablet_manager.cc:1434] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.179s	user 0.009s	sys 0.000s
I20260709 13:00:41.932384  1777 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.934129  1777 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:41.936383  1767 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.936932  1767 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:41.938560  1515 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:41.942883  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:41.940205  1767 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:41.950740  1777 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:41.944228  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:41.964576  1557 tablet_service.cc:1511] Processing CreateTablet for tablet d1256ff04335423286d7c081ec776061 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:00:41.965067  1552 tablet_service.cc:1511] Processing CreateTablet for tablet 5a08948b1fac416d8d27be0561a00db4 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:00:41.966256  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1256ff04335423286d7c081ec776061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.967060  1699 tablet_service.cc:1511] Processing CreateTablet for tablet d1256ff04335423286d7c081ec776061 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:00:41.968616  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1256ff04335423286d7c081ec776061. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:41.950706  1533 leader_election.cc:343] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:41.975376  1552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a08948b1fac416d8d27be0561a00db4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:41.975612  1532 leader_election.cc:343] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:41.979187  1532 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:41.980120  1767 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:41.981148  1550 tablet_service.cc:1511] Processing CreateTablet for tablet ca83cd5e36eb4e22b75011b87bfcd49d (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:00:41.989149  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca83cd5e36eb4e22b75011b87bfcd49d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:41.926813  1762 raft_consensus.cc:359] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.011356  1762 raft_consensus.cc:385] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.012012  1762 raft_consensus.cc:740] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.007246  1761 tablet_bootstrap.cc:492] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:42.015357  1762 consensus_queue.cc:260] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.026033  1762 ts_tablet_manager.cc:1434] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.102s	user 0.008s	sys 0.001s
I20260709 13:00:42.027375  1762 tablet_bootstrap.cc:492] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:42.021282  1699 tablet_service.cc:1511] Processing CreateTablet for tablet ca83cd5e36eb4e22b75011b87bfcd49d (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:00:42.036252  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca83cd5e36eb4e22b75011b87bfcd49d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.032997  1559 tablet_service.cc:1511] Processing CreateTablet for tablet 71b0b66754d049cfbccb38fbb16ef3cd (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:00:42.044162  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71b0b66754d049cfbccb38fbb16ef3cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.047935  1626 tablet_service.cc:1511] Processing CreateTablet for tablet d0049d6871cf4f95be599108abda58a5 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:00:42.055908  1761 tablet_bootstrap.cc:654] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.066632  1630 tablet_service.cc:1511] Processing CreateTablet for tablet d59c4d21b6cb46edbfc2b3d50b51812a (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:00:42.069278  1632 tablet_service.cc:1511] Processing CreateTablet for tablet 3905c76f9588440c8a836e5786f6667e (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:00:42.071182  1626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0049d6871cf4f95be599108abda58a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.072974  1630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d59c4d21b6cb46edbfc2b3d50b51812a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.079275  1762 tablet_bootstrap.cc:654] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.088917  1763 tablet_bootstrap.cc:654] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.090354  1769 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.094755  1769 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.099048  1625 tablet_service.cc:1511] Processing CreateTablet for tablet d1256ff04335423286d7c081ec776061 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:00:42.099779  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:42.100833  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1256ff04335423286d7c081ec776061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.104317  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:42.105101  1644 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
W20260709 13:00:42.108011  1681 leader_election.cc:343] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.108914  1769 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:42.109665  1682 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:00:42.110236  1703 tablet_service.cc:1511] Processing CreateTablet for tablet 768b897f18a644e8b1e8f8ed1c8b616f (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:00:42.111757  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 768b897f18a644e8b1e8f8ed1c8b616f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.112723  1555 tablet_service.cc:1511] Processing CreateTablet for tablet 768b897f18a644e8b1e8f8ed1c8b616f (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:00:42.114681  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 768b897f18a644e8b1e8f8ed1c8b616f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.116828  1707 tablet_service.cc:1511] Processing CreateTablet for tablet 71b0b66754d049cfbccb38fbb16ef3cd (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:00:42.118423  1769 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:42.118883  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71b0b66754d049cfbccb38fbb16ef3cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.119283  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3905c76f9588440c8a836e5786f6667e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.163010  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:42.164070  1761 tablet_bootstrap.cc:492] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
W20260709 13:00:42.165006  1607 leader_election.cc:343] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.118969  1769 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.166882  1769 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:42.139954  1633 tablet_service.cc:1511] Processing CreateTablet for tablet 71b0b66754d049cfbccb38fbb16ef3cd (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:00:42.137656  1705 tablet_service.cc:1511] Processing CreateTablet for tablet 5a08948b1fac416d8d27be0561a00db4 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:00:42.168341  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71b0b66754d049cfbccb38fbb16ef3cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.132365  1634 tablet_service.cc:1511] Processing CreateTablet for tablet ca83cd5e36eb4e22b75011b87bfcd49d (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:00:42.170032  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca83cd5e36eb4e22b75011b87bfcd49d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.124608  1625 tablet_service.cc:1511] Processing CreateTablet for tablet 5a08948b1fac416d8d27be0561a00db4 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:00:42.171770  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a08948b1fac416d8d27be0561a00db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.168341  1705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a08948b1fac416d8d27be0561a00db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.178769  1763 tablet_bootstrap.cc:492] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:42.179409  1763 ts_tablet_manager.cc:1403] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.339s	user 0.012s	sys 0.010s
I20260709 13:00:42.180347  1769 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.184275  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:00:42.186002  1681 leader_election.cc:343] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.165047  1761 ts_tablet_manager.cc:1403] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.216s	user 0.012s	sys 0.012s
I20260709 13:00:42.196372  1761 raft_consensus.cc:359] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.197563  1761 raft_consensus.cc:385] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.184242  1763 raft_consensus.cc:359] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:00:42.205605  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:42.206347  1769 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:42.207244  1718 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:00:42.200879  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:42.225368  1644 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:42.198530  1763 raft_consensus.cc:385] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.226666  1763 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.228096  1559 tablet_service.cc:1511] Processing CreateTablet for tablet ac4c23f0e4f742e38ca7a5ebd7778dfa (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:00:42.229584  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac4c23f0e4f742e38ca7a5ebd7778dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.209167  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:42.227839  1763 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:00:42.158569  1631 tablet_service.cc:1511] Processing CreateTablet for tablet 768b897f18a644e8b1e8f8ed1c8b616f (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:00:42.235242  1629 tablet_service.cc:1511] Processing CreateTablet for tablet ac4c23f0e4f742e38ca7a5ebd7778dfa (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:00:42.197930  1761 raft_consensus.cc:740] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.237372  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 768b897f18a644e8b1e8f8ed1c8b616f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.230628  1762 tablet_bootstrap.cc:492] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:42.243526  1762 ts_tablet_manager.cc:1403] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.216s	user 0.015s	sys 0.010s
I20260709 13:00:42.245699  1761 consensus_queue.cc:260] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.242599  1629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac4c23f0e4f742e38ca7a5ebd7778dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.254607  1703 tablet_service.cc:1511] Processing CreateTablet for tablet ac4c23f0e4f742e38ca7a5ebd7778dfa (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:00:42.255648  1761 ts_tablet_manager.cc:1434] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.068s	user 0.008s	sys 0.002s
I20260709 13:00:42.256311  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac4c23f0e4f742e38ca7a5ebd7778dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.257700  1761 tablet_bootstrap.cc:492] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:42.266000  1763 ts_tablet_manager.cc:1434] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.086s	user 0.009s	sys 0.000s
I20260709 13:00:42.257125  1762 raft_consensus.cc:359] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.266933  1762 raft_consensus.cc:385] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.270900  1767 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.278770  1767 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.277524  1763 tablet_bootstrap.cc:492] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:42.276953  1627 tablet_service.cc:1511] Processing CreateTablet for tablet 70a0a5f8de344870be97d83cec17c047 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:00:42.283875  1767 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:42.286996  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:42.289448  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:42.290274  1718 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 1: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:00:42.291296  1627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70a0a5f8de344870be97d83cec17c047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.292318  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:42.295300  1555 tablet_service.cc:1511] Processing CreateTablet for tablet e6dc6e12b7924255b756439991155e88 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:00:42.295675  1634 tablet_service.cc:1511] Processing CreateTablet for tablet e6dc6e12b7924255b756439991155e88 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:00:42.296758  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6dc6e12b7924255b756439991155e88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.297436  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6dc6e12b7924255b756439991155e88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.278508  1762 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.300446  1767 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:42.301390  1706 tablet_service.cc:1511] Processing CreateTablet for tablet e6dc6e12b7924255b756439991155e88 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:00:42.301713  1699 tablet_service.cc:1511] Processing CreateTablet for tablet 70a0a5f8de344870be97d83cec17c047 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:00:42.305444  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6dc6e12b7924255b756439991155e88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.306211  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70a0a5f8de344870be97d83cec17c047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.310031  1762 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.321000  1644 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:42.321707  1777 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:42.323359  1682 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:00:42.323643  1554 tablet_service.cc:1511] Processing CreateTablet for tablet 70a0a5f8de344870be97d83cec17c047 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:00:42.325434  1761 tablet_bootstrap.cc:654] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.325350  1769 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:42.328790  1632 tablet_service.cc:1511] Processing CreateTablet for tablet b3fdbd88a9d440e59ddd4e49c79ab2f2 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:00:42.330715  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3fdbd88a9d440e59ddd4e49c79ab2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.340071  1763 tablet_bootstrap.cc:654] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.341488  1767 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:42.360289  1554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70a0a5f8de344870be97d83cec17c047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.365692  1769 raft_consensus.cc:697] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:42.366750  1769 consensus_queue.cc:237] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.368594  1707 tablet_service.cc:1511] Processing CreateTablet for tablet b3fdbd88a9d440e59ddd4e49c79ab2f2 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:00:42.371316  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3fdbd88a9d440e59ddd4e49c79ab2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.391651  1762 ts_tablet_manager.cc:1434] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.148s	user 0.010s	sys 0.000s
I20260709 13:00:42.392313  1559 tablet_service.cc:1511] Processing CreateTablet for tablet 84f86a4ba55242bd8d2459c6b9df56d2 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:00:42.399114  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84f86a4ba55242bd8d2459c6b9df56d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.394779  1627 tablet_service.cc:1511] Processing CreateTablet for tablet 84f86a4ba55242bd8d2459c6b9df56d2 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:00:42.401472  1627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84f86a4ba55242bd8d2459c6b9df56d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.397526  1761 tablet_bootstrap.cc:492] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:42.335000  1550 tablet_service.cc:1511] Processing CreateTablet for tablet b3fdbd88a9d440e59ddd4e49c79ab2f2 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:00:42.407335  1761 ts_tablet_manager.cc:1403] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.150s	user 0.011s	sys 0.009s
I20260709 13:00:42.412483  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3fdbd88a9d440e59ddd4e49c79ab2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.407967  1762 tablet_bootstrap.cc:492] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:42.417044  1625 tablet_service.cc:1511] Processing CreateTablet for tablet 2a8ba0cc212d41e5b0ad586b885d07a7 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:00:42.419441  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a8ba0cc212d41e5b0ad586b885d07a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.420683  1761 raft_consensus.cc:359] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.422209  1761 raft_consensus.cc:385] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.425104  1761 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.424958  1703 tablet_service.cc:1511] Processing CreateTablet for tablet 2a8ba0cc212d41e5b0ad586b885d07a7 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:00:42.425014  1763 tablet_bootstrap.cc:492] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:42.427454  1763 ts_tablet_manager.cc:1403] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.150s	user 0.023s	sys 0.026s
I20260709 13:00:42.429488  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a8ba0cc212d41e5b0ad586b885d07a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.433128  1763 raft_consensus.cc:359] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.434093  1763 raft_consensus.cc:385] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.434569  1763 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.435621  1763 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.436343  1465 catalog_manager.cc:5697] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:42.437955  1763 ts_tablet_manager.cc:1434] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:00:42.446367  1557 tablet_service.cc:1511] Processing CreateTablet for tablet bb58673aa80747e58aa935f4fed2212d (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:00:42.448916  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb58673aa80747e58aa935f4fed2212d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.417855  1555 tablet_service.cc:1511] Processing CreateTablet for tablet 2a8ba0cc212d41e5b0ad586b885d07a7 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:00:42.457774  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a8ba0cc212d41e5b0ad586b885d07a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.429018  1699 tablet_service.cc:1511] Processing CreateTablet for tablet 84f86a4ba55242bd8d2459c6b9df56d2 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:00:42.459987  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84f86a4ba55242bd8d2459c6b9df56d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.466708  1631 tablet_service.cc:1511] Processing CreateTablet for tablet bb58673aa80747e58aa935f4fed2212d (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:00:42.469393  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb58673aa80747e58aa935f4fed2212d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.470793  1769 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.471550  1769 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.426234  1761 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.475275  1767 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.475757  1762 tablet_bootstrap.cc:654] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.475549  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:42.475806  1767 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:42.477844  1681 leader_election.cc:343] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.482159  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:42.484061  1533 leader_election.cc:343] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.485236  1767 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:42.497967  1761 ts_tablet_manager.cc:1434] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.086s	user 0.008s	sys 0.004s
I20260709 13:00:42.499661  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:42.503347  1706 tablet_service.cc:1511] Processing CreateTablet for tablet bb58673aa80747e58aa935f4fed2212d (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:00:42.504777  1763 tablet_bootstrap.cc:492] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:42.505249  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb58673aa80747e58aa935f4fed2212d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:42.505764  1532 leader_election.cc:343] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.506721  1532 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:42.508193  1792 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:42.511193  1761 tablet_bootstrap.cc:492] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:42.518162  1769 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:42.535379  1761 tablet_bootstrap.cc:654] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.535228  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:42.543290  1550 tablet_service.cc:1511] Processing CreateTablet for tablet d883c107e7bd4b6daa6d09672acc2b23 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 13:00:42.548111  1682 leader_election.cc:343] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.548631  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:42.549417  1769 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:42.556551  1632 tablet_service.cc:1511] Processing CreateTablet for tablet d883c107e7bd4b6daa6d09672acc2b23 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:00:42.558635  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d883c107e7bd4b6daa6d09672acc2b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.560137  1762 tablet_bootstrap.cc:492] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:42.560379  1763 tablet_bootstrap.cc:654] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.560729  1762 ts_tablet_manager.cc:1403] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.153s	user 0.030s	sys 0.033s
I20260709 13:00:42.567834  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d883c107e7bd4b6daa6d09672acc2b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.580569  1762 raft_consensus.cc:359] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.581784  1762 raft_consensus.cc:385] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.581630  1559 tablet_service.cc:1511] Processing CreateTablet for tablet fbfadedb9a5e4ca0abc25384a524c546 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:00:42.582145  1762 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.583473  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbfadedb9a5e4ca0abc25384a524c546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.583364  1762 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.581629  1625 tablet_service.cc:1511] Processing CreateTablet for tablet fbfadedb9a5e4ca0abc25384a524c546 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:00:42.588232  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbfadedb9a5e4ca0abc25384a524c546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.581525  1761 tablet_bootstrap.cc:492] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:42.591646  1761 ts_tablet_manager.cc:1403] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.093s	user 0.020s	sys 0.006s
I20260709 13:00:42.595259  1699 tablet_service.cc:1511] Processing CreateTablet for tablet fbfadedb9a5e4ca0abc25384a524c546 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:00:42.596844  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbfadedb9a5e4ca0abc25384a524c546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.592345  1762 ts_tablet_manager.cc:1434] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.031s	user 0.011s	sys 0.000s
I20260709 13:00:42.598181  1762 tablet_bootstrap.cc:492] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:42.592504  1706 tablet_service.cc:1511] Processing CreateTablet for tablet d883c107e7bd4b6daa6d09672acc2b23 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:00:42.609243  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d883c107e7bd4b6daa6d09672acc2b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.609704  1769 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.610208  1769 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.614629  1769 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:42.614884  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:42.616664  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:42.615492  1761 raft_consensus.cc:359] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.627800  1761 raft_consensus.cc:385] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.628152  1761 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
W20260709 13:00:42.628682  1681 leader_election.cc:343] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.628937  1555 tablet_service.cc:1511] Processing CreateTablet for tablet 6bdefe70d276420ca0283414ab16fab9 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:00:42.629031  1761 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:42.629913  1682 leader_election.cc:343] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.630542  1682 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:42.631054  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bdefe70d276420ca0283414ab16fab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.635288  1769 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:42.636340  1632 tablet_service.cc:1511] Processing CreateTablet for tablet 6bdefe70d276420ca0283414ab16fab9 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:00:42.638094  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bdefe70d276420ca0283414ab16fab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.640553  1699 tablet_service.cc:1511] Processing CreateTablet for tablet 6bdefe70d276420ca0283414ab16fab9 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:00:42.642624  1763 tablet_bootstrap.cc:492] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:42.643142  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bdefe70d276420ca0283414ab16fab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.643211  1763 ts_tablet_manager.cc:1403] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.139s	user 0.015s	sys 0.002s
I20260709 13:00:42.646831  1625 tablet_service.cc:1511] Processing CreateTablet for tablet 250e1f9f055e4fda9987138306f2e0b7 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:00:42.648547  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 250e1f9f055e4fda9987138306f2e0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.641861  1559 tablet_service.cc:1511] Processing CreateTablet for tablet 250e1f9f055e4fda9987138306f2e0b7 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:00:42.651796  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 250e1f9f055e4fda9987138306f2e0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.656729  1761 ts_tablet_manager.cc:1434] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.063s	user 0.004s	sys 0.004s
I20260709 13:00:42.658380  1761 tablet_bootstrap.cc:492] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:42.659180  1762 tablet_bootstrap.cc:654] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.659874  1763 raft_consensus.cc:359] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.662707  1763 raft_consensus.cc:385] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.663278  1763 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.664341  1763 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.666851  1763 ts_tablet_manager.cc:1434] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.002s
I20260709 13:00:42.693888  1703 tablet_service.cc:1511] Processing CreateTablet for tablet d29864f650d0477e8dd2e7f82eea9e47 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:00:42.688504  1557 tablet_service.cc:1511] Processing CreateTablet for tablet d29864f650d0477e8dd2e7f82eea9e47 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:00:42.694413  1631 tablet_service.cc:1511] Processing CreateTablet for tablet d29864f650d0477e8dd2e7f82eea9e47 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:00:42.699810  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d29864f650d0477e8dd2e7f82eea9e47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.699465  1706 tablet_service.cc:1511] Processing CreateTablet for tablet 250e1f9f055e4fda9987138306f2e0b7 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:00:42.701196  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d29864f650d0477e8dd2e7f82eea9e47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.699507  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d29864f650d0477e8dd2e7f82eea9e47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.716352  1761 tablet_bootstrap.cc:654] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.676116  1777 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.724349  1777 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:00:42.726528  1625 tablet_service.cc:1511] Processing CreateTablet for tablet 9df7dc71f943412e83fbc8d4f788c9f2 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:00:42.727510  1763 tablet_bootstrap.cc:492] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:42.728708  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:42.729900  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:00:42.731003  1607 leader_election.cc:343] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.731262  1569 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:42.726850  1777 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:42.730057  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 250e1f9f055e4fda9987138306f2e0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.739660  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:42.741456  1777 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:42.741936  1777 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.742364  1777 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:42.745514  1762 tablet_bootstrap.cc:492] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:42.729526  1550 tablet_service.cc:1511] Processing CreateTablet for tablet 9df7dc71f943412e83fbc8d4f788c9f2 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:00:42.751483  1559 tablet_service.cc:1511] Processing CreateTablet for tablet 5448ccc31ba14113b3d56168031c6714 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:00:42.752676  1707 tablet_service.cc:1511] Processing CreateTablet for tablet 5448ccc31ba14113b3d56168031c6714 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:00:42.760599  1762 ts_tablet_manager.cc:1403] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.163s	user 0.016s	sys 0.018s
I20260709 13:00:42.760921  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5448ccc31ba14113b3d56168031c6714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.754345  1777 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:00:42.755851  1625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9df7dc71f943412e83fbc8d4f788c9f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.756922  1763 tablet_bootstrap.cc:654] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.760017  1550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9df7dc71f943412e83fbc8d4f788c9f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.766829  1699 tablet_service.cc:1511] Processing CreateTablet for tablet 9df7dc71f943412e83fbc8d4f788c9f2 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:00:42.764871  1632 tablet_service.cc:1511] Processing CreateTablet for tablet 5448ccc31ba14113b3d56168031c6714 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:00:42.773875  1559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5448ccc31ba14113b3d56168031c6714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.774668  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5448ccc31ba14113b3d56168031c6714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.777175  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:42.778045  1569 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:42.778954  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9df7dc71f943412e83fbc8d4f788c9f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.764111  1761 tablet_bootstrap.cc:492] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:42.787302  1761 ts_tablet_manager.cc:1403] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.129s	user 0.016s	sys 0.004s
I20260709 13:00:42.764515  1762 raft_consensus.cc:359] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.770258  1627 tablet_service.cc:1511] Processing CreateTablet for tablet b7808a4bebf84e598198ed51ea156981 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:00:42.789484  1763 tablet_bootstrap.cc:492] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:42.789484  1627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7808a4bebf84e598198ed51ea156981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.790627  1763 ts_tablet_manager.cc:1403] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.011s
I20260709 13:00:42.792835  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:00:42.794864  1607 leader_election.cc:343] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.795492  1777 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:42.795398  1763 raft_consensus.cc:359] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.796411  1762 raft_consensus.cc:385] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.799582  1762 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.800606  1762 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.798228  1706 tablet_service.cc:1511] Processing CreateTablet for tablet b7808a4bebf84e598198ed51ea156981 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:00:42.803296  1557 tablet_service.cc:1511] Processing CreateTablet for tablet 3f22f7f5cede49bea2083e8905932fb7 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:00:42.805035  1557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f22f7f5cede49bea2083e8905932fb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.804689  1631 tablet_service.cc:1511] Processing CreateTablet for tablet 3f22f7f5cede49bea2083e8905932fb7 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:00:42.796411  1763 raft_consensus.cc:385] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.812026  1555 tablet_service.cc:1511] Processing CreateTablet for tablet b7808a4bebf84e598198ed51ea156981 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:00:42.806213  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f22f7f5cede49bea2083e8905932fb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.806470  1703 tablet_service.cc:1511] Processing CreateTablet for tablet 3f22f7f5cede49bea2083e8905932fb7 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:00:42.813823  1555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7808a4bebf84e598198ed51ea156981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.805176  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7808a4bebf84e598198ed51ea156981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.812273  1763 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.827703  1763 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.830433  1769 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:42.831003  1763 ts_tablet_manager.cc:1434] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.040s	user 0.010s	sys 0.000s
I20260709 13:00:42.790673  1761 raft_consensus.cc:359] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.821359  1762 ts_tablet_manager.cc:1434] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.001s
I20260709 13:00:42.835675  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f22f7f5cede49bea2083e8905932fb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.820402  1554 tablet_service.cc:1511] Processing CreateTablet for tablet 4594cfed297144b39e1a29473c9707d9 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:00:42.840267  1761 raft_consensus.cc:385] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.840938  1761 raft_consensus.cc:740] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.841427  1554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4594cfed297144b39e1a29473c9707d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.842221  1761 consensus_queue.cc:260] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.843840  1569 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:42.847951  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:42.856117  1632 tablet_service.cc:1511] Processing CreateTablet for tablet 4594cfed297144b39e1a29473c9707d9 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:00:42.857458  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4594cfed297144b39e1a29473c9707d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:42.861296  1777 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:42.861862  1777 raft_consensus.cc:697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:42.862690  1777 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:00:42.864182  1762 tablet_bootstrap.cc:492] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:42.871657  1792 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.864517  1761 ts_tablet_manager.cc:1434] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.077s	user 0.005s	sys 0.003s
I20260709 13:00:42.872225  1792 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.876714  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:42.877813  1761 tablet_bootstrap.cc:492] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:42.878319  1792 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:42.878597  1763 tablet_bootstrap.cc:492] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
W20260709 13:00:42.879225  1533 leader_election.cc:343] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.881767  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:42.884341  1699 tablet_service.cc:1511] Processing CreateTablet for tablet 4594cfed297144b39e1a29473c9707d9 (DEFAULT_TABLE table=test-workload [id=4e07af9de3724e878ee758dbc1188478]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:00:42.885931  1699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4594cfed297144b39e1a29473c9707d9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:00:42.920449  1532 leader_election.cc:343] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.923511  1532 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:42.925514  1762 tablet_bootstrap.cc:654] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.925361  1792 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:42.937700  1803 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.938503  1803 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.941965  1803 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:42.947239  1792 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.948101  1792 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.949309  1761 tablet_bootstrap.cc:654] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.950467  1763 tablet_bootstrap.cc:654] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:42.951124  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:42.952283  1643 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:42.954691  1533 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:42.959223  1767 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:42.959867  1767 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.960435  1767 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:42.966162  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:00:42.968487  1532 leader_election.cc:343] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.969383  1792 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:42.970510  1515 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:42.972127  1762 tablet_bootstrap.cc:492] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:42.972661  1762 ts_tablet_manager.cc:1403] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.109s	user 0.023s	sys 0.000s
I20260709 13:00:42.973541  1767 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:42.971274  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 123437us
I20260709 13:00:42.976750  1762 raft_consensus.cc:359] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:42.987409  1762 raft_consensus.cc:385] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:42.985663  1763 tablet_bootstrap.cc:492] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:42.987866  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:42.988855  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:42.990374  1569 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:00:42.993275  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:00:42.995266  1607 leader_election.cc:343] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:42.995313  1532 leader_election.cc:343] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:42.996134  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:42.996832  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:42.997033  1777 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:42.997491  1767 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:42.994383  1465 catalog_manager.cc:5697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:42.987911  1762 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:42.999006  1762 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.001345  1762 ts_tablet_manager.cc:1434] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.028s	user 0.010s	sys 0.000s
I20260709 13:00:43.003466  1762 tablet_bootstrap.cc:492] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:43.010027  1761 tablet_bootstrap.cc:492] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:43.010573  1761 ts_tablet_manager.cc:1403] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.138s	user 0.000s	sys 0.025s
I20260709 13:00:43.011634  1777 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:42.988866  1763 ts_tablet_manager.cc:1403] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.149s	user 0.019s	sys 0.007s
I20260709 13:00:43.018949  1763 raft_consensus.cc:359] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.019749  1763 raft_consensus.cc:385] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.020093  1763 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.028024  1763 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.032435  1763 ts_tablet_manager.cc:1434] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.001s
I20260709 13:00:43.014319  1761 raft_consensus.cc:359] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.033061  1769 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:43.027137  1762 tablet_bootstrap.cc:654] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.033293  1761 raft_consensus.cc:385] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.034533  1761 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.036036  1761 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.025985  1644 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:43.043952  1533 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:43.045622  1763 tablet_bootstrap.cc:492] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:43.045804  1767 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:43.047132  1767 raft_consensus.cc:697] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:43.049018  1767 consensus_queue.cc:237] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.049041  1761 ts_tablet_manager.cc:1434] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.005s
I20260709 13:00:43.052639  1761 tablet_bootstrap.cc:492] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:43.066192  1777 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.066847  1777 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.070477  1761 tablet_bootstrap.cc:654] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.076308  1777 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.081447  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:43.081310  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:43.082294  1568 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
W20260709 13:00:43.083523  1607 leader_election.cc:343] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.084353  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:43.085640  1777 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:43.086225  1777 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.086560  1777 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.093245  1763 tablet_bootstrap.cc:654] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.096845  1762 tablet_bootstrap.cc:492] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:43.097646  1762 ts_tablet_manager.cc:1403] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.094s	user 0.020s	sys 0.010s
I20260709 13:00:43.101572  1762 raft_consensus.cc:359] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:00:43.105690  1466 catalog_manager.cc:5697] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:43.102753  1762 raft_consensus.cc:385] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.112429  1767 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.112677  1762 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.105201  1777 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.113204  1767 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.116744  1762 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:00:43.118623  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:43.120837  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:00:43.122311  1607 leader_election.cc:343] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:43.122651  1533 leader_election.cc:343] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.123710  1762 ts_tablet_manager.cc:1434] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 13:00:43.124764  1762 tablet_bootstrap.cc:492] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:43.124913  1767 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.119678  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:43.127466  1777 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.127635  1569 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.131076  1761 tablet_bootstrap.cc:492] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:43.131657  1761 ts_tablet_manager.cc:1403] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.080s	user 0.025s	sys 0.007s
I20260709 13:00:43.132205  1763 tablet_bootstrap.cc:492] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:43.132900  1763 ts_tablet_manager.cc:1403] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.087s	user 0.027s	sys 0.020s
I20260709 13:00:43.135514  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:43.136224  1718 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:43.136485  1761 raft_consensus.cc:359] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.137446  1761 raft_consensus.cc:385] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.137699  1532 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:43.137445  1763 raft_consensus.cc:359] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.137993  1761 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.138866  1763 raft_consensus.cc:385] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.139218  1763 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.139453  1767 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:43.136196  1762 tablet_bootstrap.cc:654] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.140036  1767 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.140013  1761 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.146369  1767 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.141618  1763 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.147133  1569 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:43.148810  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:43.150091  1777 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:43.150568  1777 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:43.150481  1763 ts_tablet_manager.cc:1434] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.002s
I20260709 13:00:43.151978  1777 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.154007  1761 ts_tablet_manager.cc:1434] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 13:00:43.160715  1761 tablet_bootstrap.cc:492] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:43.152560  1763 tablet_bootstrap.cc:492] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:43.157634  1767 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.165134  1767 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.166324  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:43.167286  1643 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.167387  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:43.168332  1718 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.175407  1763 tablet_bootstrap.cc:654] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.179132  1813 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.181231  1813 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.186260  1761 tablet_bootstrap.cc:654] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.186371  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:43.188531  1762 tablet_bootstrap.cc:492] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:43.189357  1762 ts_tablet_manager.cc:1403] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.065s	user 0.016s	sys 0.001s
I20260709 13:00:43.192241  1718 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
W20260709 13:00:43.194204  1681 leader_election.cc:343] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.194561  1532 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:43.195739  1813 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:43.196318  1767 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:43.197291  1767 raft_consensus.cc:697] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:43.197383  1762 raft_consensus.cc:359] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.198182  1762 raft_consensus.cc:385] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.197984  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:43.198354  1767 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.199596  1762 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
W20260709 13:00:43.200922  1682 leader_election.cc:343] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.201453  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:43.201891  1643 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:43.202280  1813 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:43.201753  1762 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.205305  1762 ts_tablet_manager.cc:1434] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.002s
I20260709 13:00:43.207257  1762 tablet_bootstrap.cc:492] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:43.209331  1763 tablet_bootstrap.cc:492] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:43.210271  1763 ts_tablet_manager.cc:1403] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.058s	user 0.015s	sys 0.003s
I20260709 13:00:43.213953  1465 catalog_manager.cc:5697] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:43.216943  1763 raft_consensus.cc:359] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.218787  1763 raft_consensus.cc:385] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.219245  1763 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.220376  1763 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.221873  1761 tablet_bootstrap.cc:492] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:43.222584  1761 ts_tablet_manager.cc:1403] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.062s	user 0.015s	sys 0.000s
I20260709 13:00:43.227291  1761 raft_consensus.cc:359] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.227829  1762 tablet_bootstrap.cc:654] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.228209  1761 raft_consensus.cc:385] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.228650  1761 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.229640  1761 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.233451  1761 ts_tablet_manager.cc:1434] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 13:00:43.233780  1777 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.234851  1763 ts_tablet_manager.cc:1434] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.024s	user 0.013s	sys 0.000s
I20260709 13:00:43.234881  1761 tablet_bootstrap.cc:492] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:43.234787  1777 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.236380  1763 tablet_bootstrap.cc:492] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:43.237718  1777 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.238498  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:43.239876  1569 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:43.243263  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:43.244130  1718 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:43.248399  1761 tablet_bootstrap.cc:654] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.258648  1762 tablet_bootstrap.cc:492] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:43.259912  1762 ts_tablet_manager.cc:1403] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.053s	user 0.021s	sys 0.008s
I20260709 13:00:43.262585  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:43.264075  1761 tablet_bootstrap.cc:492] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:43.263751  1762 raft_consensus.cc:359] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.264230  1777 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:43.264478  1762 raft_consensus.cc:385] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.264640  1777 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.264745  1761 ts_tablet_manager.cc:1403] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.004s
I20260709 13:00:43.264776  1762 raft_consensus.cc:740] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.265030  1777 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.265924  1465 catalog_manager.cc:5697] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:43.268961  1761 raft_consensus.cc:359] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.270164  1761 raft_consensus.cc:385] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.270602  1761 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.272220  1761 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.276067  1761 ts_tablet_manager.cc:1434] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.011s	user 0.012s	sys 0.000s
I20260709 13:00:43.276890  1777 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.277051  1761 tablet_bootstrap.cc:492] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:43.279594  1762 consensus_queue.cc:260] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.285275  1761 tablet_bootstrap.cc:654] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.285506  1763 tablet_bootstrap.cc:654] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.286763  1762 ts_tablet_manager.cc:1434] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 13:00:43.288136  1762 tablet_bootstrap.cc:492] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:43.293076  1777 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.295344  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:43.296075  1569 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.301170  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:43.302644  1762 tablet_bootstrap.cc:654] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.303314  1718 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.309870  1813 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.310631  1813 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.311151  1717 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.313169  1813 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:43.316866  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:43.317701  1568 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:43.319594  1681 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:43.322228  1813 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:43.322651  1813 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.323067  1813 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.322821  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:43.325213  1682 leader_election.cc:343] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.332460  1761 tablet_bootstrap.cc:492] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:43.333030  1761 ts_tablet_manager.cc:1403] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.056s	user 0.030s	sys 0.019s
I20260709 13:00:43.334409  1769 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.334990  1769 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.336459  1761 raft_consensus.cc:359] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.337113  1761 raft_consensus.cc:385] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.337646  1761 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.339100  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:43.338982  1761 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.339797  1568 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:43.341248  1681 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:43.341542  1761 ts_tablet_manager.cc:1434] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:00:43.342327  1810 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:43.343041  1761 tablet_bootstrap.cc:492] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:43.343212  1810 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.343691  1810 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.345219  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:43.347980  1777 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:43.337329  1769 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:43.352515  1761 tablet_bootstrap.cc:654] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:43.363276  1682 leader_election.cc:343] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.367372  1761 tablet_bootstrap.cc:492] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:43.368124  1761 ts_tablet_manager.cc:1403] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.006s
I20260709 13:00:43.371358  1761 raft_consensus.cc:359] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.372895  1761 raft_consensus.cc:385] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.373668  1761 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.372378  1777 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:43.374989  1761 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.375928  1810 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.378170  1761 ts_tablet_manager.cc:1434] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:00:43.380146  1767 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.395711  1767 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.398293  1792 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.398901  1767 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.398816  1813 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.400213  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:43.380707  1718 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:43.400876  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:43.400926  1761 tablet_bootstrap.cc:492] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
W20260709 13:00:43.402269  1533 leader_election.cc:343] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.393731  1763 tablet_bootstrap.cc:492] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:43.398854  1792 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:43.402843  1532 leader_election.cc:343] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.403573  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:43.404119  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:43.382297  1569 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:43.405366  1777 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:43.405846  1777 raft_consensus.cc:697] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:43.406796  1767 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:43.407020  1777 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.407963  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:43.408300  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:43.408759  1718 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:43.409251  1643 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:43.409982  1532 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:43.411448  1761 tablet_bootstrap.cc:654] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.404443  1792 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.411386  1767 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:43.412285  1763 ts_tablet_manager.cc:1403] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.176s	user 0.008s	sys 0.013s
I20260709 13:00:43.412226  1767 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.414054  1767 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.415154  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:43.415762  1569 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.416144  1763 raft_consensus.cc:359] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.416815  1763 raft_consensus.cc:385] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.417093  1763 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.417289  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:43.417726  1813 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:43.417814  1763 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:43.419458  1682 leader_election.cc:343] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.421044  1763 ts_tablet_manager.cc:1434] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:00:43.422098  1763 tablet_bootstrap.cc:492] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:43.424072  1569 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:43.425959  1681 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:43.428810  1813 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:43.429389  1813 raft_consensus.cc:697] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:43.430179  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:43.430831  1643 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.430179  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:43.434414  1568 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.435281  1763 tablet_bootstrap.cc:654] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.440277  1762 tablet_bootstrap.cc:492] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:43.440685  1762 ts_tablet_manager.cc:1403] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.153s	user 0.011s	sys 0.006s
I20260709 13:00:43.432034  1810 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:43.430331  1813 consensus_queue.cc:237] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.446465  1762 raft_consensus.cc:359] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.447849  1762 raft_consensus.cc:385] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.448133  1762 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.448990  1792 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.449142  1762 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.449517  1792 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.452136  1792 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.453312  1762 ts_tablet_manager.cc:1434] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:00:43.453501  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:43.454280  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:43.454780  1532 leader_election.cc:343] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.455615  1762 tablet_bootstrap.cc:492] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
W20260709 13:00:43.455794  1533 leader_election.cc:343] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.456189  1533 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:43.458528  1792 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:43.464572  1465 catalog_manager.cc:5697] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:43.474246  1568 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:43.478385  1763 tablet_bootstrap.cc:492] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:43.478865  1761 tablet_bootstrap.cc:492] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:43.479195  1763 ts_tablet_manager.cc:1403] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.009s
I20260709 13:00:43.479543  1761 ts_tablet_manager.cc:1403] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.079s	user 0.019s	sys 0.026s
I20260709 13:00:43.483438  1767 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.486377  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:43.487051  1644 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.487828  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:43.488605  1718 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.491961  1762 tablet_bootstrap.cc:654] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.494254  1681 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:43.495813  1813 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:43.496824  1792 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:00:43.500377  1813 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:43.502184  1813 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.496886  1761 raft_consensus.cc:359] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.503405  1761 raft_consensus.cc:385] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.503719  1761 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.504385  1761 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.506544  1761 ts_tablet_manager.cc:1434] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.003s
I20260709 13:00:43.508047  1761 tablet_bootstrap.cc:492] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:43.512636  1763 raft_consensus.cc:359] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.513674  1763 raft_consensus.cc:385] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.513979  1763 raft_consensus.cc:740] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.514727  1763 consensus_queue.cc:260] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.519474  1644 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:43.520289  1718 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:43.521029  1533 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:43.520975  1466 catalog_manager.cc:5697] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:43.521967  1792 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:43.494537  1767 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.527402  1792 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:43.528930  1643 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:43.529685  1792 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.533932  1763 ts_tablet_manager.cc:1434] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.000s
I20260709 13:00:43.553689  1763 tablet_bootstrap.cc:492] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:43.556128  1777 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.556655  1777 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.558890  1761 tablet_bootstrap.cc:654] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.559517  1777 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.560556  1803 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.561100  1803 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.561772  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:43.563407  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:43.564100  1718 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:43.566421  1763 tablet_bootstrap.cc:654] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.571622  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:00:43.573625  1532 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:43.575194  1803 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.575353  1569 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:00:43.575738  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:43.562983  1568 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:43.578776  1803 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:43.579614  1803 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.580161  1803 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.586285  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:43.594528  1607 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:43.596248  1777 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.601760  1762 tablet_bootstrap.cc:492] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:43.602629  1762 ts_tablet_manager.cc:1403] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.148s	user 0.017s	sys 0.028s
I20260709 13:00:43.603138  1803 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.605954  1803 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.608042  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:43.610569  1568 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.617084  1777 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:43.620164  1568 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:43.620478  1761 tablet_bootstrap.cc:492] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:43.620944  1761 ts_tablet_manager.cc:1403] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.113s	user 0.017s	sys 0.012s
I20260709 13:00:43.621598  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:43.622737  1777 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:43.623587  1777 raft_consensus.cc:697] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:43.625094  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:43.627218  1762 raft_consensus.cc:359] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.627775  1762 raft_consensus.cc:385] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.627180  1761 raft_consensus.cc:359] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.628165  1761 raft_consensus.cc:385] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.628144  1762 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.628592  1761 raft_consensus.cc:740] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.625358  1777 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.629863  1761 consensus_queue.cc:260] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.631409  1762 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.626946  1717 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.641316  1467 catalog_manager.cc:5697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:43.643755  1838 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:43.647086  1761 ts_tablet_manager.cc:1434] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.002s
I20260709 13:00:43.648474  1761 tablet_bootstrap.cc:492] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:43.644193  1763 tablet_bootstrap.cc:492] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:43.657756  1763 ts_tablet_manager.cc:1403] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.104s	user 0.023s	sys 0.011s
I20260709 13:00:43.622677  1464 catalog_manager.cc:5697] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:43.636053  1762 ts_tablet_manager.cc:1434] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:00:43.662849  1762 tablet_bootstrap.cc:492] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:43.663870  1717 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:43.665365  1761 tablet_bootstrap.cc:654] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.678035  1763 raft_consensus.cc:359] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.678795  1763 raft_consensus.cc:385] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.679141  1763 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.679999  1763 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.682375  1792 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:43.684273  1763 ts_tablet_manager.cc:1434] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 13:00:43.685798  1763 tablet_bootstrap.cc:492] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:43.692315  1762 tablet_bootstrap.cc:654] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.700862  1810 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.701730  1810 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.705387  1761 tablet_bootstrap.cc:492] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:43.706315  1761 ts_tablet_manager.cc:1403] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.058s	user 0.018s	sys 0.007s
I20260709 13:00:43.706841  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:43.707865  1568 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 1: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:00:43.710731  1761 raft_consensus.cc:359] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.711268  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:43.711975  1761 raft_consensus.cc:385] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.712483  1761 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.704097  1464 catalog_manager.cc:5697] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:43.713959  1682 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:43.713934  1761 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.715137  1813 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.719213  1761 ts_tablet_manager.cc:1434] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 13:00:43.719890  1763 tablet_bootstrap.cc:654] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.720718  1761 tablet_bootstrap.cc:492] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:43.729095  1762 tablet_bootstrap.cc:492] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:43.729842  1762 ts_tablet_manager.cc:1403] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.031s
I20260709 13:00:43.731096  1810 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:43.739203  1762 raft_consensus.cc:359] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.740593  1762 raft_consensus.cc:385] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.741042  1762 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.741937  1762 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.744885  1762 ts_tablet_manager.cc:1434] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.001s
I20260709 13:00:43.745802  1762 tablet_bootstrap.cc:492] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:43.747076  1763 tablet_bootstrap.cc:492] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:43.747507  1763 ts_tablet_manager.cc:1403] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.062s	user 0.018s	sys 0.001s
I20260709 13:00:43.751029  1813 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:43.751691  1763 raft_consensus.cc:359] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.752749  1763 raft_consensus.cc:385] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.753064  1763 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.754330  1763 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.757081  1777 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:43.759042  1803 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:43.760373  1763 ts_tablet_manager.cc:1434] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:00:43.788913  1763 tablet_bootstrap.cc:492] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:43.792713  1762 tablet_bootstrap.cc:654] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.795140  1761 tablet_bootstrap.cc:654] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.806788  1763 tablet_bootstrap.cc:654] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.824360  1761 tablet_bootstrap.cc:492] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:43.828439  1761 ts_tablet_manager.cc:1403] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.108s	user 0.021s	sys 0.005s
I20260709 13:00:43.826277  1763 tablet_bootstrap.cc:492] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:43.830133  1802 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:43.832423  1763 ts_tablet_manager.cc:1403] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.044s	user 0.019s	sys 0.010s
I20260709 13:00:43.836308  1761 raft_consensus.cc:359] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.838428  1761 raft_consensus.cc:385] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.838887  1761 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.838163  1762 tablet_bootstrap.cc:492] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:43.840656  1762 ts_tablet_manager.cc:1403] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.095s	user 0.005s	sys 0.018s
I20260709 13:00:43.859543  1761 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.862228  1761 ts_tablet_manager.cc:1434] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.033s	user 0.019s	sys 0.000s
I20260709 13:00:43.863639  1761 tablet_bootstrap.cc:492] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:43.871083  1792 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.872231  1792 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.881556  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:43.883214  1763 raft_consensus.cc:359] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.883999  1763 raft_consensus.cc:385] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.884266  1763 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.885816  1763 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:43.888006  1533 leader_election.cc:343] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.882644  1792 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.893731  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:00:43.896548  1532 leader_election.cc:343] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:43.897487  1761 tablet_bootstrap.cc:654] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.897367  1532 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:43.902769  1838 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:43.906124  1777 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.907212  1803 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:43.907361  1777 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.909243  1763 ts_tablet_manager.cc:1434] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.076s	user 0.008s	sys 0.010s
I20260709 13:00:43.909106  1813 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:43.911752  1763 tablet_bootstrap.cc:492] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:43.916620  1777 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.919211  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:43.921697  1569 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:43.919431  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:43.923629  1716 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:43.920840  1763 tablet_bootstrap.cc:654] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:43.928462  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:43.930838  1777 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:43.931427  1777 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.931864  1777 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.937283  1763 tablet_bootstrap.cc:492] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:43.937937  1763 ts_tablet_manager.cc:1403] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.000s
I20260709 13:00:43.939692  1810 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.940853  1810 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.942376  1763 raft_consensus.cc:359] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.944267  1763 raft_consensus.cc:385] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:43.944756  1763 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:43.944850  1769 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:43.945591  1763 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.948503  1763 ts_tablet_manager.cc:1434] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 13:00:43.949499  1763 tablet_bootstrap.cc:492] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:43.956238  1810 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:43.958758  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:43.960589  1792 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.961131  1792 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:43.965461  1792 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:43.967331  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:43.968477  1717 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:43.968597  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:43.970484  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:43.974517  1532 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:43.972150  1643 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:43.979403  1792 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:43.980363  1792 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.981221  1792 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:43.970901  1681 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:43.986486  1810 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:43.995549  1763 tablet_bootstrap.cc:654] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.004098  1762 raft_consensus.cc:359] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.005061  1762 raft_consensus.cc:385] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.005343  1762 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.006003  1762 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.009426  1769 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:44.027570  1838 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:00:44.028126  1515 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:44.002362  1777 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.032122  1762 ts_tablet_manager.cc:1434] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.191s	user 0.009s	sys 0.009s
I20260709 13:00:44.041786  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:44.043115  1569 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.052443  1762 tablet_bootstrap.cc:492] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:44.060415  1769 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:44.061406  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:44.065637  1718 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.063045  1777 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:44.080600  1868 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:44.088524  1761 tablet_bootstrap.cc:492] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:44.089268  1761 ts_tablet_manager.cc:1403] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.226s	user 0.083s	sys 0.102s
I20260709 13:00:44.091432  1569 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:44.093586  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:44.094650  1718 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:44.096540  1761 raft_consensus.cc:359] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.104512  1761 raft_consensus.cc:385] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.102855  1762 tablet_bootstrap.cc:654] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.104862  1761 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.106366  1761 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.096921  1855 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:44.108001  1855 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:44.108551  1761 ts_tablet_manager.cc:1434] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:00:44.108953  1855 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.109786  1761 tablet_bootstrap.cc:492] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:44.046348  1792 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.169322  1792 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:44.176811  1761 tablet_bootstrap.cc:654] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.183691  1763 tablet_bootstrap.cc:492] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:44.184006  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:44.184604  1763 ts_tablet_manager.cc:1403] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.235s	user 0.045s	sys 0.071s
I20260709 13:00:44.185067  1643 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:44.193781  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 125638us
I20260709 13:00:44.198872  1763 raft_consensus.cc:359] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.199690  1763 raft_consensus.cc:385] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.199575  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:44.200042  1763 raft_consensus.cc:740] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.200245  1717 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.200989  1763 consensus_queue.cc:260] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.203131  1763 ts_tablet_manager.cc:1434] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:00:44.204330  1763 tablet_bootstrap.cc:492] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:44.206138  1718 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.213070  1777 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:44.213941  1868 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.216022  1868 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.229030  1762 tablet_bootstrap.cc:492] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:44.229583  1762 ts_tablet_manager.cc:1403] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.177s	user 0.024s	sys 0.070s
I20260709 13:00:44.233263  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:44.230994  1643 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:44.234591  1762 raft_consensus.cc:359] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.231000  1761 tablet_bootstrap.cc:492] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:44.235399  1762 raft_consensus.cc:385] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.235764  1762 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.235899  1761 ts_tablet_manager.cc:1403] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.127s	user 0.016s	sys 0.010s
I20260709 13:00:44.236608  1762 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.239907  1855 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.240554  1855 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.241240  1777 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:00:44.241771  1533 leader_election.cc:343] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.242543  1533 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:44.242727  1761 raft_consensus.cc:359] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.243772  1761 raft_consensus.cc:385] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.244001  1792 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:44.244072  1761 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.265921  1717 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:44.230118  1868 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:44.244550  1792 raft_consensus.cc:697] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:44.300817  1792 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.303223  1769 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.303699  1777 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.304034  1769 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.289004  1802 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.306272  1802 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.309664  1802 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:44.311460  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:44.317651  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:44.318601  1569 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:44.304171  1777 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.350142  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:44.351814  1810 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:44.352351  1810 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.352849  1810 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:44.338717  1607 leader_election.cc:343] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.383631  1868 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.476325  1777 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:44.476614  1762 ts_tablet_manager.cc:1434] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.247s	user 0.007s	sys 0.001s
I20260709 13:00:44.480986  1762 tablet_bootstrap.cc:492] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:44.488406  1761 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.490628  1769 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:44.492928  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:44.493552  1855 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:44.496266  1532 leader_election.cc:343] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.496996  1532 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:44.499611  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:44.503788  1761 ts_tablet_manager.cc:1434] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.266s	user 0.039s	sys 0.046s
I20260709 13:00:44.505144  1865 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:44.505402  1761 tablet_bootstrap.cc:492] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:44.506517  1810 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.511381  1810 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:00:44.503132  1607 leader_election.cc:343] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.517891  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:44.519057  1643 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:44.522253  1763 tablet_bootstrap.cc:654] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.478623  1868 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.525324  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:44.534654  1567 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:44.532649  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:44.536376  1802 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:44.536898  1563 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.526309  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:44.538069  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:44.542610  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:44.537532  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:44.545009  1715 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:00:44.545480  1643 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.549365  1762 tablet_bootstrap.cc:654] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.556803  1763 tablet_bootstrap.cc:492] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:44.557323  1763 ts_tablet_manager.cc:1403] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.353s	user 0.016s	sys 0.002s
W20260709 13:00:44.559209  1607 leader_election.cc:343] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.560401  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:44.561578  1777 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:44.561918  1643 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:44.564574  1464 catalog_manager.cc:5697] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:44.570214  1682 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:44.573256  1875 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:44.574100  1875 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:44.575913  1875 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.593467  1868 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:44.593782  1563 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:44.614856  1763 raft_consensus.cc:359] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.617319  1763 raft_consensus.cc:385] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.617690  1763 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.611729  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:44.619102  1763 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.619675  1761 tablet_bootstrap.cc:654] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:44.623384  1532 leader_election.cc:343] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.624774  1763 ts_tablet_manager.cc:1434] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.067s	user 0.026s	sys 0.010s
I20260709 13:00:44.625078  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:44.626670  1763 tablet_bootstrap.cc:492] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:44.611574  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:44.630406  1762 tablet_bootstrap.cc:492] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:44.630820  1762 ts_tablet_manager.cc:1403] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.150s	user 0.011s	sys 0.006s
I20260709 13:00:44.632115  1777 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:44.632591  1777 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.634224  1777 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.642895  1762 raft_consensus.cc:359] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.643561  1762 raft_consensus.cc:385] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.643833  1762 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.644471  1762 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:44.645331  1533 leader_election.cc:343] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.646939  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:44.647912  1867 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:44.650744  1762 ts_tablet_manager.cc:1434] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:00:44.651953  1762 tablet_bootstrap.cc:492] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:44.650995  1607 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:44.659588  1763 tablet_bootstrap.cc:654] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.660595  1855 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:44.661929  1762 tablet_bootstrap.cc:654] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.675077  1763 tablet_bootstrap.cc:492] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:44.675680  1763 ts_tablet_manager.cc:1403] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.050s	user 0.015s	sys 0.008s
I20260709 13:00:44.680155  1763 raft_consensus.cc:359] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.680979  1763 raft_consensus.cc:385] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.681280  1763 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.681733  1762 tablet_bootstrap.cc:492] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:44.681964  1763 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.682554  1762 ts_tablet_manager.cc:1403] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.010s
I20260709 13:00:44.686349  1762 raft_consensus.cc:359] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.687147  1762 raft_consensus.cc:385] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.687546  1762 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.688436  1762 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.691285  1810 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.691875  1762 ts_tablet_manager.cc:1434] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:00:44.691787  1810 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.693622  1762 tablet_bootstrap.cc:492] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:44.693763  1810 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:44.695614  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:44.696640  1642 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:44.699453  1682 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:44.700784  1810 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:44.701370  1810 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.701900  1810 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.706076  1762 tablet_bootstrap.cc:654] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.707567  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:44.708956  1563 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:44.712718  1777 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.713965  1813 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.714682  1813 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.717555  1813 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:44.719005  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:44.720675  1682 leader_election.cc:343] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.721858  1810 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.724793  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:44.724908  1810 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:44.727842  1763 ts_tablet_manager.cc:1434] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.052s	user 0.010s	sys 0.001s
I20260709 13:00:44.729677  1763 tablet_bootstrap.cc:492] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
W20260709 13:00:44.729192  1681 leader_election.cc:343] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.732008  1681 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:44.732784  1810 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:44.733420  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:44.735116  1867 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.735404  1762 tablet_bootstrap.cc:492] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:44.735955  1762 ts_tablet_manager.cc:1403] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.043s	user 0.021s	sys 0.011s
I20260709 13:00:44.726014  1855 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:00:44.735725  1867 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.740197  1762 raft_consensus.cc:359] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.741428  1762 raft_consensus.cc:385] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.741740  1762 raft_consensus.cc:740] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.742642  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:44.742980  1762 consensus_queue.cc:260] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.744822  1762 ts_tablet_manager.cc:1434] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:00:44.745803  1762 tablet_bootstrap.cc:492] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:44.746129  1761 tablet_bootstrap.cc:492] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:44.746500  1761 ts_tablet_manager.cc:1403] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.241s	user 0.016s	sys 0.000s
I20260709 13:00:44.744709  1643 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:44.734714  1564 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.747924  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:44.744882  1867 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:44.753650  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:44.754534  1642 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:44.755012  1532 leader_election.cc:343] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.756171  1533 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:44.757990  1792 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:44.758412  1792 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.759022  1792 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.757675  1761 raft_consensus.cc:359] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.764683  1761 raft_consensus.cc:385] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.771183  1763 tablet_bootstrap.cc:654] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.765030  1761 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.774139  1642 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:44.775935  1682 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:44.777235  1867 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:44.778307  1875 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:44.777287  1467 catalog_manager.cc:5697] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:44.788478  1761 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.788403  1813 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.790553  1813 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.795251  1875 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:44.798218  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:44.798673  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:44.799027  1568 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.798064  1875 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.800537  1813 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:44.801031  1761 ts_tablet_manager.cc:1434] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.004s
I20260709 13:00:44.802906  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:44.805025  1682 leader_election.cc:343] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.811502  1777 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:44.814777  1681 leader_election.cc:343] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.815505  1681 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:44.817251  1762 tablet_bootstrap.cc:654] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.818164  1761 tablet_bootstrap.cc:492] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:44.828222  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:00:44.834210  1607 leader_election.cc:343] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.837419  1568 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:44.819386  1813 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:44.843458  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:44.823431  1792 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.856472  1763 tablet_bootstrap.cc:492] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:44.851024  1777 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:44.857050  1763 ts_tablet_manager.cc:1403] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.128s	user 0.011s	sys 0.012s
I20260709 13:00:44.857132  1777 raft_consensus.cc:697] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:44.850564  1564 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:44.858538  1777 consensus_queue.cc:237] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.860714  1763 raft_consensus.cc:359] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.861651  1763 raft_consensus.cc:385] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.862030  1763 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.863477  1763 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.870208  1763 ts_tablet_manager.cc:1434] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.003s
I20260709 13:00:44.875859  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:44.877231  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:44.878469  1642 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:44.882351  1532 leader_election.cc:343] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.887912  1792 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:44.891499  1763 tablet_bootstrap.cc:492] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:44.891348  1867 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:44.892197  1792 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.892715  1792 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.895023  1761 tablet_bootstrap.cc:654] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.896286  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:44.900050  1533 leader_election.cc:343] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:44.900725  1642 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:44.902526  1533 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:44.903976  1867 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:44.904757  1867 raft_consensus.cc:697] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:44.907521  1763 tablet_bootstrap.cc:654] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:44.897091  1762 tablet_bootstrap.cc:492] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:44.909039  1762 ts_tablet_manager.cc:1403] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.163s	user 0.013s	sys 0.006s
I20260709 13:00:44.906311  1867 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.916532  1792 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:44.918875  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:44.919965  1713 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:44.929193  1763 tablet_bootstrap.cc:492] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:44.931411  1763 ts_tablet_manager.cc:1403] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.011s
I20260709 13:00:44.936553  1762 raft_consensus.cc:359] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.938218  1762 raft_consensus.cc:385] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.935530  1532 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:44.938838  1762 raft_consensus.cc:740] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.940979  1762 consensus_queue.cc:260] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.943670  1865 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:44.944547  1865 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.945336  1865 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:44.954845  1762 ts_tablet_manager.cc:1434] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.045s	user 0.014s	sys 0.000s
I20260709 13:00:44.956069  1762 tablet_bootstrap.cc:492] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:44.980566  1763 raft_consensus.cc:359] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.981788  1763 raft_consensus.cc:385] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:44.982100  1763 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.982568  1464 catalog_manager.cc:5697] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:44.943982  1466 catalog_manager.cc:5697] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:44.987773  1464 catalog_manager.cc:5697] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:44.995133  1761 tablet_bootstrap.cc:492] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:44.996081  1763 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:44.997771  1761 ts_tablet_manager.cc:1403] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.196s	user 0.020s	sys 0.044s
I20260709 13:00:45.001749  1762 tablet_bootstrap.cc:654] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.001533  1761 raft_consensus.cc:359] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.002225  1761 raft_consensus.cc:385] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.003016  1761 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:44.998319  1763 ts_tablet_manager.cc:1434] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.019s
I20260709 13:00:44.990859  1865 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.006114  1865 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:45.007450  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:45.008033  1761 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.007450  1763 tablet_bootstrap.cc:492] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:45.009135  1777 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.010288  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:45.009771  1777 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.011543  1642 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.012249  1467 catalog_manager.cc:5697] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:45.008471  1715 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.015681  1777 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:45.018329  1813 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.019129  1763 tablet_bootstrap.cc:654] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.018973  1813 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.019373  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:00:45.021174  1607 leader_election.cc:343] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:45.022154  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:45.022905  1564 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:45.023515  1813 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:45.024044  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:45.024799  1761 ts_tablet_manager.cc:1434] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 13:00:45.025221  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:45.025847  1761 tablet_bootstrap.cc:492] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
W20260709 13:00:45.031390  1682 leader_election.cc:343] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:45.032859  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:45.035223  1777 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:45.036008  1777 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:45.036450  1777 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.025864  1564 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:45.048062  1681 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:45.049503  1813 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:45.050096  1813 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:45.050583  1813 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.065341  1777 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:45.069291  1515 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:45.069500  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:45.070259  1564 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.074038  1777 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:45.077772  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:45.077806  1642 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:45.079792  1813 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:45.080710  1607 leader_election.cc:343] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:45.087272  1761 tablet_bootstrap.cc:654] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.085282  1715 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:45.095978  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:45.098002  1865 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:45.099176  1865 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:45.117566  1763 tablet_bootstrap.cc:492] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:45.099298  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:45.118366  1763 ts_tablet_manager.cc:1403] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.111s	user 0.005s	sys 0.021s
I20260709 13:00:45.121543  1763 raft_consensus.cc:359] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.122131  1763 raft_consensus.cc:385] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.122431  1763 raft_consensus.cc:740] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:45.123172  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:45.123868  1568 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.125746  1763 consensus_queue.cc:260] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.127929  1763 ts_tablet_manager.cc:1434] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:00:45.095964  1813 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:00:45.136535  1682 leader_election.cc:343] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:45.134135  1865 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.140292  1763 tablet_bootstrap.cc:492] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:45.140415  1762 tablet_bootstrap.cc:492] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:45.141500  1762 ts_tablet_manager.cc:1403] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.185s	user 0.011s	sys 0.037s
I20260709 13:00:45.115118  1564 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:45.188918  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:45.197024  1761 tablet_bootstrap.cc:492] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:45.200956  1761 ts_tablet_manager.cc:1403] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.175s	user 0.008s	sys 0.019s
I20260709 13:00:45.204461  1762 raft_consensus.cc:359] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.207381  1777 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.208509  1777 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.206386  1762 raft_consensus.cc:385] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.214481  1762 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:45.217386  1777 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:45.231359  1855 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:45.231402  1761 raft_consensus.cc:359] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.220938  1762 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.232592  1761 raft_consensus.cc:385] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.233013  1761 raft_consensus.cc:740] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:45.232362  1855 raft_consensus.cc:697] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:45.242084  1906 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.256242  1855 consensus_queue.cc:237] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.266258  1568 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:45.266510  1761 consensus_queue.cc:260] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.268131  1763 tablet_bootstrap.cc:654] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.287683  1681 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:45.289953  1761 ts_tablet_manager.cc:1434] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.088s	user 0.009s	sys 0.012s
I20260709 13:00:45.290172  1810 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:00:45.296679  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 109047us
I20260709 13:00:45.299003  1762 ts_tablet_manager.cc:1434] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.157s	user 0.030s	sys 0.026s
I20260709 13:00:45.291610  1810 raft_consensus.cc:697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:45.302608  1906 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.303180  1875 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.303524  1810 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.306180  1761 tablet_bootstrap.cc:492] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:45.301746  1762 tablet_bootstrap.cc:492] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:45.318598  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:45.320724  1715 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:45.343636  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:45.344655  1564 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:45.357509  1763 tablet_bootstrap.cc:492] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:45.358354  1763 ts_tablet_manager.cc:1403] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.230s	user 0.028s	sys 0.004s
I20260709 13:00:45.366008  1762 tablet_bootstrap.cc:654] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.360198  1906 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.370996  1906 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.376142  1466 catalog_manager.cc:5697] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:45.396564  1713 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.403367  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:45.368965  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:45.407011  1564 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 1: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:00:45.407368  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:45.402117  1906 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:45.412699  1907 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:00:45.413098  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 44249.6us
I20260709 13:00:45.413350  1907 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:45.413902  1907 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.415231  1763 raft_consensus.cc:359] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.416476  1763 raft_consensus.cc:385] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.416908  1763 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:45.419313  1855 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:45.423971  1792 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:00:45.429553  1763 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.434368  1763 ts_tablet_manager.cc:1434] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.076s	user 0.014s	sys 0.001s
I20260709 13:00:45.440544  1761 tablet_bootstrap.cc:654] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.444121  1682 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:45.447027  1906 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:45.448640  1810 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.452430  1763 tablet_bootstrap.cc:492] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:45.457069  1855 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.470124  1466 catalog_manager.cc:5697] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:45.475320  1467 catalog_manager.cc:5697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:45.511205  1762 tablet_bootstrap.cc:492] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:45.512104  1762 ts_tablet_manager.cc:1403] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.211s	user 0.027s	sys 0.004s
I20260709 13:00:45.516290  1907 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.535666  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:45.537185  1564 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:45.549098  1532 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:45.557927  1762 raft_consensus.cc:359] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.560521  1762 raft_consensus.cc:385] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.561039  1762 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:45.567817  1762 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.575242  1761 tablet_bootstrap.cc:492] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:45.575101  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:45.576118  1761 ts_tablet_manager.cc:1403] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.270s	user 0.012s	sys 0.017s
I20260709 13:00:45.576386  1715 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.571741  1907 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:45.585645  1810 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.586179  1810 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.588733  1810 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:45.590593  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:00:45.592681  1681 leader_election.cc:343] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:45.601990  1762 ts_tablet_manager.cc:1434] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.090s	user 0.013s	sys 0.002s
I20260709 13:00:45.604036  1762 tablet_bootstrap.cc:492] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:45.598634  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:45.609725  1715 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:45.609946  1641 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:45.614557  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:00:45.586598  1761 raft_consensus.cc:359] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.615427  1761 raft_consensus.cc:385] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.615798  1761 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:45.616436  1810 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:45.617552  1810 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:45.617951  1810 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.620669  1763 tablet_bootstrap.cc:654] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.622746  1564 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:45.630887  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:45.632330  1865 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.632624  1777 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:45.633400  1777 raft_consensus.cc:697] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:45.634193  1777 consensus_queue.cc:237] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.640842  1762 tablet_bootstrap.cc:654] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.640691  1761 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.647891  1810 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.648459  1761 ts_tablet_manager.cc:1434] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.000s
I20260709 13:00:45.649791  1761 tablet_bootstrap.cc:492] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:45.656381  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:45.665881  1865 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.666574  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:45.667497  1641 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.654220  1810 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:00:45.682564  1681 leader_election.cc:343] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:45.710862  1761 tablet_bootstrap.cc:654] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.729895  1763 tablet_bootstrap.cc:492] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:45.732681  1763 ts_tablet_manager.cc:1403] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.284s	user 0.017s	sys 0.034s
I20260709 13:00:45.729081  1762 tablet_bootstrap.cc:492] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:45.733842  1762 ts_tablet_manager.cc:1403] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.130s	user 0.012s	sys 0.014s
I20260709 13:00:45.739295  1763 raft_consensus.cc:359] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.740658  1763 raft_consensus.cc:385] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.740237  1762 raft_consensus.cc:359] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.741182  1763 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:45.741914  1762 raft_consensus.cc:385] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.742445  1762 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:45.745009  1763 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.744716  1762 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.749444  1763 ts_tablet_manager.cc:1434] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:00:45.750628  1762 ts_tablet_manager.cc:1434] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:00:45.751091  1763 tablet_bootstrap.cc:492] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:45.757241  1762 tablet_bootstrap.cc:492] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:45.730706  1641 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:45.768908  1810 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.770437  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:00:45.771567  1763 tablet_bootstrap.cc:654] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.772414  1810 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:45.776512  1810 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:45.777839  1810 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.779075  1761 tablet_bootstrap.cc:492] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:45.779641  1761 ts_tablet_manager.cc:1403] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.130s	user 0.009s	sys 0.020s
I20260709 13:00:45.783432  1641 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.792101  1762 tablet_bootstrap.cc:654] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.795503  1715 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:45.790737  1761 raft_consensus.cc:359] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.796738  1761 raft_consensus.cc:385] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.797215  1761 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:45.798014  1761 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.804162  1761 ts_tablet_manager.cc:1434] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:00:45.805606  1761 tablet_bootstrap.cc:492] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:45.814757  1762 tablet_bootstrap.cc:492] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:45.815358  1762 ts_tablet_manager.cc:1403] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.063s	user 0.015s	sys 0.000s
I20260709 13:00:45.819483  1466 catalog_manager.cc:5697] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:45.818320  1762 raft_consensus.cc:359] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.823271  1762 raft_consensus.cc:385] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.823731  1762 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:45.830109  1761 tablet_bootstrap.cc:654] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.824654  1762 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.841357  1762 ts_tablet_manager.cc:1434] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:00:45.849099  1907 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.849850  1907 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.856968  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:45.865476  1777 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.864372  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:45.866560  1762 tablet_bootstrap.cc:492] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:45.868387  1907 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:45.871066  1607 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:45.864861  1464 catalog_manager.cc:5697] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:45.878490  1907 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:45.883061  1906 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.883692  1906 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.886759  1906 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:45.953058  1762 tablet_bootstrap.cc:654] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:45.953608  1908 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.960413  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:45.960867  1813 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:45.959404  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:45.966603  1777 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:45.968760  1761 tablet_bootstrap.cc:492] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:45.969331  1761 ts_tablet_manager.cc:1403] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.164s	user 0.032s	sys 0.025s
I20260709 13:00:45.956305  1908 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.972219  1763 tablet_bootstrap.cc:492] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:45.972998  1763 ts_tablet_manager.cc:1403] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.222s	user 0.048s	sys 0.099s
I20260709 13:00:45.973168  1908 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:45.972532  1761 raft_consensus.cc:359] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.976181  1761 raft_consensus.cc:385] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.976325  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:45.976716  1761 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:45.979176  1763 raft_consensus.cc:359] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.980268  1763 raft_consensus.cc:385] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:45.980998  1763 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
W20260709 13:00:45.983453  1681 leader_election.cc:343] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:45.977391  1761 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:45.993662  1761 ts_tablet_manager.cc:1434] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.001s
I20260709 13:00:45.995134  1761 tablet_bootstrap.cc:492] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:45.996315  1906 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.996999  1906 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.000763  1906 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:00:46.002154  1682 leader_election.cc:343] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.007432  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:45.986519  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:46.010820  1908 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:46.012967  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:46.017903  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:46.020330  1761 tablet_bootstrap.cc:654] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.027581  1682 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:46.036957  1682 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:46.038787  1906 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:46.038053  1908 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:46.034628  1763 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.042796  1763 ts_tablet_manager.cc:1434] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.069s	user 0.011s	sys 0.000s
I20260709 13:00:46.048714  1761 tablet_bootstrap.cc:492] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:46.049206  1761 ts_tablet_manager.cc:1403] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.054s	user 0.016s	sys 0.000s
I20260709 13:00:46.052209  1713 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.054013  1763 tablet_bootstrap.cc:492] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:46.067956  1762 tablet_bootstrap.cc:492] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:46.068704  1762 ts_tablet_manager.cc:1403] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.226s	user 0.036s	sys 0.008s
I20260709 13:00:46.073340  1762 raft_consensus.cc:359] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.074409  1762 raft_consensus.cc:385] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.080600  1762 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.081522  1762 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.084262  1762 ts_tablet_manager.cc:1434] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:00:46.085503  1762 tablet_bootstrap.cc:492] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:46.078967  1763 tablet_bootstrap.cc:654] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.079855  1761 raft_consensus.cc:359] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.088241  1761 raft_consensus.cc:385] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.088629  1761 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.089383  1761 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.092450  1792 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.092684  1761 ts_tablet_manager.cc:1434] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.000s
I20260709 13:00:46.093003  1792 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.095080  1761 tablet_bootstrap.cc:492] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:46.096081  1907 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.096632  1907 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.098577  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:46.099277  1643 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:46.100713  1907 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.102058  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:46.103502  1761 tablet_bootstrap.cc:654] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:46.104357  1607 leader_election.cc:343] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.104986  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:46.105687  1563 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:46.107724  1533 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:46.108778  1865 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:46.108877  1762 tablet_bootstrap.cc:654] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.109225  1865 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.109609  1865 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.112475  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:46.114315  1907 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:46.114797  1907 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.115347  1907 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.116503  1867 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.116896  1792 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.119069  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:46.123467  1865 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:46.124228  1532 leader_election.cc:343] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.117491  1867 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.128711  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:46.132489  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:46.138823  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:46.143941  1865 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:46.145974  1532 leader_election.cc:343] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.153599  1865 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.154305  1865 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.161980  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:46.162905  1865 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.169977  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:00:46.173969  1532 leader_election.cc:343] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:46.175148  1532 leader_election.cc:343] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.144222  1867 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.197228  1907 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.209537  1777 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:46.211032  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:46.211894  1533 leader_election.cc:343] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.215513  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:00:46.220607  1533 leader_election.cc:343] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.221290  1533 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:46.222034  1942 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:46.223222  1643 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 1: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:00:46.225589  1867 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:46.227202  1802 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.227309  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:46.228389  1802 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.224159  1907 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.231510  1802 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.234808  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:46.228907  1569 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:00:46.236172  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:00:46.236745  1607 leader_election.cc:343] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.236862  1715 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:46.233315  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:46.239969  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:00:46.241436  1802 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:46.242362  1802 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.243001  1802 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.231730  1533 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:00:46.241923  1607 leader_election.cc:343] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.245615  1942 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:46.246043  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:46.247499  1777 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:46.251107  1761 tablet_bootstrap.cc:492] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:46.252734  1761 ts_tablet_manager.cc:1403] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.158s	user 0.027s	sys 0.037s
I20260709 13:00:46.258155  1761 raft_consensus.cc:359] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.261957  1761 raft_consensus.cc:385] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.264961  1761 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.265869  1763 tablet_bootstrap.cc:492] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:46.266659  1763 ts_tablet_manager.cc:1403] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.213s	user 0.049s	sys 0.031s
I20260709 13:00:46.266379  1761 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.268787  1867 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.269279  1867 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.270800  1563 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.272439  1762 tablet_bootstrap.cc:492] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:46.273129  1762 ts_tablet_manager.cc:1403] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.188s	user 0.072s	sys 0.081s
I20260709 13:00:46.273569  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:46.270718  1761 ts_tablet_manager.cc:1434] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:00:46.276083  1761 tablet_bootstrap.cc:492] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:46.274693  1867 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.286466  1907 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:46.288437  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:46.289474  1643 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 1: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:00:46.290433  1515 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:46.290899  1533 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:46.316367  1813 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:46.317133  1942 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:46.324548  1763 raft_consensus.cc:359] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.325277  1763 raft_consensus.cc:385] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.325574  1763 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.328759  1763 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.330731  1802 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.333564  1802 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.339382  1761 tablet_bootstrap.cc:654] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.342079  1763 ts_tablet_manager.cc:1434] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.075s	user 0.003s	sys 0.003s
I20260709 13:00:46.338658  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:46.343209  1713 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.344226  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:46.345343  1564 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.362718  1762 raft_consensus.cc:359] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.377298  1762 raft_consensus.cc:385] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.377803  1762 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.361512  1713 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:46.378655  1762 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.368320  1761 tablet_bootstrap.cc:492] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:46.371449  1813 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:00:46.380839  1761 ts_tablet_manager.cc:1403] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.105s	user 0.005s	sys 0.013s
I20260709 13:00:46.381742  1762 ts_tablet_manager.cc:1434] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.108s	user 0.008s	sys 0.004s
I20260709 13:00:46.383126  1762 tablet_bootstrap.cc:492] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:46.358071  1564 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:46.384837  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:46.385911  1777 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:46.386355  1777 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:46.386654  1761 raft_consensus.cc:359] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.387068  1777 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.387513  1761 raft_consensus.cc:385] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.388094  1761 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.388916  1761 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.392346  1761 ts_tablet_manager.cc:1434] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:00:46.374577  1763 tablet_bootstrap.cc:492] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:46.409616  1761 tablet_bootstrap.cc:492] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:46.414597  1762 tablet_bootstrap.cc:654] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.420825  1906 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.423368  1906 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.429733  1761 tablet_bootstrap.cc:654] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.431514  1763 tablet_bootstrap.cc:654] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.433583  1906 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:46.431725  1467 catalog_manager.cc:5697] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:46.435621  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:00:46.440886  1681 leader_election.cc:343] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.440918  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:46.444108  1682 leader_election.cc:343] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.445191  1682 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:46.447350  1813 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:46.455029  1762 tablet_bootstrap.cc:492] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:46.455740  1762 ts_tablet_manager.cc:1403] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.073s	user 0.012s	sys 0.005s
I20260709 13:00:46.462827  1762 raft_consensus.cc:359] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.464555  1762 raft_consensus.cc:385] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.466073  1762 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.466892  1762 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.472740  1762 ts_tablet_manager.cc:1434] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:00:46.475571  1762 tablet_bootstrap.cc:492] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:46.483219  1762 tablet_bootstrap.cc:654] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.484776  1761 tablet_bootstrap.cc:492] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:46.485396  1761 ts_tablet_manager.cc:1403] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.076s	user 0.013s	sys 0.001s
I20260709 13:00:46.495709  1761 raft_consensus.cc:359] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.496290  1761 raft_consensus.cc:385] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.496542  1761 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.497471  1761 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.507697  1763 tablet_bootstrap.cc:492] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:46.508754  1865 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.508638  1763 ts_tablet_manager.cc:1403] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.134s	user 0.018s	sys 0.001s
I20260709 13:00:46.509222  1865 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.513243  1763 raft_consensus.cc:359] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.513656  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:46.514019  1763 raft_consensus.cc:385] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.514360  1643 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:46.514494  1763 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.513804  1762 tablet_bootstrap.cc:492] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:46.515316  1762 ts_tablet_manager.cc:1403] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.003s
I20260709 13:00:46.515777  1761 ts_tablet_manager.cc:1434] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:00:46.518086  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:46.518857  1761 tablet_bootstrap.cc:492] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:46.519230  1867 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:46.519676  1867 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.520038  1867 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.515463  1763 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.521880  1762 raft_consensus.cc:359] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.522707  1762 raft_consensus.cc:385] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.523031  1762 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.516294  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:46.523563  1865 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.524166  1713 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:46.529067  1761 tablet_bootstrap.cc:654] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.525032  1762 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.590530  1865 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.591169  1865 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.593719  1865 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:46.595566  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 64696.8us
I20260709 13:00:46.598464  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:46.600907  1533 leader_election.cc:343] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.602051  1867 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.604235  1763 ts_tablet_manager.cc:1434] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.095s	user 0.017s	sys 0.007s
I20260709 13:00:46.606160  1763 tablet_bootstrap.cc:492] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:46.602475  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:46.611760  1762 ts_tablet_manager.cc:1434] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.094s	user 0.006s	sys 0.001s
I20260709 13:00:46.613065  1762 tablet_bootstrap.cc:492] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:46.621482  1761 tablet_bootstrap.cc:492] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:46.622015  1761 ts_tablet_manager.cc:1403] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.103s	user 0.026s	sys 0.013s
I20260709 13:00:46.628577  1761 raft_consensus.cc:359] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.629390  1761 raft_consensus.cc:385] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.629679  1761 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.630393  1761 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.630729  1763 tablet_bootstrap.cc:654] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:46.624402  1532 leader_election.cc:343] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.632638  1761 ts_tablet_manager.cc:1434] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:00:46.632205  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:46.633427  1865 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:46.633819  1761 tablet_bootstrap.cc:492] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:46.639333  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:46.641794  1641 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.647455  1867 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.649593  1762 tablet_bootstrap.cc:654] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.651499  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:46.652227  1713 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.655412  1641 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:46.656992  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:46.658051  1761 tablet_bootstrap.cc:654] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.658120  1867 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:46.659390  1867 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:46.660307  1867 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.664021  1713 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:46.667928  1761 tablet_bootstrap.cc:492] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:46.668967  1761 ts_tablet_manager.cc:1403] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.004s
I20260709 13:00:46.669866  1763 tablet_bootstrap.cc:492] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:46.670516  1763 ts_tablet_manager.cc:1403] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.065s	user 0.008s	sys 0.008s
I20260709 13:00:46.675292  1906 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.678468  1763 raft_consensus.cc:359] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.679703  1763 raft_consensus.cc:385] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.680087  1763 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.680955  1763 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.684305  1762 tablet_bootstrap.cc:492] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:46.684698  1763 ts_tablet_manager.cc:1434] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.002s
I20260709 13:00:46.683518  1761 raft_consensus.cc:359] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.685480  1761 raft_consensus.cc:385] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.684859  1762 ts_tablet_manager.cc:1403] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.072s	user 0.016s	sys 0.012s
I20260709 13:00:46.686370  1763 tablet_bootstrap.cc:492] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:46.686784  1761 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.689111  1948 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.689880  1948 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.691256  1761 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.694597  1761 ts_tablet_manager.cc:1434] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.004s
I20260709 13:00:46.690145  1762 raft_consensus.cc:359] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.696004  1762 raft_consensus.cc:385] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.696460  1762 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.677456  1906 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.701042  1906 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:46.702065  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:46.705372  1682 leader_election.cc:343] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.696967  1948 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.729514  1762 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.731909  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:46.731839  1761 tablet_bootstrap.cc:492] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
W20260709 13:00:46.734728  1607 leader_election.cc:343] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.738476  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:46.739391  1762 ts_tablet_manager.cc:1434] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.053s	user 0.012s	sys 0.009s
I20260709 13:00:46.739472  1564 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:46.741447  1607 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:46.740885  1466 catalog_manager.cc:5697] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:46.745883  1948 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:46.746338  1948 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.746726  1948 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.754685  1762 tablet_bootstrap.cc:492] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:46.755184  1867 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.756070  1867 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.759639  1761 tablet_bootstrap.cc:654] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.757241  1763 tablet_bootstrap.cc:654] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.771334  1762 tablet_bootstrap.cc:654] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.775549  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:46.782327  1533 leader_election.cc:343] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.787140  1761 tablet_bootstrap.cc:492] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:46.787832  1761 ts_tablet_manager.cc:1403] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.056s	user 0.018s	sys 0.011s
I20260709 13:00:46.791373  1948 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.794832  1944 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:46.795495  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:46.796293  1563 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.796530  1761 raft_consensus.cc:359] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.797307  1761 raft_consensus.cc:385] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.797612  1761 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.799419  1867 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.800263  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:46.801044  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:00:46.802141  1607 leader_election.cc:343] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.802950  1948 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:46.804384  1532 leader_election.cc:343] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.798297  1761 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.805624  1532 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:46.808008  1865 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:46.808580  1761 ts_tablet_manager.cc:1434] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.001s
I20260709 13:00:46.808907  1763 tablet_bootstrap.cc:492] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:46.809628  1761 tablet_bootstrap.cc:492] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:46.809573  1763 ts_tablet_manager.cc:1403] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.124s	user 0.014s	sys 0.001s
I20260709 13:00:46.816578  1762 tablet_bootstrap.cc:492] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:46.774886  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:46.817360  1762 ts_tablet_manager.cc:1403] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.077s	user 0.017s	sys 0.006s
I20260709 13:00:46.817875  1569 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:46.820123  1761 tablet_bootstrap.cc:654] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.823065  1948 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:46.826272  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:46.827910  1906 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:46.828510  1906 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.829001  1906 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.822044  1763 raft_consensus.cc:359] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.849354  1763 raft_consensus.cc:385] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.849928  1763 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.859907  1762 raft_consensus.cc:359] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.861433  1762 raft_consensus.cc:385] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.861860  1762 raft_consensus.cc:740] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.863464  1763 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.867679  1865 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.868907  1865 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.871161  1802 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.872529  1865 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.872732  1802 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.873512  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:46.875222  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:00:46.875581  1533 leader_election.cc:343] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.877072  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:46.877384  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:00:46.879493  1607 leader_election.cc:343] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:46.880277  1607 leader_election.cc:343] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.880707  1607 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:46.867192  1763 ts_tablet_manager.cc:1434] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.005s
I20260709 13:00:46.883250  1763 tablet_bootstrap.cc:492] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:46.868093  1762 consensus_queue.cc:260] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.887441  1906 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.888094  1762 ts_tablet_manager.cc:1434] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.064s	user 0.010s	sys 0.001s
I20260709 13:00:46.890717  1906 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:46.894454  1762 tablet_bootstrap.cc:492] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:46.894452  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:46.895758  1564 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.896842  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:00:46.898866  1682 leader_election.cc:343] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.900987  1777 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:46.880298  1532 leader_election.cc:343] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.904589  1532 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:46.904878  1762 tablet_bootstrap.cc:654] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.906507  1865 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:46.908473  1763 tablet_bootstrap.cc:654] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.903394  1802 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.923131  1763 tablet_bootstrap.cc:492] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:46.923684  1763 ts_tablet_manager.cc:1403] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.002s
I20260709 13:00:46.924353  1762 tablet_bootstrap.cc:492] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:46.925355  1762 ts_tablet_manager.cc:1403] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.007s
I20260709 13:00:46.927101  1763 raft_consensus.cc:359] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.927918  1763 raft_consensus.cc:385] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.928194  1763 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.928586  1762 raft_consensus.cc:359] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.929868  1762 raft_consensus.cc:385] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.934615  1762 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.935561  1763 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.936538  1762 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.937888  1563 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:46.940050  1762 ts_tablet_manager.cc:1434] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:00:46.942221  1762 tablet_bootstrap.cc:492] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:46.938861  1564 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:46.938666  1763 ts_tablet_manager.cc:1434] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:00:46.944242  1763 tablet_bootstrap.cc:492] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:46.944267  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:46.946169  1813 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:46.946884  1813 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:46.947865  1813 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.950718  1761 tablet_bootstrap.cc:492] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:46.951268  1761 ts_tablet_manager.cc:1403] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.142s	user 0.043s	sys 0.031s
I20260709 13:00:46.951771  1607 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:46.954685  1802 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:46.956297  1802 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:46.955781  1761 raft_consensus.cc:359] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.958652  1802 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.961277  1969 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.962036  1969 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.966612  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:46.967536  1569 raft_consensus.cc:2393] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 1: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:00:46.968230  1908 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.968695  1969 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:46.968995  1908 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.970310  1948 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.970844  1948 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.971683  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:46.958349  1761 raft_consensus.cc:385] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:46.972326  1761 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:46.973346  1777 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.973580  1682 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:46.973703  1761 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.973894  1777 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:46.975394  1906 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:46.976308  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:46.976687  1777 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.977366  1761 ts_tablet_manager.cc:1434] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.026s	user 0.011s	sys 0.000s
I20260709 13:00:46.977821  1948 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:46.978595  1761 tablet_bootstrap.cc:492] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:46.979144  1908 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:46.980412  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:46.981292  1569 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:46.982831  1681 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:46.983341  1762 tablet_bootstrap.cc:654] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.980448  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:46.984025  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:46.971701  1763 tablet_bootstrap.cc:654] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:46.985353  1908 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:46.986743  1563 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:46.987347  1908 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.989144  1908 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:46.991577  1682 leader_election.cc:343] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:46.995945  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:47.000350  1761 tablet_bootstrap.cc:654] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.005491  1969 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.006394  1969 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.010551  1908 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.011516  1969 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:47.008322  1563 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
W20260709 13:00:46.998450  1607 leader_election.cc:343] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.024564  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:47.025409  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:47.026531  1948 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:47.027009  1948 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.027395  1948 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.028621  1763 tablet_bootstrap.cc:492] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:47.029425  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:47.030349  1563 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.029407  1763 ts_tablet_manager.cc:1403] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.085s	user 0.008s	sys 0.012s
I20260709 13:00:47.038664  1762 tablet_bootstrap.cc:492] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:47.039324  1762 ts_tablet_manager.cc:1403] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.097s	user 0.009s	sys 0.018s
I20260709 13:00:47.045989  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:47.047420  1944 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:47.048019  1944 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.048529  1944 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.043820  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:00:47.109484  1607 leader_election.cc:343] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.117116  1867 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.138351  1761 tablet_bootstrap.cc:492] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:47.139156  1761 ts_tablet_manager.cc:1403] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.161s	user 0.037s	sys 0.021s
I20260709 13:00:47.118112  1867 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.142460  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:47.173365  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 63958.5us
I20260709 13:00:47.176343  1763 raft_consensus.cc:359] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.176342  1563 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:47.177753  1763 raft_consensus.cc:385] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.178282  1763 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.179448  1763 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.183092  1762 raft_consensus.cc:359] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.184445  1762 raft_consensus.cc:385] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.184803  1762 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.148209  1908 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:47.187770  1762 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.188946  1761 raft_consensus.cc:359] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.190948  1761 raft_consensus.cc:385] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.191365  1761 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.192034  1761 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.191046  1763 ts_tablet_manager.cc:1434] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.156s	user 0.010s	sys 0.011s
W20260709 13:00:47.194499  1681 leader_election.cc:343] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.196938  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:47.195546  1681 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:47.203787  1969 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:00:47.203719  1533 leader_election.cc:343] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.204340  1969 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:47.204869  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:47.205353  1969 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.206028  1464 catalog_manager.cc:5697] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:47.209479  1682 leader_election.cc:343] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.210302  1908 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.210286  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:47.211194  1908 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.211512  1948 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.219604  1763 tablet_bootstrap.cc:492] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:47.222069  1948 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:47.220606  1867 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:47.217657  1976 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:47.223308  1973 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 13:00:47.217964  1682 leader_election.cc:343] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.225152  1908 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:47.213923  1944 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.230315  1865 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:47.230522  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:47.231631  1641 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 1: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:00:47.232086  1944 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:47.219100  1762 ts_tablet_manager.cc:1434] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.179s	user 0.026s	sys 0.023s
I20260709 13:00:47.234272  1762 tablet_bootstrap.cc:492] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:47.236006  1906 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:47.243054  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 49050.6us
I20260709 13:00:47.245736  1464 catalog_manager.cc:5697] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:47.248634  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:47.261693  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:47.262689  1563 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 1: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:00:47.264024  1980 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:47.265565  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:47.265718  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:47.265575  1763 tablet_bootstrap.cc:654] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.268404  1762 tablet_bootstrap.cc:654] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.268821  1761 ts_tablet_manager.cc:1434] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.129s	user 0.007s	sys 0.002s
W20260709 13:00:47.269424  1532 leader_election.cc:343] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.269866  1761 tablet_bootstrap.cc:492] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:47.270545  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:47.272493  1865 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:47.278347  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:47.279139  1564 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.286661  1465 catalog_manager.cc:5697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:47.299839  1763 tablet_bootstrap.cc:492] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
W20260709 13:00:47.317963  1607 leader_election.cc:343] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:47.320828  1607 leader_election.cc:343] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.330310  1681 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:47.332376  1763 ts_tablet_manager.cc:1403] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.113s	user 0.021s	sys 0.010s
I20260709 13:00:47.337163  1564 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:47.340590  1762 tablet_bootstrap.cc:492] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:47.341140  1762 ts_tablet_manager.cc:1403] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.107s	user 0.034s	sys 0.014s
I20260709 13:00:47.344606  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:47.345433  1761 tablet_bootstrap.cc:654] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.346828  1979 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:47.335223  1906 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.340063  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:47.351832  1567 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.355293  1762 raft_consensus.cc:359] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.356128  1762 raft_consensus.cc:385] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.356415  1762 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.348341  1979 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:47.359926  1762 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.363325  1979 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.377518  1762 ts_tablet_manager.cc:1434] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.001s
I20260709 13:00:47.390333  1762 tablet_bootstrap.cc:492] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:47.391271  1969 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:47.400936  1763 raft_consensus.cc:359] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.402071  1763 raft_consensus.cc:385] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.402665  1763 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.406140  1763 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.422164  1763 ts_tablet_manager.cc:1434] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.089s	user 0.008s	sys 0.000s
I20260709 13:00:47.428913  1466 catalog_manager.cc:5697] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:47.440394  1763 tablet_bootstrap.cc:492] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:47.442186  1777 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.442737  1777 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.444662  1777 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:47.447031  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:00:47.450538  1607 leader_election.cc:343] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.466854  1641 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.470963  1761 tablet_bootstrap.cc:492] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:47.471622  1761 ts_tablet_manager.cc:1403] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.202s	user 0.018s	sys 0.017s
I20260709 13:00:47.479549  1567 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:47.458289  1906 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:47.483592  1981 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.485194  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:47.484436  1981 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.487586  1979 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:00:47.487979  1515 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:47.488241  1979 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:47.488997  1981 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:47.489423  1969 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.496852  1969 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.489766  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:47.500180  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:47.501469  1713 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:47.511575  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:00:47.514583  1761 raft_consensus.cc:359] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:47.515326  1533 leader_election.cc:343] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.516213  1777 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:47.516604  1777 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.516949  1777 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.516747  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:47.517496  1979 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.521885  1969 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:47.518204  1714 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:47.522553  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:47.523386  1532 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:47.515403  1761 raft_consensus.cc:385] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.524065  1761 raft_consensus.cc:740] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.525107  1761 consensus_queue.cc:260] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.525624  1777 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:47.529067  1681 leader_election.cc:343] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.529954  1981 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:47.530339  1981 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.530644  1981 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.533433  1777 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:47.534071  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:47.533797  1948 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:47.537921  1969 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.543972  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:47.544796  1763 tablet_bootstrap.cc:654] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.545413  1643 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 1: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:00:47.548977  1761 ts_tablet_manager.cc:1434] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.074s	user 0.011s	sys 0.000s
I20260709 13:00:47.547731  1969 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.557291  1682 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:47.559466  1777 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:47.560813  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:47.561447  1714 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:47.568787  1607 leader_election.cc:343] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.572623  1762 tablet_bootstrap.cc:654] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.573555  1761 tablet_bootstrap.cc:492] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:47.571931  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:47.581763  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:47.563156  1969 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:47.586278  1564 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 1: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:00:47.596397  1681 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:47.603165  1813 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.603027  1802 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.608631  1802 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.613543  1802 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:47.619863  1969 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:00:47.626077  1465 catalog_manager.cc:5697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:47.624009  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:47.624447  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:47.653810  1763 tablet_bootstrap.cc:492] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:47.654307  1763 ts_tablet_manager.cc:1403] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.219s	user 0.012s	sys 0.005s
I20260709 13:00:47.638351  1813 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:47.656926  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:47.639792  1714 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:47.654356  1981 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.664896  1763 raft_consensus.cc:359] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.668912  1763 raft_consensus.cc:385] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.669579  1906 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:47.669276  1763 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.674619  1763 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.691074  1761 tablet_bootstrap.cc:654] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.691468  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:47.690656  1906 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
W20260709 13:00:47.693939  1533 leader_election.cc:343] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.677091  1979 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:47.698633  1763 ts_tablet_manager.cc:1434] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.002s
I20260709 13:00:47.700543  1763 tablet_bootstrap.cc:492] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:47.702032  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:47.703024  1714 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.706341  1981 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:47.699334  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:00:47.709800  1762 tablet_bootstrap.cc:492] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:47.711133  1762 ts_tablet_manager.cc:1403] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.331s	user 0.008s	sys 0.022s
I20260709 13:00:47.712440  1714 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:47.715287  1532 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:47.715996  1761 tablet_bootstrap.cc:492] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:47.716588  1981 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:47.717170  1981 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:47.718374  1981 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.722241  1761 ts_tablet_manager.cc:1403] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.159s	user 0.011s	sys 0.017s
I20260709 13:00:47.711374  1979 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:47.723682  1979 raft_consensus.cc:697] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:47.725534  1979 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.722210  1762 raft_consensus.cc:359] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.728397  1762 raft_consensus.cc:385] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.728683  1762 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.736673  1643 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.737351  1763 tablet_bootstrap.cc:654] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.731559  1762 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.745921  1762 ts_tablet_manager.cc:1434] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.002s
I20260709 13:00:47.747650  1762 tablet_bootstrap.cc:492] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:47.749436  1969 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:47.757567  1762 tablet_bootstrap.cc:654] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:47.762154  1607 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:47.746312  1761 raft_consensus.cc:359] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.763386  1761 raft_consensus.cc:385] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.765007  1761 raft_consensus.cc:740] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.766408  1761 consensus_queue.cc:260] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.781338  1466 catalog_manager.cc:5697] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:47.795559  1761 ts_tablet_manager.cc:1434] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.073s	user 0.008s	sys 0.000s
I20260709 13:00:47.796665  1761 tablet_bootstrap.cc:492] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:47.802785  1465 catalog_manager.cc:5697] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:47.805567  1763 tablet_bootstrap.cc:492] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:47.812899  1763 ts_tablet_manager.cc:1403] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.113s	user 0.012s	sys 0.008s
I20260709 13:00:47.812711  1969 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.813562  1969 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.815174  1761 tablet_bootstrap.cc:654] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.816154  1969 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:47.817385  1762 tablet_bootstrap.cc:492] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:47.818837  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:47.819684  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:47.820077  1762 ts_tablet_manager.cc:1403] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.073s	user 0.024s	sys 0.036s
W20260709 13:00:47.820495  1681 leader_election.cc:343] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:47.822018  1682 leader_election.cc:343] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.822552  1981 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.822430  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:47.823462  1981 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.823897  1762 raft_consensus.cc:359] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.824681  1762 raft_consensus.cc:385] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.824908  1906 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:47.824957  1762 raft_consensus.cc:740] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.823931  1763 raft_consensus.cc:359] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.826503  1763 raft_consensus.cc:385] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.826818  1763 raft_consensus.cc:740] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.827575  1763 consensus_queue.cc:260] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.827837  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:47.827855  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:47.829862  1981 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:47.830705  1533 leader_election.cc:343] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.830837  1762 consensus_queue.cc:260] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:47.832253  1532 leader_election.cc:343] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.834481  1532 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:47.843624  1981 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:47.834901  1762 ts_tablet_manager.cc:1434] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:00:47.840250  1763 ts_tablet_manager.cc:1434] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:00:47.840700  1761 tablet_bootstrap.cc:492] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:47.846704  1762 tablet_bootstrap.cc:492] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:47.848906  1763 tablet_bootstrap.cc:492] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:47.846689  1761 ts_tablet_manager.cc:1403] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.050s	user 0.018s	sys 0.003s
I20260709 13:00:47.858546  1761 raft_consensus.cc:359] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.859879  1761 raft_consensus.cc:385] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.860255  1761 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.862494  1944 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:47.862399  1761 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.866336  1761 ts_tablet_manager.cc:1434] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 13:00:47.873798  1761 tablet_bootstrap.cc:492] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:47.877012  1763 tablet_bootstrap.cc:654] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.874524  1813 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.879845  1813 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.880445  1762 tablet_bootstrap.cc:654] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.881961  1813 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:47.884382  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:47.886557  1563 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:47.888091  1681 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:47.890308  1969 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:47.891403  1969 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.890118  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:47.892164  1969 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:47.893536  1682 leader_election.cc:343] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.899709  1761 tablet_bootstrap.cc:654] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.908937  1762 tablet_bootstrap.cc:492] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:47.911381  1762 ts_tablet_manager.cc:1403] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.065s	user 0.021s	sys 0.000s
I20260709 13:00:47.915800  1763 tablet_bootstrap.cc:492] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:47.915565  1762 raft_consensus.cc:359] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.916376  1762 raft_consensus.cc:385] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.916824  1762 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.916944  1763 ts_tablet_manager.cc:1403] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.070s	user 0.014s	sys 0.011s
I20260709 13:00:47.918184  1762 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.920619  1762 ts_tablet_manager.cc:1434] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:00:47.921111  1763 raft_consensus.cc:359] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.921918  1762 tablet_bootstrap.cc:492] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:47.921931  1763 raft_consensus.cc:385] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.929250  1763 raft_consensus.cc:740] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.929610  1762 tablet_bootstrap.cc:654] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.930438  1763 consensus_queue.cc:260] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.931357  1761 tablet_bootstrap.cc:492] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:47.932689  1761 ts_tablet_manager.cc:1403] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.059s	user 0.017s	sys 0.000s
I20260709 13:00:47.936462  1761 raft_consensus.cc:359] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.935477  1981 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.938832  1981 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.943332  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:47.944628  1641 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:47.946465  1533 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:47.947813  1980 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:47.948249  1980 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.948561  1980 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:47.949414  1763 ts_tablet_manager.cc:1434] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.032s	user 0.010s	sys 0.000s
I20260709 13:00:47.950057  1813 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.935372  1969 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.950712  1813 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.953492  1813 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:47.954042  1969 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:47.950757  1763 tablet_bootstrap.cc:492] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:47.957858  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:47.958532  1563 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.960359  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:47.961539  1564 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:47.962416  1981 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:47.964321  1761 raft_consensus.cc:385] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:47.969730  1761 raft_consensus.cc:740] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:47.970513  1761 consensus_queue.cc:260] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.972765  1763 tablet_bootstrap.cc:654] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:47.975571  1681 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:47.977262  1969 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:47.977722  1969 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.978183  1969 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.981791  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:47.982174  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:47.982595  1643 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:47.985481  1761 ts_tablet_manager.cc:1434] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.000s
I20260709 13:00:47.986750  1981 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.987350  1981 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:47.991642  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:47.992350  1641 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:47.992586  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:47.993498  1715 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:47.994360  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:47.996479  1865 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:47.997195  1865 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.997555  1865 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:47.998245  1682 leader_election.cc:343] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.994177  1969 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.002218  1969 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:48.005026  1763 tablet_bootstrap.cc:492] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:48.010459  1763 ts_tablet_manager.cc:1403] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.060s	user 0.018s	sys 0.002s
I20260709 13:00:48.010972  1643 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:48.015354  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:48.016846  1682 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:48.018018  1813 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:48.019524  1813 raft_consensus.cc:697] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:48.020120  1763 raft_consensus.cc:359] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.021121  1763 raft_consensus.cc:385] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.021466  1763 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.022686  1813 consensus_queue.cc:237] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.005937  1563 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:48.026297  1981 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:48.027606  1682 leader_election.cc:343] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:47.968819  1980 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.034149  1761 tablet_bootstrap.cc:492] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:48.022794  1763 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.034478  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:48.038312  1714 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:48.040472  1763 ts_tablet_manager.cc:1434] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.028s	user 0.010s	sys 0.000s
I20260709 13:00:48.045563  1763 tablet_bootstrap.cc:492] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:48.057894  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:48.041424  1980 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:48.045439  1865 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:48.061456  1532 leader_election.cc:343] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:48.063527  1763 tablet_bootstrap.cc:654] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.067961  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:48.063772  1865 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:48.097698  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:48.079082  1714 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:00:48.102664  1563 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:48.081918  1944 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.104012  1944 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.107431  1944 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:48.075137  1532 leader_election.cc:343] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:48.109673  1532 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:48.111446  1865 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:48.112318  1865 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:48.113173  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:48.114496  1715 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:48.114991  1802 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:48.116732  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:48.119190  1802 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:48.120810  1802 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:48.124020  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:48.123394  1802 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:48.129078  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:48.124972  1643 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:48.130240  1564 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:48.134105  1979 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:00:48.144953  1563 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:48.146721  1681 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:48.148705  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:48.148967  1969 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:48.149777  1641 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:48.150549  1969 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:48.152449  1969 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.159111  1944 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.159869  1944 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.164264  1761 tablet_bootstrap.cc:654] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.166070  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:48.167320  1641 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:48.167593  1713 raft_consensus.cc:2393] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:00:48.113480  1865 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.162726  1944 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:48.178861  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:48.182962  1643 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:00:48.180313  1802 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.186813  1644 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:48.201654  1802 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:48.204824  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:48.206153  1762 tablet_bootstrap.cc:492] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:48.207084  1762 ts_tablet_manager.cc:1403] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.285s	user 0.098s	sys 0.114s
I20260709 13:00:48.211447  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:48.213826  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:48.214800  1564 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:48.217031  1607 consensus_peers.cc:597] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:48.220317  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:48.221333  1713 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:48.209616  2011 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:48.229513  2011 raft_consensus.cc:697] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:48.245697  2011 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.247844  1763 tablet_bootstrap.cc:492] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:48.248560  1763 ts_tablet_manager.cc:1403] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.203s	user 0.024s	sys 0.028s
I20260709 13:00:48.248368  1762 raft_consensus.cc:359] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.249080  1762 raft_consensus.cc:385] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.249492  1762 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.250208  1762 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.253082  1466 catalog_manager.cc:5697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:48.254109  1762 ts_tablet_manager.cc:1434] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.047s	user 0.011s	sys 0.006s
I20260709 13:00:48.255380  1762 tablet_bootstrap.cc:492] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:48.255383  1466 catalog_manager.cc:5697] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:48.264981  1763 raft_consensus.cc:359] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.266310  1763 raft_consensus.cc:385] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.266685  1763 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.267863  1763 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.271682  1980 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:48.275761  1763 ts_tablet_manager.cc:1434] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.027s	user 0.013s	sys 0.001s
I20260709 13:00:48.277209  1763 tablet_bootstrap.cc:492] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:48.281477  1802 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:48.285877  1761 tablet_bootstrap.cc:492] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:48.287003  1761 ts_tablet_manager.cc:1403] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.301s	user 0.007s	sys 0.017s
I20260709 13:00:48.289147  2013 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:00:48.321022  1564 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:48.341745  1761 raft_consensus.cc:359] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.345314  1761 raft_consensus.cc:385] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.345971  1761 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.349375  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:00:48.350383  1670 tablet.cc:2406] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:00:48.350497  1802 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:48.351155  1802 raft_consensus.cc:697] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:48.352142  1802 consensus_queue.cc:237] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.355965  1713 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:48.356778  1763 tablet_bootstrap.cc:654] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.360045  1762 tablet_bootstrap.cc:654] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.360572  1761 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.367147  1777 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:00:48.367723  2020 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.065s	user 0.003s	sys 0.000s
I20260709 13:00:48.371193  1761 ts_tablet_manager.cc:1434] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.084s	user 0.009s	sys 0.001s
I20260709 13:00:48.372812  1761 tablet_bootstrap.cc:492] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:48.381032  1763 tablet_bootstrap.cc:492] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:48.382264  1763 ts_tablet_manager.cc:1403] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.105s	user 0.016s	sys 0.000s
I20260709 13:00:48.389568  1762 tablet_bootstrap.cc:492] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:48.390302  1762 ts_tablet_manager.cc:1403] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.135s	user 0.014s	sys 0.003s
I20260709 13:00:48.394084  1762 raft_consensus.cc:359] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.397145  1762 raft_consensus.cc:385] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.396016  1763 raft_consensus.cc:359] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.397783  1763 raft_consensus.cc:385] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.398309  1763 raft_consensus.cc:740] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.399407  1763 consensus_queue.cc:260] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.401844  1763 ts_tablet_manager.cc:1434] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:00:48.403475  1763 tablet_bootstrap.cc:492] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:48.402311  1762 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.407372  1762 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.410281  1762 ts_tablet_manager.cc:1434] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.004s
I20260709 13:00:48.411545  1762 tablet_bootstrap.cc:492] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:48.415477  1466 catalog_manager.cc:5697] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 2, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:48.421161  1763 tablet_bootstrap.cc:654] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.422367  1761 tablet_bootstrap.cc:654] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.433871  1908 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.434500  1908 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.440315  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:48.445861  1643 raft_consensus.cc:2393] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 1: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:00:48.438737  1969 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:00:48.439149  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:48.446461  2012 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.454139  2012 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.438333  1908 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:48.458995  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:48.459859  2012 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:48.458992  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:48.464736  1682 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:00:48.470039  1532 leader_election.cc:343] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:48.479339  1533 leader_election.cc:343] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:48.479894  1533 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:48.481173  2012 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:48.481177  1908 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:48.479475  1762 tablet_bootstrap.cc:654] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.509224  1761 tablet_bootstrap.cc:492] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:48.506857  1465 catalog_manager.cc:5697] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:48.509809  1761 ts_tablet_manager.cc:1403] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.138s	user 0.025s	sys 0.002s
I20260709 13:00:48.508277  1908 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:48.520556  1761 raft_consensus.cc:359] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.521826  1761 raft_consensus.cc:385] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.522182  1761 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.522893  1761 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.526156  1908 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.527709  1763 tablet_bootstrap.cc:492] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:48.528311  1763 ts_tablet_manager.cc:1403] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.125s	user 0.011s	sys 0.010s
I20260709 13:00:48.527292  1908 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.534600  1761 ts_tablet_manager.cc:1434] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:00:48.534708  1908 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:48.534745  1464 catalog_manager.cc:5697] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:48.540079  1762 tablet_bootstrap.cc:492] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:48.565339  1762 ts_tablet_manager.cc:1403] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.154s	user 0.022s	sys 0.004s
I20260709 13:00:48.561868  1908 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:00:48.568032  1763 raft_consensus.cc:359] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.543766  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:48.569235  1763 raft_consensus.cc:385] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.569293  1813 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:48.543669  1761 tablet_bootstrap.cc:492] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:48.573277  1762 raft_consensus.cc:359] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.573999  1762 raft_consensus.cc:385] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.574770  1762 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.575459  1762 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.578418  1762 ts_tablet_manager.cc:1434] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.003s
I20260709 13:00:48.580508  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:48.585248  1762 tablet_bootstrap.cc:492] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:48.569764  1763 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.598280  1763 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.601480  1763 ts_tablet_manager.cc:1434] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.073s	user 0.004s	sys 0.006s
I20260709 13:00:48.610388  1713 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:00:48.612874  2011 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:48.589953  1761 tablet_bootstrap.cc:654] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:48.591580  1681 leader_election.cc:343] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:48.602789  1763 tablet_bootstrap.cc:492] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
W20260709 13:00:48.615758  1682 leader_election.cc:343] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:48.622702  1682 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:48.627482  2034 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:48.638123  1762 tablet_bootstrap.cc:654] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.640125  1641 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:48.661195  2030 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:00:48.665982  1763 tablet_bootstrap.cc:654] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.668392  2011 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.669160  2011 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.672294  2011 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:48.675253  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:48.678304  1515 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:00:48.684333  1533 leader_election.cc:343] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:48.679070  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:48.685883  1712 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:48.688118  1532 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:48.689752  2012 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:48.690150  2012 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:48.690677  2012 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:48.707944  1761 tablet_bootstrap.cc:492] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:48.708688  1761 ts_tablet_manager.cc:1403] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.165s	user 0.022s	sys 0.007s
I20260709 13:00:48.721560  2011 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:48.722149  1761 raft_consensus.cc:359] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.722831  1761 raft_consensus.cc:385] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.723145  1761 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.723793  1761 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.731527  1643 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:00:48.733992  2011 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:48.735991  1761 ts_tablet_manager.cc:1434] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.001s
I20260709 13:00:48.768157  1979 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:48.777828  1813 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.778380  1813 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.778898  1761 tablet_bootstrap.cc:492] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:48.783648  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:48.785017  1644 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:48.787724  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:48.788594  1564 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:48.791368  1681 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:48.792498  2034 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:48.792917  2034 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:48.793833  2034 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:48.771183  2012 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.799902  2012 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:48.775689  1762 tablet_bootstrap.cc:492] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:48.801146  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:48.802138  1712 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:48.801276  1762 ts_tablet_manager.cc:1403] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.216s	user 0.020s	sys 0.029s
I20260709 13:00:48.807760  1813 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:48.812333  1762 raft_consensus.cc:359] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.813402  1762 raft_consensus.cc:385] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.814320  1762 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.813467  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:48.831601  1865 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:48.820564  1761 tablet_bootstrap.cc:654] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:48.836939  1762 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.831753  2034 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.851537  1802 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.854802  1712 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:48.858475  1802 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.861372  1802 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:48.861896  1533 leader_election.cc:343] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:48.861207  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:48.865387  1568 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:48.867174  1762 ts_tablet_manager.cc:1434] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.005s
I20260709 13:00:48.869206  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:48.869328  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:48.870111  1563 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:48.870424  1763 tablet_bootstrap.cc:492] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
W20260709 13:00:48.870841  1607 leader_election.cc:343] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:48.871165  1763 ts_tablet_manager.cc:1403] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.269s	user 0.046s	sys 0.052s
I20260709 13:00:48.875020  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:48.876121  1979 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:48.876556  1979 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:48.876917  1979 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:48.880611  1762 tablet_bootstrap.cc:492] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:48.884572  2034 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:48.888859  1532 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:48.890854  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:48.898497  1644 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:48.911026  2011 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:48.911794  2011 raft_consensus.cc:697] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:48.912890  1568 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:48.913273  2011 consensus_queue.cc:237] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.917593  1761 tablet_bootstrap.cc:492] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:48.918399  1761 ts_tablet_manager.cc:1403] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.140s	user 0.021s	sys 0.001s
I20260709 13:00:48.925730  1681 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:48.926965  2034 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:48.927531  2034 raft_consensus.cc:697] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:48.928440  2034 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.931254  1644 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:48.932433  1761 raft_consensus.cc:359] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.933203  1761 raft_consensus.cc:385] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.933638  1761 raft_consensus.cc:740] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.935653  1908 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.936640  1761 consensus_queue.cc:260] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.936275  1908 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.920662  1948 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:00:48.939859  1908 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:48.943272  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:48.940460  1761 ts_tablet_manager.cc:1434] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.003s
I20260709 13:00:48.944290  1763 raft_consensus.cc:359] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.948019  1763 raft_consensus.cc:385] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:48.948318  1763 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:48.959992  1763 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:48.971532  1761 tablet_bootstrap.cc:492] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:48.967304  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:48.978566  1682 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:48.987083  1763 ts_tablet_manager.cc:1434] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.115s	user 0.010s	sys 0.001s
I20260709 13:00:48.982956  1813 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:48.995777  1762 tablet_bootstrap.cc:654] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.003640  1465 catalog_manager.cc:5697] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:49.008980  1763 tablet_bootstrap.cc:492] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:49.010699  1979 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.032359  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:49.032971  1564 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.033116  1979 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:49.038196  1761 tablet_bootstrap.cc:654] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.041946  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:00:49.048688  1607 leader_election.cc:343] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.055263  1763 tablet_bootstrap.cc:654] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.055663  1564 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:49.057193  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:49.058130  1762 tablet_bootstrap.cc:492] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:49.058713  1979 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:49.058763  1762 ts_tablet_manager.cc:1403] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.184s	user 0.010s	sys 0.010s
I20260709 13:00:49.059372  1979 raft_consensus.cc:697] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:49.082369  1979 consensus_queue.cc:237] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.088514  1464 catalog_manager.cc:5697] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:49.103314  1763 tablet_bootstrap.cc:492] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:49.103847  1763 ts_tablet_manager.cc:1403] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.095s	user 0.012s	sys 0.002s
I20260709 13:00:49.108587  1763 raft_consensus.cc:359] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.109519  1763 raft_consensus.cc:385] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.109834  1763 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.091919  1762 raft_consensus.cc:359] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.115352  1762 raft_consensus.cc:385] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.115624  1762 raft_consensus.cc:740] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.116251  1762 consensus_queue.cc:260] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.096158  1761 tablet_bootstrap.cc:492] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:49.119270  1761 ts_tablet_manager.cc:1403] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.165s	user 0.027s	sys 0.012s
I20260709 13:00:49.118788  1762 ts_tablet_manager.cc:1434] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.000s
I20260709 13:00:49.126119  1464 catalog_manager.cc:5697] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:49.127805  1761 raft_consensus.cc:359] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.129066  1761 raft_consensus.cc:385] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.129377  1762 tablet_bootstrap.cc:492] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:49.129338  1761 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.130324  1761 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.134071  1763 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.135363  1761 ts_tablet_manager.cc:1434] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.005s
I20260709 13:00:49.136685  1761 tablet_bootstrap.cc:492] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:49.140089  1762 tablet_bootstrap.cc:654] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.136763  1763 ts_tablet_manager.cc:1434] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.002s
I20260709 13:00:49.141660  1763 tablet_bootstrap.cc:492] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:49.147832  1762 tablet_bootstrap.cc:492] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:49.148438  1762 ts_tablet_manager.cc:1403] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.000s
I20260709 13:00:49.151494  1763 tablet_bootstrap.cc:654] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.153286  1762 raft_consensus.cc:359] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.154795  1762 raft_consensus.cc:385] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.155262  1762 raft_consensus.cc:740] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.156267  1762 consensus_queue.cc:260] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.159067  1762 ts_tablet_manager.cc:1434] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:00:49.159758  1761 tablet_bootstrap.cc:654] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.160823  1762 tablet_bootstrap.cc:492] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:49.161155  1763 tablet_bootstrap.cc:492] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:49.161736  1763 ts_tablet_manager.cc:1403] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.000s
I20260709 13:00:49.162113  2045 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.162688  2045 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.164741  1979 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.165474  1979 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.167280  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:49.167934  1643 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:49.169186  1979 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:49.170100  1762 tablet_bootstrap.cc:654] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.171103  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:49.172029  1713 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:49.172068  1763 raft_consensus.cc:359] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.173259  1763 raft_consensus.cc:385] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.173629  1763 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.175010  1763 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.181006  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:49.182855  1979 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:49.183506  1979 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.183969  1979 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.189467  1533 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:49.190866  2013 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:49.191527  2013 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.192025  2013 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.192320  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:49.199468  2045 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:49.205358  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:49.206148  1714 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:49.212265  1763 ts_tablet_manager.cc:1434] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.050s	user 0.013s	sys 0.000s
I20260709 13:00:49.219173  1948 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.220096  1948 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.238505  1948 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:49.240592  2013 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.240988  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:00:49.243919  1607 leader_election.cc:343] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.244546  1564 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:00:49.244961  2013 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:49.252504  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:49.259593  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:49.261749  1563 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:49.266161  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:49.267261  1714 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.276870  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:49.279699  1802 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:49.280246  1802 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.280622  1802 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.311931  1908 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.312893  1908 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.313987  1762 tablet_bootstrap.cc:492] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:49.314716  1762 ts_tablet_manager.cc:1403] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.154s	user 0.057s	sys 0.063s
I20260709 13:00:49.320426  1763 tablet_bootstrap.cc:492] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:49.320451  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:49.318710  1908 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:49.301009  1979 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.319685  1802 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.332165  1641 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 1: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:00:49.334221  1979 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:49.338763  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:49.339975  1564 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:00:49.337141  1567 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.341262  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:49.342067  1568 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 1: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:00:49.342051  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:49.370867  1762 raft_consensus.cc:359] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.372645  1762 raft_consensus.cc:385] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.372946  1762 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.375494  1762 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.373281  2045 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.380342  1813 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.380834  1813 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.382166  2045 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.383832  1762 ts_tablet_manager.cc:1434] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.068s	user 0.008s	sys 0.001s
I20260709 13:00:49.383881  1813 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:49.391104  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:49.395424  1761 tablet_bootstrap.cc:492] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:49.396018  1761 ts_tablet_manager.cc:1403] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.260s	user 0.071s	sys 0.092s
I20260709 13:00:49.399504  1979 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:49.407388  1762 tablet_bootstrap.cc:492] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:49.408329  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:49.409876  1641 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.411377  2013 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.412170  1802 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:49.412006  2013 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.415086  2011 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:49.414637  2013 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:49.420486  1682 leader_election.cc:343] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.419847  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:49.421845  1564 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:49.422577  1761 raft_consensus.cc:359] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.423911  1761 raft_consensus.cc:385] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.424949  1718 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.425791  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:49.426795  2045 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:49.427706  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:00:49.428279  1533 leader_election.cc:343] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.428368  1713 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:49.429919  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:49.430454  1532 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:49.430593  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:00:49.432276  1532 leader_election.cc:343] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.430313  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:49.424204  1761 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
W20260709 13:00:49.433765  1533 leader_election.cc:343] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.439512  1715 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.434757  2045 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:49.438218  1763 tablet_bootstrap.cc:654] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.439836  1533 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:49.433980  1681 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:49.449334  1813 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.450044  1681 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:49.440514  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:49.441372  1761 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.458137  1568 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.447940  2045 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.448451  2044 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:49.461438  2045 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.457584  1908 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:49.461954  1908 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.462339  1908 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:49.456881  1979 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.468039  1979 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.472123  1813 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:49.474254  2030 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:49.419654  1714 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:49.478811  1761 ts_tablet_manager.cc:1434] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.082s	user 0.006s	sys 0.003s
I20260709 13:00:49.476583  1979 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:49.490067  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:49.497545  1532 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:49.497586  1761 tablet_bootstrap.cc:492] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:49.497670  1908 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.485710  1568 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:49.506223  2044 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:49.518649  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:49.528669  1813 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:49.531633  1715 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:49.537005  2044 raft_consensus.cc:697] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:49.536792  2013 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:49.538211  2044 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:49.581301  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 73207.9us
I20260709 13:00:49.585954  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:49.593750  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:49.587317  1802 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:00:49.597311  1979 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:49.621342  1763 tablet_bootstrap.cc:492] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:49.621882  1908 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:49.618508  1802 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:49.621918  1763 ts_tablet_manager.cc:1403] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.311s	user 0.013s	sys 0.017s
I20260709 13:00:49.627491  1979 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:49.644347  1762 tablet_bootstrap.cc:654] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:49.695581  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 53419.6us
I20260709 13:00:49.697676  1761 tablet_bootstrap.cc:654] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.640137  1979 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.646903  2045 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:49.731557  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 79232.2us
I20260709 13:00:49.769115  1763 raft_consensus.cc:359] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.770272  1763 raft_consensus.cc:385] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.771171  1763 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.777419  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:49.778528  2066 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:00:49.779659  1682 leader_election.cc:343] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:49.767060  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 107545us
I20260709 13:00:49.799912  1762 tablet_bootstrap.cc:492] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:49.800592  1762 ts_tablet_manager.cc:1403] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.394s	user 0.010s	sys 0.014s
I20260709 13:00:49.805027  2045 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:49.806742  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:00:49.807646  2065 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.108s	user 0.000s	sys 0.002s
I20260709 13:00:49.819643  1568 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 1: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:00:49.823488  1762 raft_consensus.cc:359] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.824532  1762 raft_consensus.cc:385] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:49.825063  1762 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.826527  1762 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.832284  1762 ts_tablet_manager.cc:1434] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.004s
W20260709 13:00:49.848182  2077 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.068s	user 0.005s	sys 0.000s
I20260709 13:00:49.863561  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:49.864442  1718 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 1: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:00:49.867204  1762 tablet_bootstrap.cc:492] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:49.869179  2075 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.874794  1681 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:49.900599  2063 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:49.870219  2075 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.895571  1761 tablet_bootstrap.cc:492] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:49.902525  1761 ts_tablet_manager.cc:1403] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.410s	user 0.021s	sys 0.008s
W20260709 13:00:49.924337  1515 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:49.923945  1763 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.898550  1467 catalog_manager.cc:5697] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:49.925748  1761 raft_consensus.cc:359] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.926431  1763 ts_tablet_manager.cc:1434] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.300s	user 0.008s	sys 0.005s
I20260709 13:00:49.926651  1761 raft_consensus.cc:385] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:00:49.927060  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 52392.7us
I20260709 13:00:49.920867  1466 catalog_manager.cc:5697] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:49.928668  1763 tablet_bootstrap.cc:492] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:49.869493  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:49.932968  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:00:49.934790  1532 leader_election.cc:343] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:49.940426  2075 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:49.945180  1762 tablet_bootstrap.cc:654] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.951040  1761 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:49.953624  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:49.953991  1761 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.957103  1761 ts_tablet_manager.cc:1434] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.054s	user 0.011s	sys 0.002s
I20260709 13:00:49.958400  1761 tablet_bootstrap.cc:492] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:49.961292  2044 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:49.975894  1761 tablet_bootstrap.cc:654] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:49.992633  1979 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.993450  1979 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:49.996871  1979 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:49.999744  1533 leader_election.cc:343] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:50.003495  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:50.035815  1566 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:50.037506  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:50.038652  1979 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:50.000558  1533 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:50.039358  1979 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.040026  1979 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.040481  2072 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:50.034217  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:50.042382  1718 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
W20260709 13:00:50.044653  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 44983.7us
W20260709 13:00:50.046784  1533 leader_election.cc:343] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:50.047398  1533 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:50.050325  2072 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:50.052551  1762 tablet_bootstrap.cc:492] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:50.053085  1762 ts_tablet_manager.cc:1403] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.186s	user 0.035s	sys 0.047s
I20260709 13:00:50.054898  2072 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.055536  2072 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.056319  1762 raft_consensus.cc:359] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.058393  1762 raft_consensus.cc:385] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.058697  1762 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.059412  1762 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.062637  1762 ts_tablet_manager.cc:1434] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:00:50.068918  1762 tablet_bootstrap.cc:492] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:50.076205  2086 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:00:50.081594  2072 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:50.083539  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:50.084848  1761 tablet_bootstrap.cc:492] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:50.085420  1761 ts_tablet_manager.cc:1403] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.127s	user 0.032s	sys 0.053s
I20260709 13:00:50.089272  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:50.097503  1533 leader_election.cc:343] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:50.100591  2090 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:00:50.103967  1763 tablet_bootstrap.cc:654] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.104799  1761 raft_consensus.cc:359] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.105614  1761 raft_consensus.cc:385] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.105978  1761 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.107306  1761 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:50.107687  1532 leader_election.cc:343] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:50.111363  1532 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:50.112169  2072 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:50.103047  1979 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.116485  1762 tablet_bootstrap.cc:654] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.124814  2086 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.125385  2086 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.128675  1761 ts_tablet_manager.cc:1434] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.043s	user 0.003s	sys 0.005s
I20260709 13:00:50.133384  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:50.130983  2086 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:50.135196  1532 leader_election.cc:343] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:50.136173  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:50.137849  1533 leader_election.cc:343] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:50.138260  1533 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:50.141757  2074 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:50.142175  1761 tablet_bootstrap.cc:492] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:50.152323  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:50.153678  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:50.154171  1565 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.148662  1979 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:50.154482  1712 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.186477  1565 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:50.191465  1763 tablet_bootstrap.cc:492] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:50.192680  1763 ts_tablet_manager.cc:1403] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.264s	user 0.024s	sys 0.004s
I20260709 13:00:50.198230  1763 raft_consensus.cc:359] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.199251  1763 raft_consensus.cc:385] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.199638  1763 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.200354  1763 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.205687  1763 ts_tablet_manager.cc:1434] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:00:50.207245  1761 tablet_bootstrap.cc:654] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.208567  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:50.209506  1979 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:50.211297  1979 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:50.212529  1979 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.210345  1712 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:50.210595  1763 tablet_bootstrap.cc:492] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:50.272143  1763 tablet_bootstrap.cc:654] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.279006  1761 tablet_bootstrap.cc:492] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:50.279637  1761 ts_tablet_manager.cc:1403] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.138s	user 0.024s	sys 0.036s
I20260709 13:00:50.280467  2066 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:50.281682  1464 catalog_manager.cc:5697] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.285599  1762 tablet_bootstrap.cc:492] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:50.286362  1762 ts_tablet_manager.cc:1403] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.220s	user 0.035s	sys 0.059s
I20260709 13:00:50.295006  1761 raft_consensus.cc:359] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.296298  1761 raft_consensus.cc:385] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.297050  1761 raft_consensus.cc:740] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.301064  1762 raft_consensus.cc:359] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.301899  1762 raft_consensus.cc:385] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.302275  1762 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.303722  1762 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.301467  1761 consensus_queue.cc:260] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.307971  1761 ts_tablet_manager.cc:1434] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.001s
I20260709 13:00:50.308293  1763 tablet_bootstrap.cc:492] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:50.308853  1763 ts_tablet_manager.cc:1403] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.098s	user 0.017s	sys 0.003s
I20260709 13:00:50.309432  1761 tablet_bootstrap.cc:492] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:50.318776  1761 tablet_bootstrap.cc:654] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.318933  1763 raft_consensus.cc:359] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.332337  1763 raft_consensus.cc:385] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.332705  1763 raft_consensus.cc:740] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.333575  1763 consensus_queue.cc:260] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.324555  1762 ts_tablet_manager.cc:1434] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 13:00:50.341728  1763 ts_tablet_manager.cc:1434] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:00:50.349707  1763 tablet_bootstrap.cc:492] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:50.341825  1762 tablet_bootstrap.cc:492] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:50.358310  1763 tablet_bootstrap.cc:654] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.360536  1761 tablet_bootstrap.cc:492] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:50.366104  1761 ts_tablet_manager.cc:1403] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.057s	user 0.020s	sys 0.008s
I20260709 13:00:50.369601  1761 raft_consensus.cc:359] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.370314  2049 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:50.370292  1761 raft_consensus.cc:385] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.372457  1761 raft_consensus.cc:740] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.373594  1761 consensus_queue.cc:260] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.377477  1761 ts_tablet_manager.cc:1434] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:00:50.383867  1761 tablet_bootstrap.cc:492] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:50.393632  1762 tablet_bootstrap.cc:654] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.397185  1763 tablet_bootstrap.cc:492] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:50.398742  1761 tablet_bootstrap.cc:654] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.409477  1763 ts_tablet_manager.cc:1403] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.060s	user 0.021s	sys 0.016s
I20260709 13:00:50.412667  1763 raft_consensus.cc:359] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.413390  1763 raft_consensus.cc:385] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.413682  1763 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.414579  1763 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.416441  1763 ts_tablet_manager.cc:1434] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:00:50.418574  1763 tablet_bootstrap.cc:492] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:50.421912  1762 tablet_bootstrap.cc:492] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:50.422557  1762 ts_tablet_manager.cc:1403] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.081s	user 0.019s	sys 0.000s
I20260709 13:00:50.424384  1761 tablet_bootstrap.cc:492] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:50.424986  1761 ts_tablet_manager.cc:1403] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.014s
I20260709 13:00:50.425454  1763 tablet_bootstrap.cc:654] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.425251  1762 raft_consensus.cc:359] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.426587  1762 raft_consensus.cc:385] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.427213  1762 raft_consensus.cc:740] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.427973  1762 consensus_queue.cc:260] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.428242  1761 raft_consensus.cc:359] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.429343  1761 raft_consensus.cc:385] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.429787  1761 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.430789  1761 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.435645  1762 ts_tablet_manager.cc:1434] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:00:50.436801  1762 tablet_bootstrap.cc:492] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:50.452443  1762 tablet_bootstrap.cc:654] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.463734  1761 ts_tablet_manager.cc:1434] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 13:00:50.465560  1761 tablet_bootstrap.cc:492] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:50.466301  1762 tablet_bootstrap.cc:492] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:50.467134  1762 ts_tablet_manager.cc:1403] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.001s
I20260709 13:00:50.471206  1762 raft_consensus.cc:359] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.472651  1762 raft_consensus.cc:385] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.473032  1762 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.474306  1762 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.475183  2034 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.475733  2034 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.480132  1763 tablet_bootstrap.cc:492] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:50.481383  2034 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:50.482657  1813 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.483140  1813 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.483566  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:50.484257  1641 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:50.485477  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:50.486270  1908 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:50.486621  1908 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.487336  1908 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.488950  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:50.489614  1569 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:50.491230  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:50.491886  1565 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:50.494220  1681 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:50.495796  2063 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:50.496380  2063 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.496917  2063 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.508175  1763 ts_tablet_manager.cc:1403] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.090s	user 0.028s	sys 0.026s
I20260709 13:00:50.508257  1813 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:50.510006  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:50.517191  1763 raft_consensus.cc:359] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.518056  1763 raft_consensus.cc:385] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.518316  1908 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.520910  1908 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:50.524991  1762 ts_tablet_manager.cc:1434] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.058s	user 0.011s	sys 0.000s
I20260709 13:00:50.526787  1762 tablet_bootstrap.cc:492] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:50.529630  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:50.530313  1643 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.531013  1761 tablet_bootstrap.cc:654] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.518338  1763 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.535980  1763 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.538271  1641 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:50.539810  1643 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:50.541383  1763 ts_tablet_manager.cc:1434] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.001s
I20260709 13:00:50.541602  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:50.542374  2108 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:50.543074  2108 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:50.544581  2108 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.545755  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:50.546433  1569 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.553526  1763 tablet_bootstrap.cc:492] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:50.555473  2049 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.556156  2049 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.560439  2066 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.561513  1762 tablet_bootstrap.cc:654] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.561189  2066 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.563751  2075 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.565340  2075 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.568480  1761 tablet_bootstrap.cc:492] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:50.568912  2066 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:50.569002  1761 ts_tablet_manager.cc:1403] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.104s	user 0.015s	sys 0.000s
I20260709 13:00:50.569463  2049 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:50.568970  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:50.570604  1714 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:50.570968  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:50.571805  1716 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:00:50.573521  1607 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:50.575110  2049 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:50.585867  2049 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.586576  2049 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.589715  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:50.576400  1763 tablet_bootstrap.cc:654] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.572281  1761 raft_consensus.cc:359] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.591620  1761 raft_consensus.cc:385] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.578279  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:50.581223  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:50.584081  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:50.601325  1563 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:00:50.592306  1761 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
W20260709 13:00:50.594894  1533 leader_election.cc:343] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:50.601035  1564 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:50.607030  2075 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:50.609344  1532 leader_election.cc:343] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:50.610049  1532 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:50.603926  1761 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.613463  2074 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:50.619809  1761 ts_tablet_manager.cc:1434] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.001s
I20260709 13:00:50.620895  1761 tablet_bootstrap.cc:492] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
W20260709 13:00:50.625280  2110 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:00:50.628767  1569 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:50.599601  2063 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.632237  2049 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.641639  1762 tablet_bootstrap.cc:492] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:50.642553  1762 ts_tablet_manager.cc:1403] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.116s	user 0.011s	sys 0.044s
I20260709 13:00:50.644918  2063 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:50.646096  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:50.646988  1643 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.649639  1607 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:50.650993  2112 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.655251  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:50.657335  1569 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.661365  1761 tablet_bootstrap.cc:654] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.662117  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:50.662827  1718 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.659811  1763 tablet_bootstrap.cc:492] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:50.666482  1762 raft_consensus.cc:359] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.667249  1762 raft_consensus.cc:385] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.667351  1763 ts_tablet_manager.cc:1403] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.114s	user 0.016s	sys 0.002s
I20260709 13:00:50.667575  1762 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.668339  1762 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.635682  2049 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:50.671720  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:50.672465  1564 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.672966  1763 raft_consensus.cc:359] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.673648  1763 raft_consensus.cc:385] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.673926  1763 raft_consensus.cc:740] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.664251  1643 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:50.675895  1682 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:50.675525  1763 consensus_queue.cc:260] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.678501  2075 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.679246  2075 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.682060  2075 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:50.683120  1908 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:50.685156  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:50.685907  1718 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:50.686124  1641 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:50.670775  1762 ts_tablet_manager.cc:1434] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.000s
I20260709 13:00:50.688241  1908 raft_consensus.cc:697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:50.678488  2112 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:50.688452  1762 tablet_bootstrap.cc:492] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:50.692168  1564 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:50.691466  1533 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:50.689446  1908 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.697592  2072 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:50.698542  2072 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.699100  2072 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.702067  1569 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:50.683257  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:50.697628  2086 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.711673  1761 tablet_bootstrap.cc:492] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:50.711651  1466 catalog_manager.cc:5697] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:50.713117  1761 ts_tablet_manager.cc:1403] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.092s	user 0.011s	sys 0.004s
I20260709 13:00:50.713034  2072 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.716997  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:50.719868  2072 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:50.705879  1607 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:50.717964  1761 raft_consensus.cc:359] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.728042  1761 raft_consensus.cc:385] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:50.728507  1761 raft_consensus.cc:740] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:50.729652  1761 consensus_queue.cc:260] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.730592  1714 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:50.734495  1763 ts_tablet_manager.cc:1434] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 13:00:50.730746  2086 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.736595  1763 tablet_bootstrap.cc:492] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:50.748202  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:50.753379  1761 ts_tablet_manager.cc:1434] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.040s	user 0.011s	sys 0.000s
I20260709 13:00:50.753822  2086 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:50.755087  1532 leader_election.cc:343] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:50.745730  1979 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:00:50.761076  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 69698.1us
I20260709 13:00:50.760607  1979 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:50.777503  2115 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.776202  1761 tablet_bootstrap.cc:492] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:50.760458  2066 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:00:50.778591  1979 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.778240  2115 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.848371  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:50.849318  1641 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:50.849968  2115 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:50.863111  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:50.868870  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:50.873214  1533 leader_election.cc:343] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:50.873703  1533 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:00:50.876848  1533 leader_election.cc:343] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:50.885798  1641 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:50.887022  1533 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:50.875197  2074 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:50.887778  2115 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:50.895999  1762 tablet_bootstrap.cc:654] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.897392  2115 raft_consensus.cc:697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:50.899776  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:50.900784  1761 tablet_bootstrap.cc:654] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:50.905313  1532 leader_election.cc:343] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:50.906073  1532 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:50.904244  1802 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:00:50.911587  1464 catalog_manager.cc:5697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:50.924000  1763 tablet_bootstrap.cc:654] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:50.922091  2075 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:50.899776  2115 consensus_queue.cc:237] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:50.964964  1467 catalog_manager.cc:5697] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:51.045136  2034 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.045855  2034 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.072525  2034 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:51.076022  1464 catalog_manager.cc:5697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:51.086581  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:51.096309  1682 leader_election.cc:343] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:51.119330  1762 tablet_bootstrap.cc:492] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:51.120090  1762 ts_tablet_manager.cc:1403] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.432s	user 0.034s	sys 0.046s
I20260709 13:00:51.123951  1761 tablet_bootstrap.cc:492] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:51.124711  1761 ts_tablet_manager.cc:1403] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.354s	user 0.023s	sys 0.026s
W20260709 13:00:51.197451  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 233738us
I20260709 13:00:51.252853  1761 raft_consensus.cc:359] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.263810  2116 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.323076  2116 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.325551  2116 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:51.325465  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:51.313975  1763 tablet_bootstrap.cc:492] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:51.326854  1763 ts_tablet_manager.cc:1403] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.591s	user 0.035s	sys 0.045s
I20260709 13:00:51.326707  1569 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
W20260709 13:00:51.329267  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 187645us
I20260709 13:00:51.331429  1762 raft_consensus.cc:359] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.333923  2034 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:51.334497  1762 raft_consensus.cc:385] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:51.334834  1762 raft_consensus.cc:740] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:51.341557  1763 raft_consensus.cc:359] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.341804  1762 consensus_queue.cc:260] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.342404  1763 raft_consensus.cc:385] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:51.348848  1763 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:51.348656  1762 ts_tablet_manager.cc:1434] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.228s	user 0.016s	sys 0.004s
I20260709 13:00:51.350150  1762 tablet_bootstrap.cc:492] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:51.351382  1761 raft_consensus.cc:385] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:51.351897  1761 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:51.354310  1761 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.349833  1763 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.357254  1681 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:51.359506  2034 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:51.360127  2034 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.361238  2034 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:51.363932  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:51.367138  1533 leader_election.cc:343] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:51.374866  2034 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.382990  1763 ts_tablet_manager.cc:1434] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.001s
I20260709 13:00:51.383462  1762 tablet_bootstrap.cc:654] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:51.385830  1763 tablet_bootstrap.cc:492] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:51.385346  1908 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:51.394302  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:51.395426  1566 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:51.396092  1515 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:51.402442  2034 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:51.404947  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:51.410805  1566 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:51.411978  1763 tablet_bootstrap.cc:654] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:51.412079  1813 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:51.414299  2119 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:51.414666  1813 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:51.420109  1681 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:51.414350  1762 tablet_bootstrap.cc:492] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:51.421296  2116 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:51.422155  1762 ts_tablet_manager.cc:1403] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.072s	user 0.009s	sys 0.004s
I20260709 13:00:51.387574  1761 ts_tablet_manager.cc:1434] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.262s	user 0.015s	sys 0.015s
I20260709 13:00:51.392465  1979 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.426645  1761 tablet_bootstrap.cc:492] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:51.426491  1979 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:51.427431  1532 leader_election.cc:343] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:51.427920  1532 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:51.428785  1979 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:51.429206  2116 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:51.430392  1908 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:51.431231  1908 raft_consensus.cc:697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:51.432605  1908 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.461484  1761 tablet_bootstrap.cc:654] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:51.466446  1762 raft_consensus.cc:359] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.467504  1762 raft_consensus.cc:385] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:51.467931  1762 raft_consensus.cc:740] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:51.469000  1762 consensus_queue.cc:260] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.471598  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:51.473104  1641 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:51.487453  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:51.488595  1762 ts_tablet_manager.cc:1434] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
W20260709 13:00:51.504700  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 54577.9us
I20260709 13:00:51.487490  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:51.488852  1567 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:00:51.513785  1762 tablet_bootstrap.cc:492] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:51.519722  1641 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:51.523940  1607 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:51.524696  1813 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.525202  1813 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.528059  1979 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:51.529560  1813 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:51.529912  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:51.530586  1564 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:51.532707  1681 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:51.533914  1716 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:51.534135  2135 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:51.534590  2135 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.534987  2135 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:51.531247  2112 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:00:51.545187  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:51.558040  2075 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:00:51.568790  1682 leader_election.cc:343] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:51.581132  2115 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.581890  2115 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.581524  1467 catalog_manager.cc:5697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:51.616904  1763 tablet_bootstrap.cc:492] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:51.617565  1763 ts_tablet_manager.cc:1403] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.232s	user 0.037s	sys 0.033s
I20260709 13:00:51.631865  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:51.629889  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:51.639819  1761 tablet_bootstrap.cc:492] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:51.640334  1761 ts_tablet_manager.cc:1403] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.214s	user 0.014s	sys 0.005s
I20260709 13:00:51.616698  2115 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:51.648401  1533 leader_election.cc:343] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:51.648368  1763 raft_consensus.cc:359] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.649441  1763 raft_consensus.cc:385] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:51.649881  1763 raft_consensus.cc:740] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
W20260709 13:00:51.653492  1532 leader_election.cc:343] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:51.654183  1532 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:51.655310  2075 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:51.656157  1763 consensus_queue.cc:260] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.662106  1761 raft_consensus.cc:359] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.662880  1761 raft_consensus.cc:385] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:51.665256  1761 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:51.655340  1908 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.666905  1763 ts_tablet_manager.cc:1434] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.002s
I20260709 13:00:51.670980  1761 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.654469  2135 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.681970  1908 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.682469  1762 tablet_bootstrap.cc:654] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:51.689981  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:51.692588  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:51.693311  1641 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:51.689762  1908 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:51.690263  2135 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:00:51.694994  1682 leader_election.cc:343] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:51.696439  1682 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:51.683550  1761 ts_tablet_manager.cc:1434] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.004s
I20260709 13:00:51.697620  2034 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:51.698031  2034 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.698376  2034 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:51.695248  1763 tablet_bootstrap.cc:492] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:51.700381  1761 tablet_bootstrap.cc:492] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:51.709100  2034 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.713759  1762 tablet_bootstrap.cc:492] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:51.715976  2152 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.716627  1762 ts_tablet_manager.cc:1403] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.203s	user 0.011s	sys 0.006s
I20260709 13:00:51.716532  2152 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.719336  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:51.720167  1566 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:51.720993  1762 raft_consensus.cc:359] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.721946  1762 raft_consensus.cc:385] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:51.722312  1762 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:51.724493  1761 tablet_bootstrap.cc:654] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:51.724236  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:51.725001  1566 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:51.726320  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:51.727429  1763 tablet_bootstrap.cc:654] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:51.727355  1641 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:51.730147  1533 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:51.728011  1762 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.731045  2075 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:51.731478  2075 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.732014  2075 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:51.758376  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:51.758478  1762 ts_tablet_manager.cc:1434] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.041s	user 0.013s	sys 0.004s
I20260709 13:00:51.759186  1716 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:51.761394  1762 tablet_bootstrap.cc:492] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:51.763232  2152 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:51.764281  1761 tablet_bootstrap.cc:492] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:51.764772  1761 ts_tablet_manager.cc:1403] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.002s
I20260709 13:00:51.768141  1761 raft_consensus.cc:359] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.769017  1761 raft_consensus.cc:385] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:51.769356  1761 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:51.770377  2034 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:51.770054  1761 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.789588  1566 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:51.790516  1761 ts_tablet_manager.cc:1434] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.002s
I20260709 13:00:51.791899  1761 tablet_bootstrap.cc:492] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:51.772437  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:51.795197  1641 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:51.773725  2075 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.776976  1763 tablet_bootstrap.cc:492] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:51.802270  1763 ts_tablet_manager.cc:1403] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.107s	user 0.017s	sys 0.014s
I20260709 13:00:51.807770  2075 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:51.809839  1641 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:51.811475  1681 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:51.813076  1682 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:51.804061  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:51.804462  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:51.814208  2135 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:51.814651  1643 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:51.814597  1716 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:51.814879  2135 raft_consensus.cc:697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:51.816671  2135 consensus_queue.cc:237] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.820138  1761 tablet_bootstrap.cc:654] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:51.812851  2034 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:51.777766  1762 tablet_bootstrap.cc:654] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:51.836896  2034 raft_consensus.cc:697] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:51.830511  1763 raft_consensus.cc:359] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.844875  1763 raft_consensus.cc:385] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:51.845582  1763 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:51.838155  1716 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:00:51.846814  1761 tablet_bootstrap.cc:492] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:51.847618  1761 ts_tablet_manager.cc:1403] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.056s	user 0.013s	sys 0.002s
I20260709 13:00:51.848305  1763 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.850117  1532 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:51.842500  1643 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:00:51.845466  2034 consensus_queue.cc:237] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.851284  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:51.855473  1564 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:51.853397  2075 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:51.853556  1761 raft_consensus.cc:359] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.873159  1761 raft_consensus.cc:385] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:51.873482  1761 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:51.877836  1761 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.885154  2075 raft_consensus.cc:697] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:51.887444  1763 ts_tablet_manager.cc:1434] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.067s	user 0.009s	sys 0.000s
I20260709 13:00:51.892445  1564 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:51.893865  2075 consensus_queue.cc:237] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:51.899740  1761 ts_tablet_manager.cc:1434] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.001s
I20260709 13:00:51.905046  1813 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:51.908798  1467 catalog_manager.cc:5697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:51.912002  1763 tablet_bootstrap.cc:492] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:51.932351  1761 tablet_bootstrap.cc:492] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:51.954998  1762 tablet_bootstrap.cc:492] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:51.955672  1762 ts_tablet_manager.cc:1403] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.195s	user 0.019s	sys 0.016s
I20260709 13:00:51.964380  1813 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:51.958607  2119 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.979377  2119 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.044276  1761 tablet_bootstrap.cc:654] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:52.044471  1762 raft_consensus.cc:359] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.045480  1762 raft_consensus.cc:385] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:52.046049  1762 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:52.048101  1763 tablet_bootstrap.cc:654] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:52.053573  1762 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.056303  2034 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.063416  1762 ts_tablet_manager.cc:1434] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.107s	user 0.005s	sys 0.009s
I20260709 13:00:52.061820  2034 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.069633  2034 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:52.072253  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:52.074287  1682 leader_election.cc:343] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:52.076691  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:52.094995  1762 tablet_bootstrap.cc:492] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:52.080718  2075 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.079813  2119 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:52.107478  2034 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.113257  2034 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.119648  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:52.120739  1567 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:52.111454  2075 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.121553  1568 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:52.125267  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:52.126178  1712 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:52.127171  2034 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:52.130069  1532 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:52.169003  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:52.173369  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:52.170317  1569 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:52.130561  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:52.176734  1761 tablet_bootstrap.cc:492] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:52.174098  2152 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:52.209975  2152 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:52.210139  1533 leader_election.cc:343] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:52.210407  2152 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:52.213145  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:52.223348  1466 catalog_manager.cc:5697] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:52.226660  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:52.227465  1713 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:52.173967  1762 tablet_bootstrap.cc:654] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:52.128683  2075 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:52.171485  2116 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.240180  2116 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.244426  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:52.246086  1533 leader_election.cc:343] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:52.247869  2116 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:52.251560  1681 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:52.253554  1681 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:52.254974  1908 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:52.253557  2034 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:52.255362  1908 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.255746  2034 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.256397  1908 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:52.256551  2034 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:52.257308  1763 tablet_bootstrap.cc:492] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:52.257783  1763 ts_tablet_manager.cc:1403] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.346s	user 0.023s	sys 0.011s
I20260709 13:00:52.265754  2152 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.200984  1761 ts_tablet_manager.cc:1403] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.287s	user 0.021s	sys 0.025s
I20260709 13:00:52.168606  2074 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:52.203621  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 73150.8us
I20260709 13:00:52.204304  2170 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:52.280247  1467 catalog_manager.cc:5697] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 2, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:52.281278  1908 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.291640  1763 raft_consensus.cc:359] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.292783  1763 raft_consensus.cc:385] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:52.293264  1763 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:52.294893  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:52.295306  1763 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.282099  2170 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.275677  2074 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:52.303885  1532 leader_election.cc:343] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:52.312959  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:52.313402  1908 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:52.315426  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:52.316092  1641 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:52.282220  2160 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20260709 13:00:52.321969  2116 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:52.323838  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:52.325424  1568 raft_consensus.cc:2393] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 1: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:00:52.307478  2170 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:52.356549  2074 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:52.357635  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:52.358973  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:52.370522  1641 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
W20260709 13:00:52.372300  1533 leader_election.cc:343] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:52.374333  1682 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:00:52.339282  2034 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.377818  1763 ts_tablet_manager.cc:1434] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.120s	user 0.010s	sys 0.003s
I20260709 13:00:52.380097  2152 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:52.380496  2034 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:52.385527  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:52.386323  2160 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:52.386961  2160 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:52.384296  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:52.382874  1761 raft_consensus.cc:359] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.392009  1761 raft_consensus.cc:385] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:52.392462  1761 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:52.393564  1761 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.398825  1762 tablet_bootstrap.cc:492] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:52.389030  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:52.440232  1762 ts_tablet_manager.cc:1403] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.346s	user 0.021s	sys 0.025s
W20260709 13:00:52.445834  1682 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:52.448959  1763 tablet_bootstrap.cc:492] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
W20260709 13:00:52.452129  1532 leader_election.cc:343] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:52.452642  1532 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:52.454140  2116 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:52.454243  1532 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:52.456537  2116 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:52.466481  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:52.467150  1563 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:52.388130  2160 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.444006  2170 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.502460  1813 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:52.505317  1563 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:52.511495  1761 ts_tablet_manager.cc:1434] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.238s	user 0.008s	sys 0.002s
I20260709 13:00:52.513183  1681 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:52.514161  1813 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:52.521587  1638 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:00:52.522295  1813 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:52.523101  1813 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.527321  1643 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 1: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:00:52.542984  2127 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:52.535480  2075 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:00:52.570600  1762 raft_consensus.cc:359] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.571517  1762 raft_consensus.cc:385] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:52.571830  1762 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:52.572300  1763 tablet_bootstrap.cc:654] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:52.572944  2170 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:52.574179  1761 tablet_bootstrap.cc:492] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:52.575800  1762 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.585193  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:52.600811  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:52.601969  1762 ts_tablet_manager.cc:1434] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.161s	user 0.010s	sys 0.000s
I20260709 13:00:52.613392  1762 tablet_bootstrap.cc:492] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
W20260709 13:00:52.613672  1515 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:52.615749  1565 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:00:52.621270  1711 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:00:52.623448  2149 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:52.648011  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:52.648130  1467 catalog_manager.cc:5697] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:52.649705  1802 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:00:52.650647  1467 catalog_manager.cc:5697] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:52.706880  1762 tablet_bootstrap.cc:654] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:52.716030  1761 tablet_bootstrap.cc:654] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:52.744697  1762 tablet_bootstrap.cc:492] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:52.745203  1762 ts_tablet_manager.cc:1403] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.132s	user 0.011s	sys 0.002s
I20260709 13:00:52.748235  1763 tablet_bootstrap.cc:492] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:52.754364  1761 tablet_bootstrap.cc:492] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:52.760486  1761 ts_tablet_manager.cc:1403] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.231s	user 0.008s	sys 0.012s
I20260709 13:00:52.764462  1761 raft_consensus.cc:359] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.765312  1761 raft_consensus.cc:385] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:52.765606  1761 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:52.751231  1762 raft_consensus.cc:359] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.771260  1762 raft_consensus.cc:385] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:52.771785  1762 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:52.772658  1762 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.774348  1762 ts_tablet_manager.cc:1434] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:00:52.777112  1763 ts_tablet_manager.cc:1403] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.378s	user 0.035s	sys 0.018s
I20260709 13:00:52.780234  1761 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.782879  1762 tablet_bootstrap.cc:492] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:52.789072  1763 raft_consensus.cc:359] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.789784  1763 raft_consensus.cc:385] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:52.790091  1763 raft_consensus.cc:740] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:52.790747  1763 consensus_queue.cc:260] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.791203  2149 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.792055  1761 ts_tablet_manager.cc:1434] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 13:00:52.791756  2149 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.793210  1763 ts_tablet_manager.cc:1434] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:00:52.793543  1761 tablet_bootstrap.cc:492] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:52.794562  1763 tablet_bootstrap.cc:492] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:52.803650  2130 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.804283  1762 tablet_bootstrap.cc:654] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:52.804198  2130 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.805496  2149 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:52.805891  1763 tablet_bootstrap.cc:654] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:52.809139  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:52.809669  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:52.812426  1761 tablet_bootstrap.cc:654] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:52.813282  1682 leader_election.cc:343] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:52.813854  2130 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:00:52.814968  1532 leader_election.cc:343] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:52.819916  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:52.821460  1533 leader_election.cc:343] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:52.821903  1533 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:52.823602  2115 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:52.825773  2135 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.826318  2135 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.826565  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:52.829015  2135 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:52.831183  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:52.832158  1566 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:52.831741  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:52.833783  1682 leader_election.cc:343] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:52.830997  1565 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:52.825981  1761 tablet_bootstrap.cc:492] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:52.835306  1761 ts_tablet_manager.cc:1403] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.042s	user 0.017s	sys 0.001s
I20260709 13:00:52.838217  1761 raft_consensus.cc:359] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.838392  1681 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:52.838997  1761 raft_consensus.cc:385] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:52.839403  1761 raft_consensus.cc:740] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:52.840205  2135 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:52.840170  1761 consensus_queue.cc:260] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.840895  1763 tablet_bootstrap.cc:492] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:52.840965  2135 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.841449  1763 ts_tablet_manager.cc:1403] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.011s
I20260709 13:00:52.841567  2135 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:52.842392  1761 ts_tablet_manager.cc:1434] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:00:52.843706  1761 tablet_bootstrap.cc:492] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:52.844369  1763 raft_consensus.cc:359] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.845050  1763 raft_consensus.cc:385] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:52.845300  1763 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:52.846590  1763 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.852334  1763 ts_tablet_manager.cc:1434] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:00:52.858225  1761 tablet_bootstrap.cc:654] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:52.859069  1763 tablet_bootstrap.cc:492] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:52.864507  1681 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:52.865800  1813 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:52.866461  1813 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.868881  1813 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:52.869900  1763 tablet_bootstrap.cc:654] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:52.881927  1813 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.895032  2130 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:52.904187  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:52.906270  1563 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:52.925451  1763 tablet_bootstrap.cc:492] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:52.926496  1763 ts_tablet_manager.cc:1403] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.068s	user 0.022s	sys 0.005s
I20260709 13:00:52.927678  1813 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:52.929616  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:52.930256  1642 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:52.930619  2135 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.937178  1563 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:52.937492  1762 tablet_bootstrap.cc:492] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:52.938272  1762 ts_tablet_manager.cc:1403] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.156s	user 0.021s	sys 0.031s
I20260709 13:00:52.939087  2160 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:52.939859  1681 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:52.941145  2160 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:52.947824  2160 raft_consensus.cc:697] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:52.950898  2160 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.948990  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:52.949127  1761 tablet_bootstrap.cc:492] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:52.954864  1762 raft_consensus.cc:359] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.959333  1566 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:52.962438  1762 raft_consensus.cc:385] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:52.963006  1762 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:52.963742  1762 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.960189  1761 ts_tablet_manager.cc:1403] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.117s	user 0.011s	sys 0.026s
W20260709 13:00:52.979988  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 48914.5us
I20260709 13:00:52.955077  1643 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:52.976218  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:52.994961  1762 ts_tablet_manager.cc:1434] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.056s	user 0.010s	sys 0.000s
I20260709 13:00:52.996475  1763 raft_consensus.cc:359] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.997087  1763 raft_consensus.cc:385] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:52.997390  1763 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:52.955526  2135 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:53.004786  2063 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:53.005838  2130 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:00:53.004841  1682 leader_election.cc:343] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:53.007344  1763 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:52.984714  1566 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:53.010437  1763 ts_tablet_manager.cc:1434] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.002s
I20260709 13:00:53.011557  1763 tablet_bootstrap.cc:492] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:53.019556  1762 tablet_bootstrap.cc:492] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:53.032958  1681 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:53.036899  1813 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:53.037393  1813 raft_consensus.cc:697] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:53.038283  1813 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.064177  1761 raft_consensus.cc:359] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.064936  1761 raft_consensus.cc:385] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:53.065438  1761 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:53.066143  1761 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.066484  2115 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.067567  2115 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.068351  1761 ts_tablet_manager.cc:1434] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.083s	user 0.004s	sys 0.004s
I20260709 13:00:53.071868  2115 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:53.074470  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:53.074182  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:53.075881  1532 leader_election.cc:343] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:53.076689  1533 leader_election.cc:343] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:53.077172  1533 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:53.078275  2149 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:53.055714  2127 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.083398  2127 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.085872  2127 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:53.086356  1802 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:00:53.086935  1802 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.087301  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:53.088754  1567 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:53.097004  1762 tablet_bootstrap.cc:654] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:53.099355  1681 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:53.100955  2160 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:53.104249  1763 tablet_bootstrap.cc:654] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:53.101943  2160 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.115556  2160 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:53.128662  1802 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:53.129594  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:53.134893  2149 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.135551  2149 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.140300  2149 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:53.143224  1682 leader_election.cc:343] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:53.149672  2115 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.154148  2115 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.157693  2115 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:53.161139  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:53.161274  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:53.163194  1533 leader_election.cc:343] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:00:53.163489  1532 leader_election.cc:343] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:53.164803  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:00:53.167244  1532 leader_election.cc:343] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:53.167950  1532 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:53.169353  2075 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:53.172122  2160 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.174798  2160 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:53.170269  1761 tablet_bootstrap.cc:492] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Bootstrap starting.
I20260709 13:00:53.177219  1464 catalog_manager.cc:5697] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:53.203466  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:53.203706  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:00:53.208926  1533 leader_election.cc:343] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:53.209400  1533 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:00:53.210009  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 104276us
I20260709 13:00:53.212311  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:53.185910  1642 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:53.214187  2115 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:53.229712  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:53.228102  1802 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.230705  1802 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.216665  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:53.254119  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:53.255672  1564 raft_consensus.cc:2393] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:00:53.246296  1565 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:53.263140  1802 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:53.290655  2193 log.cc:927] Time spent T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:00:53.330428  1763 tablet_bootstrap.cc:492] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:53.331212  1763 ts_tablet_manager.cc:1403] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.320s	user 0.019s	sys 0.012s
I20260709 13:00:53.332067  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:00:53.337280  1682 leader_election.cc:343] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:53.345196  1761 tablet_bootstrap.cc:654] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:53.348938  1763 raft_consensus.cc:359] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.349802  1763 raft_consensus.cc:385] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:53.350193  1763 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:53.353269  1763 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.358134  1762 tablet_bootstrap.cc:492] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:53.360662  1762 ts_tablet_manager.cc:1403] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.351s	user 0.031s	sys 0.019s
I20260709 13:00:53.370718  1565 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:53.399344  1681 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:53.404340  2127 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:53.404901  2127 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:53.405762  2127 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.427449  2063 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:53.442667  1763 ts_tablet_manager.cc:1434] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.111s	user 0.006s	sys 0.003s
I20260709 13:00:53.472054  1763 tablet_bootstrap.cc:492] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
W20260709 13:00:53.488440  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 198043us
I20260709 13:00:53.493724  1762 raft_consensus.cc:359] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.497279  1762 raft_consensus.cc:385] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:53.500023  1762 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:53.503053  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:53.510367  1761 tablet_bootstrap.cc:492] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: No bootstrap required, opened a new log
I20260709 13:00:53.511858  2119 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:53.542701  1761 ts_tablet_manager.cc:1403] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Time spent bootstrapping tablet: real 0.373s	user 0.016s	sys 0.005s
I20260709 13:00:53.543046  1762 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.549058  1607 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:53.562476  1762 ts_tablet_manager.cc:1434] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.201s	user 0.009s	sys 0.004s
I20260709 13:00:53.564683  1761 raft_consensus.cc:359] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.572105  1761 raft_consensus.cc:385] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:53.572127  2164 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:53.572875  1761 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Initialized, Role: FOLLOWER
I20260709 13:00:53.573974  1761 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.576911  1763 tablet_bootstrap.cc:654] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:53.579118  1761 ts_tablet_manager.cc:1434] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.003s
I20260709 13:00:53.579429  2063 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.582818  2063 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.585906  2063 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:53.587064  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:53.588059  1566 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:53.589365  1681 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:00:53.592137  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:53.597267  2164 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:53.597903  2063 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:53.598299  2063 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.598699  2063 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:53.601171  1682 leader_election.cc:343] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:53.607192  1762 tablet_bootstrap.cc:492] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:53.610244  1763 tablet_bootstrap.cc:492] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:53.610750  1763 ts_tablet_manager.cc:1403] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.155s	user 0.010s	sys 0.008s
I20260709 13:00:53.629113  1763 raft_consensus.cc:359] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.630326  1763 raft_consensus.cc:385] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:53.630714  1763 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:53.631670  2135 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:53.630784  2127 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:00:53.631594  1763 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.635771  1763 ts_tablet_manager.cc:1434] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:00:53.643451  2066 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.644008  2066 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.647809  2066 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:53.651895  1762 tablet_bootstrap.cc:654] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:53.664942  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:53.665953  1566 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:00:53.652449  2063 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.657445  1763 tablet_bootstrap.cc:492] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:53.664633  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:53.679756  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:53.682190  1802 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:53.688921  2063 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:53.694262  1763 tablet_bootstrap.cc:654] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:53.695385  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:53.701112  1802 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:53.707386  1682 leader_election.cc:343] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:53.714114  1762 tablet_bootstrap.cc:492] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:53.714650  1762 ts_tablet_manager.cc:1403] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.123s	user 0.011s	sys 0.002s
I20260709 13:00:53.717365  1762 raft_consensus.cc:359] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.718274  1762 raft_consensus.cc:385] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:53.718587  1762 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:53.703778  1466 catalog_manager.cc:5697] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:53.720316  1466 catalog_manager.cc:5697] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:53.721760  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:53.722420  1565 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:53.723467  1763 tablet_bootstrap.cc:492] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:53.723946  1763 ts_tablet_manager.cc:1403] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.067s	user 0.014s	sys 0.000s
I20260709 13:00:53.727190  1763 raft_consensus.cc:359] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.727890  1763 raft_consensus.cc:385] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:53.728258  1763 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:53.734229  1762 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.745939  1565 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:53.746703  1763 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.746236  1762 ts_tablet_manager.cc:1434] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.000s
I20260709 13:00:53.748646  1762 tablet_bootstrap.cc:492] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:53.754022  2127 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:53.754498  1763 ts_tablet_manager.cc:1434] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:00:53.756026  1763 tablet_bootstrap.cc:492] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:53.767314  1681 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:53.769196  2127 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:53.769917  2127 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:53.770763  2127 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.767274  1762 tablet_bootstrap.cc:654] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
W20260709 13:00:53.780167  1515 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:00:53.784184  1763 tablet_bootstrap.cc:654] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:53.825357  1762 tablet_bootstrap.cc:492] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:53.825904  1762 ts_tablet_manager.cc:1403] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.003s
I20260709 13:00:53.825157  1466 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:53.829031  1762 raft_consensus.cc:359] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.830042  2127 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.830578  2127 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.832203  1763 tablet_bootstrap.cc:492] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:53.832665  1763 ts_tablet_manager.cc:1403] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.077s	user 0.022s	sys 0.006s
I20260709 13:00:53.834371  2127 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:53.837823  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:53.837903  1763 raft_consensus.cc:359] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.839042  1763 raft_consensus.cc:385] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:53.839260  1638 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:53.839352  1763 raft_consensus.cc:740] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:53.840570  1763 consensus_queue.cc:260] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.841368  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:53.829697  1762 raft_consensus.cc:385] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:53.842155  2127 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:53.842384  1762 raft_consensus.cc:740] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:53.842856  2127 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.843580  2127 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:53.844082  1762 consensus_queue.cc:260] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.847291  1762 ts_tablet_manager.cc:1434] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.001s
I20260709 13:00:53.848505  1762 tablet_bootstrap.cc:492] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:53.851279  1763 ts_tablet_manager.cc:1434] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:00:53.854650  1763 tablet_bootstrap.cc:492] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:53.855484  1762 tablet_bootstrap.cc:654] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:53.862103  1763 tablet_bootstrap.cc:654] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:53.867815  2127 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.873116  1762 tablet_bootstrap.cc:492] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:53.873657  1762 ts_tablet_manager.cc:1403] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.006s
I20260709 13:00:53.876391  1762 raft_consensus.cc:359] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.877779  2127 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:53.878163  1762 raft_consensus.cc:385] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:53.878520  1762 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:53.879298  1762 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.880291  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:53.881233  1640 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:53.882838  1762 ts_tablet_manager.cc:1434] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:00:53.884881  1763 tablet_bootstrap.cc:492] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:53.884264  1762 tablet_bootstrap.cc:492] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Bootstrap starting.
I20260709 13:00:53.885284  1802 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.885639  1763 ts_tablet_manager.cc:1403] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.006s
I20260709 13:00:53.885731  1802 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.888960  1802 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:53.889442  1763 raft_consensus.cc:359] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.892297  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:53.907626  1763 raft_consensus.cc:385] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:53.894778  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:53.908032  1714 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:53.908046  1763 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:53.908413  1568 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:53.896170  1762 tablet_bootstrap.cc:654] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:53.896167  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:53.909107  1763 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.909642  1566 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:53.911553  1763 ts_tablet_manager.cc:1434] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.001s
I20260709 13:00:53.913476  1763 tablet_bootstrap.cc:492] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6: Bootstrap starting.
I20260709 13:00:53.919917  1763 tablet_bootstrap.cc:654] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6: Neither blocks nor log segments found. Creating new log.
I20260709 13:00:53.921690  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:53.924207  2066 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:53.924862  2066 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.925148  1566 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:53.936123  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:53.937063  1568 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:53.925686  2066 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:53.899892  1640 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:53.939182  2108 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:53.933403  1763 tablet_bootstrap.cc:492] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6: No bootstrap required, opened a new log
I20260709 13:00:53.941227  1763 ts_tablet_manager.cc:1403] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.005s
I20260709 13:00:53.944031  1763 raft_consensus.cc:359] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.944856  1763 raft_consensus.cc:385] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:53.945163  1763 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Initialized, Role: FOLLOWER
I20260709 13:00:53.945818  1763 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.947990  1763 ts_tablet_manager.cc:1434] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:00:53.950623  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:53.953600  2135 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:53.954779  2135 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:53.955859  2135 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.961936  2202 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.962607  2202 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:53.968676  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:53.970964  2202 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:00:53.971680  1532 leader_election.cc:343] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:53.975726  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:53.976698  1638 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 0.
I20260709 13:00:53.978717  1533 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:53.979619  2203 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:53.980073  2203 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.980742  2203 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:54.000700  2203 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:54.016649  1813 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:54.025811  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:54.026652  1639 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:54.031327  2066 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:54.047314  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:54.048308  1564 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:54.074256  2203 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:54.075042  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:00:54.076606  1532 leader_election.cc:343] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:00:54.111713  1564 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:54.100256  2160 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.119944  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:54.127900  1802 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:54.128862  1802 raft_consensus.cc:697] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:00:54.117316  2066 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:54.130530  1802 consensus_queue.cc:237] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:54.127195  2160 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:54.207732  1762 tablet_bootstrap.cc:492] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: No bootstrap required, opened a new log
I20260709 13:00:54.208331  1762 ts_tablet_manager.cc:1403] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent bootstrapping tablet: real 0.324s	user 0.032s	sys 0.089s
I20260709 13:00:54.233278  1762 raft_consensus.cc:359] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:54.234265  1762 raft_consensus.cc:385] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:00:54.234613  1762 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Initialized, Role: FOLLOWER
I20260709 13:00:54.236625  1762 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:54.242707  1762 ts_tablet_manager.cc:1434] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent starting tablet: real 0.034s	user 0.003s	sys 0.004s
I20260709 13:00:54.249308  2160 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:54.231568  2196 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.254907  2196 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:54.257858  2196 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:54.264366  1466 catalog_manager.cc:5697] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.268472  1639 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:00:54.280521  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:00:54.281581  1712 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:00:54.339318  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 46391.6us
I20260709 13:00:54.342962  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:54.354995  1565 raft_consensus.cc:2393] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 1: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:00:54.357170  1533 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:00:54.363435  2227 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:54.363612  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:54.363981  2227 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:00:54.364866  2227 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:54.367184  1712 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:00:54.374774  1464 catalog_manager.cc:5697] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:54.442602  2229 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.443385  2229 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:00:54.469079  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 307503us
W20260709 13:00:54.474066  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 104719us
I20260709 13:00:54.481477  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:54.483137  1642 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:54.486747  2229 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:54.496980  1640 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:54.505987  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:54.506785  1565 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 0.
I20260709 13:00:54.519901  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:00:54.520709  2164 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:00:54.523334  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 149786us
I20260709 13:00:54.526485  1682 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:54.527315  2226 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:00:54.527702  2226 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.528056  2226 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:54.528024  2127 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:54.541600  2226 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:54.551908  2226 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:54.557192  2034 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:54.558358  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:00:54.559041  1640 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:54.567545  1640 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:54.569092  1682 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:54.569736  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:54.569123  1464 catalog_manager.cc:5697] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:54.570394  1565 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 0.
I20260709 13:00:54.571880  2228 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:54.578338  2226 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:00:54.578856  2226 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:00:54.579691  2226 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:54.587018  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:00:54.596009  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:00:54.596685  1565 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:54.611326  1565 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:00:54.630522  1464 catalog_manager.cc:5697] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.642691  1607 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:00:54.644821  2232 raft_consensus.cc:2764] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:00:54.666245  2232 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:54.691076  2226 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:54.692142  2196 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:54.805769  2230 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.806344  2230 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:00:54.808661  2230 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:54.814677  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:00:54.822527  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:00:54.827782  1642 raft_consensus.cc:2393] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 1: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:00:54.843209  1682 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:00:54.844746  2230 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:00:54.863186  2230 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:00:54.878856  2202 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:54.902833  2238 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:00:55.000422  2226 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:00:55.013870  2127 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:00:55.419129   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:00:55.465112  2259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:55.465386  2260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:00:55.466610  2262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:00:55.476953   431 server_base.cc:1061] running on GCE node
I20260709 13:00:55.479679   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:00:55.480021   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:00:55.480252   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602055480234 us; error 0 us; skew 500 ppm
I20260709 13:00:55.480888   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:00:55.484953   431 webserver.cc:533] Webserver started at http://127.0.107.196:38143/ using document root <none> and password file <none>
I20260709 13:00:55.486202   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:00:55.486644   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:00:55.487303   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:00:55.493289   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-3-root/instance:
uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
format_stamp: "Formatted at 2026-07-09 13:00:55 on dist-test-slave-cq4t"
I20260709 13:00:55.544667   431 fs_manager.cc:696] Time spent creating directory manager: real 0.050s	user 0.013s	sys 0.005s
I20260709 13:00:55.551597  2267 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:55.552944   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:00:55.553584   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-3-root
uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
format_stamp: "Formatted at 2026-07-09 13:00:55 on dist-test-slave-cq4t"
I20260709 13:00:55.554162   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:00:55.579838   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:00:55.582878   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:00:55.598886   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:00:55.603297   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:00:55.603643   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:55.604015   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:00:55.604278   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:00:55.760412   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.196:38715
I20260709 13:00:55.760567  2329 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.196:38715 every 8 connection(s)
I20260709 13:00:55.766001   431 auto_leader_rebalancer-test.cc:384] add a tserver: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:00:55.795358  2330 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37201
I20260709 13:00:55.796002  2330 heartbeater.cc:461] Registering TS with master...
I20260709 13:00:55.797150  2330 heartbeater.cc:507] Master 127.0.107.254:37201 requested a full tablet report, sending...
I20260709 13:00:55.800199  1465 ts_manager.cc:194] Registered new tserver with Master: 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:00:55.802412  1465 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60118
I20260709 13:00:56.048200  1643 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:56.059393  1714 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:56.065433  1566 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:56.065490  2239 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:56.067566  2196 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:56.091674  1609 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2792e3e29993414a8d6f55078ae13994. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:56.100843  2239 raft_consensus.cc:2955] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:56.104753  1709 raft_consensus.cc:2955] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:56.109753  1566 raft_consensus.cc:2955] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:56.138602  1465 catalog_manager.cc:5697] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:56.169813  1710 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:56.179978  1643 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:56.184638  2063 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:56.196074  1567 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:56.203116  2063 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:56.219099  2342 raft_consensus.cc:2955] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
W20260709 13:00:56.243800  1683 consensus_peers.cc:597] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6dc6e12b7924255b756439991155e88. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:56.262730  1642 raft_consensus.cc:2955] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:56.268292  1464 catalog_manager.cc:5697] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:56.277998  1566 raft_consensus.cc:2955] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:56.304224  1565 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:56.325137  1710 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:56.329810  2202 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:56.343477  1642 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:56.345093  2357 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:56.348778  2201 raft_consensus.cc:2955] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
W20260709 13:00:56.351720  1534 consensus_peers.cc:597] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3fe2946ff23d498f98c4627ee968a4e3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:56.364322  1710 raft_consensus.cc:2955] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:56.369160  1642 raft_consensus.cc:2955] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:56.372934  1714 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:56.384622  1464 catalog_manager.cc:5697] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
W20260709 13:00:56.401096  1683 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7808a4bebf84e598198ed51ea156981. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:56.403652  1642 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:56.404182  1566 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:56.406049  2230 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:56.409471  2342 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:56.424479  2230 raft_consensus.cc:2955] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:56.427100  1566 raft_consensus.cc:2955] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:56.428434  1642 raft_consensus.cc:2955] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:56.444124  1465 catalog_manager.cc:5697] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:56.585570  2365 ts_tablet_manager.cc:933] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:00:56.588081  2365 tablet_copy_client.cc:323] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:40023
I20260709 13:00:56.618733  1654 tablet_copy_service.cc:140] P 5184c101be204393adf3ac8b31f25ad6: Received BeginTabletCopySession request for tablet 2792e3e29993414a8d6f55078ae13994 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:53222)
I20260709 13:00:56.619413  1654 tablet_copy_service.cc:161] P 5184c101be204393adf3ac8b31f25ad6: Beginning new tablet copy session on tablet 2792e3e29993414a8d6f55078ae13994 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:53222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-2792e3e29993414a8d6f55078ae13994
I20260709 13:00:56.630026  1654 tablet_copy_source_session.cc:215] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:56.635867  2365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2792e3e29993414a8d6f55078ae13994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:56.656018  2369 ts_tablet_manager.cc:933] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:56.660712  2365 tablet_copy_client.cc:806] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:56.661639  2365 tablet_copy_client.cc:670] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:56.665315  2369 tablet_copy_client.cc:323] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:00:56.667322  2365 tablet_copy_client.cc:538] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:56.677631  2365 tablet_bootstrap.cc:492] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:00:56.686305  1728 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet e6dc6e12b7924255b756439991155e88 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:00:56.687053  1728 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet e6dc6e12b7924255b756439991155e88 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-e6dc6e12b7924255b756439991155e88
I20260709 13:00:56.703187  1728 tablet_copy_source_session.cc:215] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:56.709437  2365 tablet_bootstrap.cc:492] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:56.711130  2365 tablet_bootstrap.cc:492] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:00:56.712172  2365 ts_tablet_manager.cc:1403] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.035s	user 0.034s	sys 0.001s
I20260709 13:00:56.716984  2369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6dc6e12b7924255b756439991155e88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:56.717244  2365 raft_consensus.cc:359] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:56.718277  2365 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:00:56.719074  2365 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:56.722100  2330 heartbeater.cc:499] Master 127.0.107.254:37201 was elected leader, sending a full tablet report...
I20260709 13:00:56.722188  2365 ts_tablet_manager.cc:1434] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:00:56.728042  1654 tablet_copy_service.cc:342] P 5184c101be204393adf3ac8b31f25ad6: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-2792e3e29993414a8d6f55078ae13994 received from {username='slave'} at 127.0.0.1:53222
I20260709 13:00:56.728617  1654 tablet_copy_service.cc:434] P 5184c101be204393adf3ac8b31f25ad6: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-2792e3e29993414a8d6f55078ae13994 on tablet 2792e3e29993414a8d6f55078ae13994 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:00:56.746877  2369 tablet_copy_client.cc:806] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:56.747759  2369 tablet_copy_client.cc:670] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:56.753225  2369 tablet_copy_client.cc:538] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:56.763232  2369 tablet_bootstrap.cc:492] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:00:56.803344  2369 tablet_bootstrap.cc:492] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:56.804391  2369 tablet_bootstrap.cc:492] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:00:56.805377  2369 ts_tablet_manager.cc:1403] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.043s	user 0.028s	sys 0.016s
I20260709 13:00:56.808446  2369 raft_consensus.cc:359] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:56.809437  2369 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:00:56.810406  2369 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:56.812942  2369 ts_tablet_manager.cc:1434] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:00:56.821074  1728 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-e6dc6e12b7924255b756439991155e88 received from {username='slave'} at 127.0.0.1:33280
I20260709 13:00:56.821597  1728 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-e6dc6e12b7924255b756439991155e88 on tablet e6dc6e12b7924255b756439991155e88 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:00:56.893680  2369 ts_tablet_manager.cc:933] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
I20260709 13:00:56.895640  2369 tablet_copy_client.cc:323] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:36547
I20260709 13:00:56.900692  2365 ts_tablet_manager.cc:933] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:00:56.903831  2365 tablet_copy_client.cc:323] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:00:56.905807  1728 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet b7808a4bebf84e598198ed51ea156981 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:00:56.906486  1728 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet b7808a4bebf84e598198ed51ea156981 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-b7808a4bebf84e598198ed51ea156981
I20260709 13:00:56.913601  1579 tablet_copy_service.cc:140] P 3f64b76180a54492bb015e405caeacec: Received BeginTabletCopySession request for tablet 3fe2946ff23d498f98c4627ee968a4e3 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:35222)
I20260709 13:00:56.914371  1579 tablet_copy_service.cc:161] P 3f64b76180a54492bb015e405caeacec: Beginning new tablet copy session on tablet 3fe2946ff23d498f98c4627ee968a4e3 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:35222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-3fe2946ff23d498f98c4627ee968a4e3
I20260709 13:00:56.922354  1728 tablet_copy_source_session.cc:215] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:56.923700  1579 tablet_copy_source_session.cc:215] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:56.926023  2365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7808a4bebf84e598198ed51ea156981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:56.933296  2369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe2946ff23d498f98c4627ee968a4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:56.952767  2365 tablet_copy_client.cc:806] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:56.953575  2365 tablet_copy_client.cc:670] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:56.963089  2365 tablet_copy_client.cc:538] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:56.975214  2369 tablet_copy_client.cc:806] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:56.975956  2369 tablet_copy_client.cc:670] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:56.976971  2365 tablet_bootstrap.cc:492] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:00:56.985301  2369 tablet_copy_client.cc:538] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:56.993244  2369 tablet_bootstrap.cc:492] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:00:57.004499  2365 tablet_bootstrap.cc:492] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:57.006090  2365 tablet_bootstrap.cc:492] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:00:57.008755  2365 ts_tablet_manager.cc:1403] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.032s	user 0.020s	sys 0.005s
I20260709 13:00:57.019599  2365 raft_consensus.cc:359] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:57.020483  2365 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:00:57.021296  2365 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:57.024032  2365 ts_tablet_manager.cc:1434] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:00:57.025713  1728 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-b7808a4bebf84e598198ed51ea156981 received from {username='slave'} at 127.0.0.1:33280
I20260709 13:00:57.026145  1728 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-b7808a4bebf84e598198ed51ea156981 on tablet b7808a4bebf84e598198ed51ea156981 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:00:57.032097  2369 tablet_bootstrap.cc:492] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:57.033181  2369 tablet_bootstrap.cc:492] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:00:57.035660  2369 ts_tablet_manager.cc:1403] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.041s	user 0.023s	sys 0.005s
I20260709 13:00:57.038740  2369 raft_consensus.cc:359] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:57.039625  2369 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:00:57.040253  2369 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:57.044629  2369 ts_tablet_manager.cc:1434] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:00:57.046875  1579 tablet_copy_service.cc:342] P 3f64b76180a54492bb015e405caeacec: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-3fe2946ff23d498f98c4627ee968a4e3 received from {username='slave'} at 127.0.0.1:35222
I20260709 13:00:57.047370  1579 tablet_copy_service.cc:434] P 3f64b76180a54492bb015e405caeacec: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-3fe2946ff23d498f98c4627ee968a4e3 on tablet 3fe2946ff23d498f98c4627ee968a4e3 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:00:57.052331  2305 raft_consensus.cc:1217] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:57.159529  2305 raft_consensus.cc:1217] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:57.439957  2305 raft_consensus.cc:1217] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:57.457572  2305 raft_consensus.cc:1217] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:57.574323  2364 raft_consensus.cc:1064] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:00:57.580866  2364 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:00:57.607167  2305 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:57.607828  1567 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:57.609830  2337 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:57.611276  1714 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:57.617833  2337 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:57.618525  2351 raft_consensus.cc:1064] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:00:57.621694  2351 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:00:57.637876  2382 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:57.666898  1644 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:57.657020  2382 raft_consensus.cc:2955] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.679423  1567 raft_consensus.cc:2955] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.692896  2352 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:00:57.698510  2305 raft_consensus.cc:2955] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.726014  2304 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:57.729235  1466 catalog_manager.cc:5697] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:57.731851  1566 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:57.741091  2387 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:57.762823  1710 raft_consensus.cc:2955] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.782241  2389 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:57.792815  2388 raft_consensus.cc:2955] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.796615  1644 raft_consensus.cc:2955] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.847996  2305 raft_consensus.cc:2955] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.867192  1644 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:00:57.871714  1566 raft_consensus.cc:2955] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.879097  1464 catalog_manager.cc:5697] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.900168  1565 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:57.902192  2364 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:57.915798  2305 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:57.918275  2382 raft_consensus.cc:2955] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.918895  2196 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:57.927099  2305 raft_consensus.cc:2955] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.928548  1565 raft_consensus.cc:2955] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.948484  1467 catalog_manager.cc:5697] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.935464  1710 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:00:57.960233  2305 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:57.962390  2230 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:57.964550  1699 tablet_service.cc:1558] Processing DeleteTablet for tablet 2792e3e29993414a8d6f55078ae13994 with delete_type TABLET_DATA_TOMBSTONED (TS 28bc59d37f664ce0a6f5784f2619f0b5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36054
I20260709 13:00:57.964980  2359 raft_consensus.cc:1064] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:00:57.967751  2359 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:00:57.974794  2388 raft_consensus.cc:2955] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.979606  1644 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:57.981328  1452 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2792e3e29993414a8d6f55078ae13994 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:57.982688  2398 tablet_replica.cc:333] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:00:57.984599  2398 raft_consensus.cc:2243] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:57.985842  2398 raft_consensus.cc:2272] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:57.987680  2387 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:57.993299  1644 raft_consensus.cc:2955] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:57.996712  1640 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:57.998668  2398 ts_tablet_manager.cc:1916] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:57.999013  1710 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:57.998756  2304 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:58.001593  2380 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:00:58.008200  2380 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:58.012918  2357 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:58.017239  2387 raft_consensus.cc:1064] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:00:58.020000  2304 raft_consensus.cc:2955] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.020062  2387 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:00:58.037714  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e6dc6e12b7924255b756439991155e88 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:58.056114  1467 catalog_manager.cc:5697] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.069141  2303 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:58.071859  1640 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:58.074393  1565 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:58.074448  2387 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:58.076277  2230 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:58.078711  2351 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:58.084214  2357 raft_consensus.cc:2955] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.087328  2305 raft_consensus.cc:2955] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.095665  2398 ts_tablet_manager.cc:1929] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:58.096257  2398 log.cc:1199] T 2792e3e29993414a8d6f55078ae13994 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/wals/2792e3e29993414a8d6f55078ae13994
I20260709 13:00:58.098546  1451 catalog_manager.cc:5028] TS 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): tablet 2792e3e29993414a8d6f55078ae13994 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:00:58.104954  1710 raft_consensus.cc:2955] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.121464  2388 raft_consensus.cc:2955] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.126091  1554 tablet_service.cc:1558] Processing DeleteTablet for tablet e6dc6e12b7924255b756439991155e88 with delete_type TABLET_DATA_TOMBSTONED (TS 3f64b76180a54492bb015e405caeacec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41262
I20260709 13:00:58.130892  1644 raft_consensus.cc:2955] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.133731  1565 raft_consensus.cc:2955] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.155787  2408 tablet_replica.cc:333] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:00:58.166217  1466 catalog_manager.cc:5697] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.167538  2408 raft_consensus.cc:2243] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.168421  2408 raft_consensus.cc:2272] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.173213  2408 ts_tablet_manager.cc:1916] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:58.187206  2304 raft_consensus.cc:2955] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.209728  1640 raft_consensus.cc:2955] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.224936  2408 ts_tablet_manager.cc:1929] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:58.225503  2408 log.cc:1199] T e6dc6e12b7924255b756439991155e88 P 3f64b76180a54492bb015e405caeacec: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/wals/e6dc6e12b7924255b756439991155e88
I20260709 13:00:58.224709  1466 catalog_manager.cc:5697] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:58.232342  1566 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:00:58.248662  1451 catalog_manager.cc:5028] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): tablet e6dc6e12b7924255b756439991155e88 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:00:58.253541  1709 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:00:58.256759  2305 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:58.259310  2380 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:58.269097  2357 raft_consensus.cc:2955] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.271193  2303 raft_consensus.cc:2955] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.320425  1714 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:58.326676  2359 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:58.330336  1565 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:58.342157  2305 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:58.347012  1710 raft_consensus.cc:2955] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.362720  1466 catalog_manager.cc:5697] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.366698  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3fe2946ff23d498f98c4627ee968a4e3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:58.368531  2387 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:58.374402  2390 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:58.376314  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 3fe2946ff23d498f98c4627ee968a4e3 with delete_type TABLET_DATA_TOMBSTONED (TS 5184c101be204393adf3ac8b31f25ad6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60370
I20260709 13:00:58.391757  2390 raft_consensus.cc:2955] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.395701  2305 raft_consensus.cc:2955] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.417632  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b7808a4bebf84e598198ed51ea156981 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:58.419075  1464 catalog_manager.cc:5697] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.430583  1567 raft_consensus.cc:2955] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:00:58.438261  2410 tablet_replica.cc:333] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:00:58.443265  2410 raft_consensus.cc:2243] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.446277  2410 raft_consensus.cc:2272] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.455506  2410 ts_tablet_manager.cc:1916] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:58.469101  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet b7808a4bebf84e598198ed51ea156981 with delete_type TABLET_DATA_TOMBSTONED (TS 5184c101be204393adf3ac8b31f25ad6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60370
I20260709 13:00:58.486383  2410 ts_tablet_manager.cc:1929] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:58.487020  2410 log.cc:1199] T 3fe2946ff23d498f98c4627ee968a4e3 P 5184c101be204393adf3ac8b31f25ad6: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/wals/3fe2946ff23d498f98c4627ee968a4e3
I20260709 13:00:58.490054  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet 3fe2946ff23d498f98c4627ee968a4e3 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:00:58.490108  2410 tablet_replica.cc:333] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:00:58.491432  2410 raft_consensus.cc:2243] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:58.492754  2410 raft_consensus.cc:2272] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:58.496592  2410 ts_tablet_manager.cc:1916] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:58.515369  2410 ts_tablet_manager.cc:1929] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:58.515828  2410 log.cc:1199] T b7808a4bebf84e598198ed51ea156981 P 5184c101be204393adf3ac8b31f25ad6: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/wals/b7808a4bebf84e598198ed51ea156981
I20260709 13:00:58.517771  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet b7808a4bebf84e598198ed51ea156981 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:00:59.537875  1643 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:59.548743  1710 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:59.550773  2412 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:59.554312  1609 consensus_peers.cc:597] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d1256ff04335423286d7c081ec776061. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:59.558141  1567 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:59.560179  2364 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:59.563728  2382 raft_consensus.cc:2955] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:59.565418  1710 raft_consensus.cc:2955] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:59.569687  1567 raft_consensus.cc:2955] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:59.582276  1710 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:59.586454  1464 catalog_manager.cc:5697] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:59.596127  1643 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:59.596866  1566 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:59.598899  2387 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:59.599954  1683 consensus_peers.cc:597] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fbfadedb9a5e4ca0abc25384a524c546. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:59.600523  2390 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:59.620453  2387 raft_consensus.cc:2955] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:59.630002  1566 raft_consensus.cc:2955] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:59.667820  1466 catalog_manager.cc:5697] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
W20260709 13:00:59.673695  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 48398.8us
I20260709 13:00:59.676362  1643 raft_consensus.cc:2955] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:59.700788  1710 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:00:59.713362  1683 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6bdefe70d276420ca0283414ab16fab9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:59.715693  1643 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:59.716954  1566 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:59.721609  2390 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:59.723923  2423 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:59.738751  2423 raft_consensus.cc:2955] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:59.742441  1567 raft_consensus.cc:2955] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:59.743453  1643 raft_consensus.cc:2955] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:59.760725  1466 catalog_manager.cc:5697] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:59.756987  1710 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:00:59.774991  1642 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:59.775408  1683 consensus_peers.cc:597] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9df7dc71f943412e83fbc8d4f788c9f2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:59.777384  1566 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:59.777609  2390 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:59.785617  2390 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:59.811000  2423 raft_consensus.cc:2955] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:59.815399  1643 raft_consensus.cc:2955] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:59.817674  1567 raft_consensus.cc:2955] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:00:59.837200  1466 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:00.089269  2432 ts_tablet_manager.cc:933] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:00.091444  2432 tablet_copy_client.cc:323] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:00.095870  1728 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet fbfadedb9a5e4ca0abc25384a524c546 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:00.096591  1728 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet fbfadedb9a5e4ca0abc25384a524c546 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-fbfadedb9a5e4ca0abc25384a524c546
I20260709 13:01:00.102876  1728 tablet_copy_source_session.cc:215] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:00.106388  2432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbfadedb9a5e4ca0abc25384a524c546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:00.113090  2434 ts_tablet_manager.cc:933] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:00.119685  2434 tablet_copy_client.cc:323] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:40023
I20260709 13:01:00.122479  1654 tablet_copy_service.cc:140] P 5184c101be204393adf3ac8b31f25ad6: Received BeginTabletCopySession request for tablet d1256ff04335423286d7c081ec776061 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:53222)
I20260709 13:01:00.123175  1654 tablet_copy_service.cc:161] P 5184c101be204393adf3ac8b31f25ad6: Beginning new tablet copy session on tablet d1256ff04335423286d7c081ec776061 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:53222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-d1256ff04335423286d7c081ec776061
I20260709 13:01:00.129813  1654 tablet_copy_source_session.cc:215] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:00.134538  2434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1256ff04335423286d7c081ec776061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:00.135730  2432 tablet_copy_client.cc:806] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:00.136602  2432 tablet_copy_client.cc:670] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:00.141829  2432 tablet_copy_client.cc:538] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:00.156883  2434 tablet_copy_client.cc:806] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:00.157871  2434 tablet_copy_client.cc:670] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:00.161036  2432 tablet_bootstrap.cc:492] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:00.162587  2434 tablet_copy_client.cc:538] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:00.175177  2434 tablet_bootstrap.cc:492] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:00.187273  2432 tablet_bootstrap.cc:492] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:00.188335  2432 tablet_bootstrap.cc:492] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:00.189483  2432 ts_tablet_manager.cc:1403] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.029s	user 0.021s	sys 0.006s
I20260709 13:01:00.194427  2432 raft_consensus.cc:359] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:00.196254  2432 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:00.197289  2432 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:00.207710  2434 tablet_bootstrap.cc:492] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:00.209287  2434 tablet_bootstrap.cc:492] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:00.210134  2434 ts_tablet_manager.cc:1403] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.035s	user 0.021s	sys 0.013s
I20260709 13:01:00.212464  2432 ts_tablet_manager.cc:1434] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.007s
I20260709 13:01:00.213807  2434 raft_consensus.cc:359] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:00.214947  2434 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:00.215682  2434 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:00.229063  1728 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-fbfadedb9a5e4ca0abc25384a524c546 received from {username='slave'} at 127.0.0.1:33280
I20260709 13:01:00.229561  1728 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-fbfadedb9a5e4ca0abc25384a524c546 on tablet fbfadedb9a5e4ca0abc25384a524c546 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:00.241587  2434 ts_tablet_manager.cc:1434] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.031s	user 0.017s	sys 0.007s
I20260709 13:01:00.244949  1654 tablet_copy_service.cc:342] P 5184c101be204393adf3ac8b31f25ad6: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-d1256ff04335423286d7c081ec776061 received from {username='slave'} at 127.0.0.1:53222
I20260709 13:01:00.245410  1654 tablet_copy_service.cc:434] P 5184c101be204393adf3ac8b31f25ad6: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-d1256ff04335423286d7c081ec776061 on tablet d1256ff04335423286d7c081ec776061 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:00.252619  2432 ts_tablet_manager.cc:933] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:00.266147  2432 tablet_copy_client.cc:323] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:00.267989  1728 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet 9df7dc71f943412e83fbc8d4f788c9f2 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:00.268558  1728 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet 9df7dc71f943412e83fbc8d4f788c9f2 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-9df7dc71f943412e83fbc8d4f788c9f2
I20260709 13:01:00.276664  1728 tablet_copy_source_session.cc:215] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:00.279808  2432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9df7dc71f943412e83fbc8d4f788c9f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:00.298732  2432 tablet_copy_client.cc:806] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:00.299525  2432 tablet_copy_client.cc:670] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:00.306398  2432 tablet_copy_client.cc:538] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:00.312533  2434 ts_tablet_manager.cc:933] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:00.314364  2434 tablet_copy_client.cc:323] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:00.316254  1728 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet 6bdefe70d276420ca0283414ab16fab9 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:00.316558  2432 tablet_bootstrap.cc:492] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:00.316906  1728 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet 6bdefe70d276420ca0283414ab16fab9 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-6bdefe70d276420ca0283414ab16fab9
I20260709 13:01:00.326211  1728 tablet_copy_source_session.cc:215] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:00.329635  2434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bdefe70d276420ca0283414ab16fab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:00.350189  2432 tablet_bootstrap.cc:492] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:00.351394  2432 tablet_bootstrap.cc:492] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:00.352296  2432 ts_tablet_manager.cc:1403] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.036s	user 0.024s	sys 0.009s
I20260709 13:01:00.355830  2432 raft_consensus.cc:359] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:00.356874  2432 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:00.357786  2432 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:00.369408  2432 ts_tablet_manager.cc:1434] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.003s
I20260709 13:01:00.370566  2434 tablet_copy_client.cc:806] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:00.373553  1728 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-9df7dc71f943412e83fbc8d4f788c9f2 received from {username='slave'} at 127.0.0.1:33280
I20260709 13:01:00.379341  1728 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-9df7dc71f943412e83fbc8d4f788c9f2 on tablet 9df7dc71f943412e83fbc8d4f788c9f2 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:00.381018  2434 tablet_copy_client.cc:670] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:00.387465  2434 tablet_copy_client.cc:538] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:00.397832  2434 tablet_bootstrap.cc:492] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:00.420233  2434 tablet_bootstrap.cc:492] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:00.421249  2434 tablet_bootstrap.cc:492] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:00.421983  2434 ts_tablet_manager.cc:1403] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.024s	user 0.019s	sys 0.005s
I20260709 13:01:00.427050  2434 raft_consensus.cc:359] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:00.428000  2434 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:00.428706  2434 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:00.431274  2434 ts_tablet_manager.cc:1434] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:01:00.433209  1728 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-6bdefe70d276420ca0283414ab16fab9 received from {username='slave'} at 127.0.0.1:33280
I20260709 13:01:00.433691  1728 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-6bdefe70d276420ca0283414ab16fab9 on tablet 6bdefe70d276420ca0283414ab16fab9 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:00.492671  2305 raft_consensus.cc:1217] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:00.505975  2305 raft_consensus.cc:1217] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:00.731465  2305 raft_consensus.cc:1217] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:00.765028  2305 raft_consensus.cc:1217] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:00.974031  2424 raft_consensus.cc:1064] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:00.976913  2424 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:00.987273  1642 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:00.989496  1567 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:00.990341  2305 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:00.990590  2423 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:00.992079  2424 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:00.993249  2423 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:01.010375  1567 raft_consensus.cc:2955] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.010881  2424 raft_consensus.cc:2955] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.012574  2305 raft_consensus.cc:2955] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.025684  1642 raft_consensus.cc:2955] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.040028  2382 raft_consensus.cc:1064] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:01.043303  2382 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:01.042093  1467 catalog_manager.cc:5697] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:01.067507  1709 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:01.069831  2412 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:01.081912  2305 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:01.083489  2382 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:01.089989  1567 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:01.097939  2412 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:01.120468  2382 raft_consensus.cc:2955] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.125954  2305 raft_consensus.cc:2955] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.130662  1565 raft_consensus.cc:2955] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.151265  1709 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:01.156399  1710 raft_consensus.cc:2955] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.161527  1466 catalog_manager.cc:5697] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.199723  2305 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:01.205286  2444 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:01.213197  1642 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:01.215595  2444 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:01.224681  2303 raft_consensus.cc:2955] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.218582  2390 raft_consensus.cc:2955] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.241118  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fbfadedb9a5e4ca0abc25384a524c546 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:01.248629  1642 raft_consensus.cc:2955] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.259712  1643 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:01.253620  1467 catalog_manager.cc:5697] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.272944  2303 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:01.275575  2364 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:01.276019  1567 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:01.278317  2364 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:01.283326  2457 raft_consensus.cc:2955] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.286628  1554 tablet_service.cc:1558] Processing DeleteTablet for tablet fbfadedb9a5e4ca0abc25384a524c546 with delete_type TABLET_DATA_TOMBSTONED (TS 3f64b76180a54492bb015e405caeacec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41262
I20260709 13:01:01.288635  1567 raft_consensus.cc:2955] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.287168  2303 raft_consensus.cc:2955] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.305688  2424 raft_consensus.cc:1064] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:01.308075  2424 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:01.323714  2305 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:01.324790  1565 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:01.327019  1642 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:01.326840  2387 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:01.328943  2387 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:01.336025  1466 catalog_manager.cc:5697] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.342962  2390 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:01.351624  1699 tablet_service.cc:1558] Processing DeleteTablet for tablet d1256ff04335423286d7c081ec776061 with delete_type TABLET_DATA_TOMBSTONED (TS 28bc59d37f664ce0a6f5784f2619f0b5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36054
I20260709 13:01:01.369623  2423 raft_consensus.cc:1064] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:01.371173  2463 tablet_replica.cc:333] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:01:01.374629  1452 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d1256ff04335423286d7c081ec776061 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:01.378155  2423 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:01.451545  2458 tablet_replica.cc:333] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
W20260709 13:01:01.451867  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 69620.6us
I20260709 13:01:01.453503  2458 raft_consensus.cc:2243] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:01.454430  2458 raft_consensus.cc:2272] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:01.455853  2424 raft_consensus.cc:2955] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.473078  2304 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:01.475674  1643 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:01:01.477705  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 95933.3us
I20260709 13:01:01.478336  1564 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:01.482391  2387 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:01.485562  2390 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:01.465894  2303 raft_consensus.cc:2955] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.513695  1565 raft_consensus.cc:2955] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:01:01.515225  2464 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:01:01.524255  2463 raft_consensus.cc:2243] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:01.526821  2466 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:01.531479  2463 raft_consensus.cc:2272] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:01.533452  1465 catalog_manager.cc:5697] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.537722  2463 ts_tablet_manager.cc:1916] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:01.518976  1643 raft_consensus.cc:2955] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.511519  2458 ts_tablet_manager.cc:1916] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:01.578614  1714 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:01.584319  2303 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:01.585080  1565 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:01.586028  2424 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:01.586059  2387 raft_consensus.cc:2955] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.587206  2423 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:01.593310  1640 raft_consensus.cc:2955] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.605800  2463 ts_tablet_manager.cc:1929] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:01.606346  2463 log.cc:1199] T d1256ff04335423286d7c081ec776061 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/wals/d1256ff04335423286d7c081ec776061
I20260709 13:01:01.606992  1567 raft_consensus.cc:2955] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.612082  1451 catalog_manager.cc:5028] TS 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): tablet d1256ff04335423286d7c081ec776061 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:01.616048  2304 raft_consensus.cc:2955] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.617671  2458 ts_tablet_manager.cc:1929] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:01.618122  2458 log.cc:1199] T fbfadedb9a5e4ca0abc25384a524c546 P 3f64b76180a54492bb015e405caeacec: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/wals/fbfadedb9a5e4ca0abc25384a524c546
I20260709 13:01:01.618021  1564 raft_consensus.cc:2955] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.597885  2467 raft_consensus.cc:2955] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.619884  1451 catalog_manager.cc:5028] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): tablet fbfadedb9a5e4ca0abc25384a524c546 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:01.632963  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6bdefe70d276420ca0283414ab16fab9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:01.642987  1466 catalog_manager.cc:5697] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.649019  1467 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.637414  2304 raft_consensus.cc:2955] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.664955  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 6bdefe70d276420ca0283414ab16fab9 with delete_type TABLET_DATA_TOMBSTONED (TS 5184c101be204393adf3ac8b31f25ad6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60370
I20260709 13:01:01.671139  2471 tablet_replica.cc:333] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:01:01.676046  1714 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:01.672492  2471 raft_consensus.cc:2243] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:01.679307  2471 raft_consensus.cc:2272] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:01.684798  2471 ts_tablet_manager.cc:1916] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:01.687299  2304 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:01.688925  2424 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:01.696139  1640 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:01.699072  2467 raft_consensus.cc:2955] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.704340  2387 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:01.706420  2305 raft_consensus.cc:2955] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.711162  2471 ts_tablet_manager.cc:1929] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:01.711700  2471 log.cc:1199] T 6bdefe70d276420ca0283414ab16fab9 P 5184c101be204393adf3ac8b31f25ad6: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/wals/6bdefe70d276420ca0283414ab16fab9
I20260709 13:01:01.712966  1643 raft_consensus.cc:2955] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.717240  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet 6bdefe70d276420ca0283414ab16fab9 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:01.731105  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9df7dc71f943412e83fbc8d4f788c9f2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:01.736083  1465 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:01.750687  1554 tablet_service.cc:1558] Processing DeleteTablet for tablet 9df7dc71f943412e83fbc8d4f788c9f2 with delete_type TABLET_DATA_TOMBSTONED (TS 3f64b76180a54492bb015e405caeacec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41262
I20260709 13:01:01.751643  2458 tablet_replica.cc:333] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:01:01.752705  2458 raft_consensus.cc:2243] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:01.753314  2458 raft_consensus.cc:2272] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:01.756860  2458 ts_tablet_manager.cc:1916] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:01.767998  2458 ts_tablet_manager.cc:1929] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:01.768319  2458 log.cc:1199] T 9df7dc71f943412e83fbc8d4f788c9f2 P 3f64b76180a54492bb015e405caeacec: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/wals/9df7dc71f943412e83fbc8d4f788c9f2
I20260709 13:01:01.769723  1451 catalog_manager.cc:5028] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): tablet 9df7dc71f943412e83fbc8d4f788c9f2 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:02.825289  1565 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:02.836221  1534 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d4bc96896acb4fcf8b5ec5b7206248ca. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:02.836519  1643 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:02.837184  1709 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:02.838405  2465 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:02.839545  2409 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:02.847868  2465 raft_consensus.cc:2955] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:02.849723  1643 raft_consensus.cc:2955] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:02.852408  1709 raft_consensus.cc:2955] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:02.861325  1567 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:02.868695  1465 catalog_manager.cc:5697] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:02.875036  1534 consensus_peers.cc:597] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c85446f87564ae6865060bb988ee1b8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:02.879323  1643 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:02.881326  1709 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:02.883186  2465 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:02.885358  2465 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:02.906423  2409 raft_consensus.cc:2955] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:02.908499  1709 raft_consensus.cc:2955] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:02.928655  1642 raft_consensus.cc:2955] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:02.931726  1709 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:02.951143  1642 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:02.951403  1567 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:02.951675  1683 consensus_peers.cc:597] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cdf54b70d44944ef9013c4adb0970687. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:02.952755  2467 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:02.953130  1466 catalog_manager.cc:5697] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:02.955776  2467 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:02.981724  2467 raft_consensus.cc:2955] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:02.985283  1642 raft_consensus.cc:2955] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:02.997799  1567 raft_consensus.cc:2955] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:03.002226  1643 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:03.010218  1465 catalog_manager.cc:5697] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:03.021271  1710 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:03.022781  2457 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:03.023803  1609 consensus_peers.cc:597] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 147a8e85db5b4380bf845a82736ec5fd. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:03.024960  1567 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:03.026430  2457 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:03.047003  2412 raft_consensus.cc:2955] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:03.049190  1710 raft_consensus.cc:2955] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:03.049031  1567 raft_consensus.cc:2955] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:03.064985  1464 catalog_manager.cc:5697] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:03.379765  2493 ts_tablet_manager.cc:933] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
I20260709 13:01:03.383903  2493 tablet_copy_client.cc:323] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:36547
I20260709 13:01:03.385607  1579 tablet_copy_service.cc:140] P 3f64b76180a54492bb015e405caeacec: Received BeginTabletCopySession request for tablet d4bc96896acb4fcf8b5ec5b7206248ca from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:35222)
I20260709 13:01:03.386171  1579 tablet_copy_service.cc:161] P 3f64b76180a54492bb015e405caeacec: Beginning new tablet copy session on tablet d4bc96896acb4fcf8b5ec5b7206248ca from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:35222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-d4bc96896acb4fcf8b5ec5b7206248ca
I20260709 13:01:03.392078  1579 tablet_copy_source_session.cc:215] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:03.394834  2493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4bc96896acb4fcf8b5ec5b7206248ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:03.411645  2493 tablet_copy_client.cc:806] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:03.412230  2493 tablet_copy_client.cc:670] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:03.416779  2493 tablet_copy_client.cc:538] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:03.425202  2493 tablet_bootstrap.cc:492] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:03.441910  2495 ts_tablet_manager.cc:933] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:03.461949  2493 tablet_bootstrap.cc:492] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:03.462688  2495 tablet_copy_client.cc:323] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:40023
I20260709 13:01:03.464399  2493 tablet_bootstrap.cc:492] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:03.464655  2496 ts_tablet_manager.cc:933] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
I20260709 13:01:03.464825  1654 tablet_copy_service.cc:140] P 5184c101be204393adf3ac8b31f25ad6: Received BeginTabletCopySession request for tablet 147a8e85db5b4380bf845a82736ec5fd from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:53222)
I20260709 13:01:03.465296  2493 ts_tablet_manager.cc:1403] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.040s	user 0.020s	sys 0.008s
I20260709 13:01:03.465449  1654 tablet_copy_service.cc:161] P 5184c101be204393adf3ac8b31f25ad6: Beginning new tablet copy session on tablet 147a8e85db5b4380bf845a82736ec5fd from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:53222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-147a8e85db5b4380bf845a82736ec5fd
I20260709 13:01:03.468649  2493 raft_consensus.cc:359] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:03.469563  2493 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:03.470229  2493 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:03.480083  2493 ts_tablet_manager.cc:1434] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 13:01:03.480647  1654 tablet_copy_source_session.cc:215] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:03.485512  1579 tablet_copy_service.cc:342] P 3f64b76180a54492bb015e405caeacec: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-d4bc96896acb4fcf8b5ec5b7206248ca received from {username='slave'} at 127.0.0.1:35222
I20260709 13:01:03.486004  1579 tablet_copy_service.cc:434] P 3f64b76180a54492bb015e405caeacec: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-d4bc96896acb4fcf8b5ec5b7206248ca on tablet d4bc96896acb4fcf8b5ec5b7206248ca with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:03.487092  2495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 147a8e85db5b4380bf845a82736ec5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:03.487531  2496 tablet_copy_client.cc:323] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:36547
I20260709 13:01:03.492967  1579 tablet_copy_service.cc:140] P 3f64b76180a54492bb015e405caeacec: Received BeginTabletCopySession request for tablet 8c85446f87564ae6865060bb988ee1b8 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:35222)
I20260709 13:01:03.493606  1579 tablet_copy_service.cc:161] P 3f64b76180a54492bb015e405caeacec: Beginning new tablet copy session on tablet 8c85446f87564ae6865060bb988ee1b8 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:35222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-8c85446f87564ae6865060bb988ee1b8
I20260709 13:01:03.499766  1579 tablet_copy_source_session.cc:215] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:03.504009  2493 ts_tablet_manager.cc:933] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:03.505328  2495 tablet_copy_client.cc:806] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:03.505856  2496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c85446f87564ae6865060bb988ee1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:03.505967  2495 tablet_copy_client.cc:670] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:03.510407  2495 tablet_copy_client.cc:538] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:03.517994  2493 tablet_copy_client.cc:323] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:03.519949  1728 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet cdf54b70d44944ef9013c4adb0970687 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:03.520586  1728 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet cdf54b70d44944ef9013c4adb0970687 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-cdf54b70d44944ef9013c4adb0970687
I20260709 13:01:03.521947  2495 tablet_bootstrap.cc:492] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:03.537932  1728 tablet_copy_source_session.cc:215] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:03.545061  2493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdf54b70d44944ef9013c4adb0970687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:03.559317  2495 tablet_bootstrap.cc:492] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:03.560372  2495 tablet_bootstrap.cc:492] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:03.561040  2495 ts_tablet_manager.cc:1403] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.011s
I20260709 13:01:03.564672  2495 raft_consensus.cc:359] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:03.565565  2495 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:03.566237  2495 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:03.569383  2495 ts_tablet_manager.cc:1434] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:01:03.573501  1654 tablet_copy_service.cc:342] P 5184c101be204393adf3ac8b31f25ad6: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-147a8e85db5b4380bf845a82736ec5fd received from {username='slave'} at 127.0.0.1:53222
I20260709 13:01:03.576550  1654 tablet_copy_service.cc:434] P 5184c101be204393adf3ac8b31f25ad6: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-147a8e85db5b4380bf845a82736ec5fd on tablet 147a8e85db5b4380bf845a82736ec5fd with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:03.587754  2493 tablet_copy_client.cc:806] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:03.588531  2493 tablet_copy_client.cc:670] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:03.600926  2493 tablet_copy_client.cc:538] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:03.604596  2496 tablet_copy_client.cc:806] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:03.605276  2496 tablet_copy_client.cc:670] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:03.609221  2496 tablet_copy_client.cc:538] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:03.629730  2496 tablet_bootstrap.cc:492] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:03.642421  2493 tablet_bootstrap.cc:492] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:03.688834  2496 tablet_bootstrap.cc:492] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:03.689177  2493 tablet_bootstrap.cc:492] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:03.689674  2496 tablet_bootstrap.cc:492] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:03.690119  2493 tablet_bootstrap.cc:492] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:03.690310  2496 ts_tablet_manager.cc:1403] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.014s
I20260709 13:01:03.691045  2493 ts_tablet_manager.cc:1403] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.049s	user 0.028s	sys 0.005s
I20260709 13:01:03.692863  2496 raft_consensus.cc:359] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:03.693531  2496 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:03.693985  2493 raft_consensus.cc:359] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:03.694885  2493 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:03.694156  2496 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:03.695595  2493 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:03.697070  2496 ts_tablet_manager.cc:1434] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:01:03.698339  2493 ts_tablet_manager.cc:1434] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:01:03.700584  1579 tablet_copy_service.cc:342] P 3f64b76180a54492bb015e405caeacec: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-8c85446f87564ae6865060bb988ee1b8 received from {username='slave'} at 127.0.0.1:35222
I20260709 13:01:03.700882  1728 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-cdf54b70d44944ef9013c4adb0970687 received from {username='slave'} at 127.0.0.1:33280
I20260709 13:01:03.701123  1579 tablet_copy_service.cc:434] P 3f64b76180a54492bb015e405caeacec: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-8c85446f87564ae6865060bb988ee1b8 on tablet 8c85446f87564ae6865060bb988ee1b8 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:03.701361  1728 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-cdf54b70d44944ef9013c4adb0970687 on tablet cdf54b70d44944ef9013c4adb0970687 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:03.888651  2304 raft_consensus.cc:1217] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:03.966557  2304 raft_consensus.cc:1217] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:03.971907  2304 raft_consensus.cc:1217] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:03.990469  2304 raft_consensus.cc:1217] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:04.402016  2412 raft_consensus.cc:1064] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
W20260709 13:01:04.403631  2472 consensus_queue.cc:1175] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3f64b76180a54492bb015e405caeacec" responder_term: 1 status { last_received { term: 1 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 1 index: 2 } }
I20260709 13:01:04.404986  2412 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:04.415207  2304 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.415818  1709 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.416293  1565 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.417198  2504 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:04.418597  2412 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:04.419324  2508 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:04.445708  2508 raft_consensus.cc:2955] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.454181  2304 raft_consensus.cc:2955] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.480157  1466 catalog_manager.cc:5697] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.481253  1709 raft_consensus.cc:2955] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.495921  2484 raft_consensus.cc:1064] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:04.498602  2484 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:04.499254  1565 raft_consensus.cc:2955] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.529749  2503 raft_consensus.cc:1064] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:04.532765  2503 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:04.538451  1709 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.539532  2305 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.540077  2465 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:04.542200  2465 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:04.551035  1640 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:04.557226  2304 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:04.559983  2507 raft_consensus.cc:1064] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:04.562260  2412 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:04.562618  2507 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:04.564916  1714 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:04.570044  2304 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.571873  2472 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:04.579840  1714 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.587142  2484 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:04.589268  2412 raft_consensus.cc:2955] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.597659  2465 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:04.600660  1452 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 147a8e85db5b4380bf845a82736ec5fd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:04.601224  2304 raft_consensus.cc:2955] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.606986  1714 raft_consensus.cc:2955] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.625332  1467 catalog_manager.cc:5697] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:04.646368  2503 raft_consensus.cc:2955] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.648967  2518 raft_consensus.cc:2955] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.648277  2305 raft_consensus.cc:2955] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.661597  2304 raft_consensus.cc:2955] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.666043  1642 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.668411  2424 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:04.674831  1554 tablet_service.cc:1558] Processing DeleteTablet for tablet 147a8e85db5b4380bf845a82736ec5fd with delete_type TABLET_DATA_TOMBSTONED (TS 3f64b76180a54492bb015e405caeacec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41262
I20260709 13:01:04.676307  1640 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.681037  1640 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.694604  1467 catalog_manager.cc:5697] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.695899  2465 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:04.699277  2465 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:04.703991  1567 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.705977  2305 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:04.706631  1714 raft_consensus.cc:2955] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.712147  2528 tablet_replica.cc:333] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:01:04.705862  2424 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:04.713423  2528 raft_consensus.cc:2243] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:04.714071  2467 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:04.714567  2528 raft_consensus.cc:2272] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:04.717244  1465 catalog_manager.cc:5697] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.725656  1710 raft_consensus.cc:2955] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.732676  2528 ts_tablet_manager.cc:1916] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:04.757423  1640 raft_consensus.cc:2955] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.763495  1642 raft_consensus.cc:2955] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.798393  2528 ts_tablet_manager.cc:1929] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:04.801291  2528 log.cc:1199] T 147a8e85db5b4380bf845a82736ec5fd P 3f64b76180a54492bb015e405caeacec: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/wals/147a8e85db5b4380bf845a82736ec5fd
I20260709 13:01:04.803067  1451 catalog_manager.cc:5028] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): tablet 147a8e85db5b4380bf845a82736ec5fd (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:04.802832  2467 raft_consensus.cc:2955] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.806638  2305 raft_consensus.cc:2955] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.807888  1567 raft_consensus.cc:2955] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.822320  1465 catalog_manager.cc:5697] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.835239  1640 raft_consensus.cc:2955] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.854075  1710 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:04.863772  1567 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:04.864166  2305 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:04.866606  2424 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:04.870239  2467 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:04.879024  2489 raft_consensus.cc:2955] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.881718  2305 raft_consensus.cc:2955] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.894850  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cdf54b70d44944ef9013c4adb0970687 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:04.902141  1465 catalog_manager.cc:5697] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.918043  1567 raft_consensus.cc:2955] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:04.924571  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet cdf54b70d44944ef9013c4adb0970687 with delete_type TABLET_DATA_TOMBSTONED (TS 5184c101be204393adf3ac8b31f25ad6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60370
I20260709 13:01:04.927788  2534 tablet_replica.cc:333] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:01:04.929811  2534 raft_consensus.cc:2243] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:04.936163  2534 raft_consensus.cc:2272] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:04.943020  2534 ts_tablet_manager.cc:1916] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:04.961826  2534 ts_tablet_manager.cc:1929] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:04.962291  2534 log.cc:1199] T cdf54b70d44944ef9013c4adb0970687 P 5184c101be204393adf3ac8b31f25ad6: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/wals/cdf54b70d44944ef9013c4adb0970687
I20260709 13:01:04.964706  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet cdf54b70d44944ef9013c4adb0970687 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:05.127924  1564 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:05.133260  2305 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:05.133509  1640 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:05.135244  2526 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:05.136870  2518 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:05.142771  2526 raft_consensus.cc:2955] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:05.145879  1640 raft_consensus.cc:2955] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:05.145392  2305 raft_consensus.cc:2955] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:05.157337  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d4bc96896acb4fcf8b5ec5b7206248ca with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:05.169310  1467 catalog_manager.cc:5697] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:05.187446  1699 tablet_service.cc:1558] Processing DeleteTablet for tablet d4bc96896acb4fcf8b5ec5b7206248ca with delete_type TABLET_DATA_TOMBSTONED (TS 28bc59d37f664ce0a6f5784f2619f0b5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36054
I20260709 13:01:05.191844  2535 tablet_replica.cc:333] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:01:05.193854  2535 raft_consensus.cc:2243] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:05.194639  2535 raft_consensus.cc:2272] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:05.199625  2535 ts_tablet_manager.cc:1916] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:05.216758  2535 ts_tablet_manager.cc:1929] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:05.217165  2535 log.cc:1199] T d4bc96896acb4fcf8b5ec5b7206248ca P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/wals/d4bc96896acb4fcf8b5ec5b7206248ca
I20260709 13:01:05.218969  1451 catalog_manager.cc:5028] TS 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): tablet d4bc96896acb4fcf8b5ec5b7206248ca (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:05.293105  1565 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:05.301295  2305 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:05.303488  2526 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:05.303812  1642 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:05.305269  2518 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:05.311175  2465 raft_consensus.cc:2955] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:05.316033  1642 raft_consensus.cc:2955] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:05.323735  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8c85446f87564ae6865060bb988ee1b8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:05.323532  2305 raft_consensus.cc:2955] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:05.337652  1467 catalog_manager.cc:5697] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:05.449009  1699 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c85446f87564ae6865060bb988ee1b8 with delete_type TABLET_DATA_TOMBSTONED (TS 28bc59d37f664ce0a6f5784f2619f0b5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36054
I20260709 13:01:05.455765  2535 tablet_replica.cc:333] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:01:05.460577  2535 raft_consensus.cc:2243] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:05.463888  2535 raft_consensus.cc:2272] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:05.473742  2535 ts_tablet_manager.cc:1916] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:05.503886  2535 ts_tablet_manager.cc:1929] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:05.506240  2535 log.cc:1199] T 8c85446f87564ae6865060bb988ee1b8 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/wals/8c85446f87564ae6865060bb988ee1b8
I20260709 13:01:05.509480  1451 catalog_manager.cc:5028] TS 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): tablet 8c85446f87564ae6865060bb988ee1b8 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:06.539405  1714 consensus_queue.cc:237] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:06.546820  1642 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:06.548543  1564 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:06.549057  2542 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:06.550742  2539 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:06.553332  1683 consensus_peers.cc:597] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0049d6871cf4f95be599108abda58a5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:06.561777  2542 raft_consensus.cc:2955] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:06.568418  1564 raft_consensus.cc:2955] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:06.564606  1642 raft_consensus.cc:2955] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:06.587263  1467 catalog_manager.cc:5697] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:06.588033  1714 consensus_queue.cc:237] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:06.600282  1683 consensus_peers.cc:597] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d9ba36df4f014c3a916b0393305ea44a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:06.600401  1642 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:06.600486  1564 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:06.602149  2542 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:06.607098  2542 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:06.620316  2538 raft_consensus.cc:2955] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:06.624528  1567 raft_consensus.cc:2955] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:06.625540  1642 raft_consensus.cc:2955] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:06.639361  1565 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:06.645804  1465 catalog_manager.cc:5697] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:06.653029  1714 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:06.657019  2518 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:06.660218  1534 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9f6815b361554c338159f274bd78c0e9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:06.664806  1642 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:06.667716  2526 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:06.682329  2553 raft_consensus.cc:2955] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:06.684723  1714 raft_consensus.cc:2955] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:06.684947  1642 raft_consensus.cc:2955] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:06.700817  1465 catalog_manager.cc:5697] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:06.706411  1642 consensus_queue.cc:237] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:06.718396  1609 consensus_peers.cc:597] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d5ccce7ae459467a984bd58d73f701ae. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:06.718534  1714 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:06.721798  2412 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:06.722175  1564 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:06.726855  2472 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:06.737069  2540 raft_consensus.cc:2955] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:06.737676  1564 raft_consensus.cc:2955] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:06.738256  1710 raft_consensus.cc:2955] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:06.776579  1467 catalog_manager.cc:5697] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:07.038318  2560 ts_tablet_manager.cc:933] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:07.040529  2560 tablet_copy_client.cc:323] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:07.043139  1728 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet d9ba36df4f014c3a916b0393305ea44a from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:07.043720  1728 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet d9ba36df4f014c3a916b0393305ea44a from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-d9ba36df4f014c3a916b0393305ea44a
I20260709 13:01:07.045425  2562 ts_tablet_manager.cc:933] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:07.061707  2562 tablet_copy_client.cc:323] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:07.062904  1728 tablet_copy_source_session.cc:215] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:07.063778  1727 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet d0049d6871cf4f95be599108abda58a5 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:07.064357  1727 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet d0049d6871cf4f95be599108abda58a5 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-d0049d6871cf4f95be599108abda58a5
I20260709 13:01:07.066941  2560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9ba36df4f014c3a916b0393305ea44a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:07.073941  1727 tablet_copy_source_session.cc:215] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:07.077996  2562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0049d6871cf4f95be599108abda58a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:07.086688  2560 tablet_copy_client.cc:806] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:07.087395  2560 tablet_copy_client.cc:670] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:07.100566  2560 tablet_copy_client.cc:538] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:07.102308  2562 tablet_copy_client.cc:806] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:07.102974  2562 tablet_copy_client.cc:670] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:07.112583  2560 tablet_bootstrap.cc:492] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:07.113397  2562 tablet_copy_client.cc:538] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:07.126907  2562 tablet_bootstrap.cc:492] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:07.156524  2560 tablet_bootstrap.cc:492] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:07.160233  2560 tablet_bootstrap.cc:492] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:07.166101  2560 ts_tablet_manager.cc:1403] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.054s	user 0.022s	sys 0.012s
I20260709 13:01:07.171151  2560 raft_consensus.cc:359] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:07.172266  2560 raft_consensus.cc:740] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:07.173193  2560 consensus_queue.cc:260] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:07.178323  2562 tablet_bootstrap.cc:492] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:07.178385  2560 ts_tablet_manager.cc:1434] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.003s
I20260709 13:01:07.181280  2562 tablet_bootstrap.cc:492] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:07.182137  2562 ts_tablet_manager.cc:1403] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.055s	user 0.029s	sys 0.009s
I20260709 13:01:07.182267  1727 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-d9ba36df4f014c3a916b0393305ea44a received from {username='slave'} at 127.0.0.1:33280
I20260709 13:01:07.182886  1727 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-d9ba36df4f014c3a916b0393305ea44a on tablet d9ba36df4f014c3a916b0393305ea44a with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:07.186976  2562 raft_consensus.cc:359] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:07.187781  2562 raft_consensus.cc:740] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:07.188448  2562 consensus_queue.cc:260] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:07.191617  2562 ts_tablet_manager.cc:1434] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:01:07.196555  1727 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-d0049d6871cf4f95be599108abda58a5 received from {username='slave'} at 127.0.0.1:33280
I20260709 13:01:07.197088  1727 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-d0049d6871cf4f95be599108abda58a5 on tablet d0049d6871cf4f95be599108abda58a5 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:07.230986  2562 ts_tablet_manager.cc:933] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:07.232697  2562 tablet_copy_client.cc:323] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:40023
I20260709 13:01:07.234786  1654 tablet_copy_service.cc:140] P 5184c101be204393adf3ac8b31f25ad6: Received BeginTabletCopySession request for tablet d5ccce7ae459467a984bd58d73f701ae from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:53222)
I20260709 13:01:07.235531  1654 tablet_copy_service.cc:161] P 5184c101be204393adf3ac8b31f25ad6: Beginning new tablet copy session on tablet d5ccce7ae459467a984bd58d73f701ae from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:53222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-d5ccce7ae459467a984bd58d73f701ae
I20260709 13:01:07.244946  2560 ts_tablet_manager.cc:933] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
I20260709 13:01:07.248032  1654 tablet_copy_source_session.cc:215] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:07.252177  2560 tablet_copy_client.cc:323] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:36547
I20260709 13:01:07.252151  2562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5ccce7ae459467a984bd58d73f701ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:07.256742  1579 tablet_copy_service.cc:140] P 3f64b76180a54492bb015e405caeacec: Received BeginTabletCopySession request for tablet 9f6815b361554c338159f274bd78c0e9 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:35222)
I20260709 13:01:07.257395  1579 tablet_copy_service.cc:161] P 3f64b76180a54492bb015e405caeacec: Beginning new tablet copy session on tablet 9f6815b361554c338159f274bd78c0e9 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:35222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-9f6815b361554c338159f274bd78c0e9
I20260709 13:01:07.272742  1579 tablet_copy_source_session.cc:215] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:07.279532  2560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6815b361554c338159f274bd78c0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:07.293387  2562 tablet_copy_client.cc:806] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:07.293995  2562 tablet_copy_client.cc:670] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:07.298341  2562 tablet_copy_client.cc:538] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:07.307156  2560 tablet_copy_client.cc:806] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:07.307912  2560 tablet_copy_client.cc:670] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:07.308266  2562 tablet_bootstrap.cc:492] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:07.312491  2560 tablet_copy_client.cc:538] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:07.322167  2560 tablet_bootstrap.cc:492] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:07.342469  2562 tablet_bootstrap.cc:492] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:07.343425  2562 tablet_bootstrap.cc:492] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:07.344151  2562 ts_tablet_manager.cc:1403] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.036s	user 0.020s	sys 0.015s
I20260709 13:01:07.347107  2562 raft_consensus.cc:359] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:07.347966  2562 raft_consensus.cc:740] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:07.348682  2562 consensus_queue.cc:260] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:07.355634  2560 tablet_bootstrap.cc:492] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:07.356778  2560 tablet_bootstrap.cc:492] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:07.357517  2560 ts_tablet_manager.cc:1403] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.012s
I20260709 13:01:07.359078  2562 ts_tablet_manager.cc:1434] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.001s
I20260709 13:01:07.362826  2560 raft_consensus.cc:359] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:07.363771  2560 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:07.364429  2560 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:07.367316  2560 ts_tablet_manager.cc:1434] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:01:07.369021  1579 tablet_copy_service.cc:342] P 3f64b76180a54492bb015e405caeacec: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-9f6815b361554c338159f274bd78c0e9 received from {username='slave'} at 127.0.0.1:35222
I20260709 13:01:07.369470  1579 tablet_copy_service.cc:434] P 3f64b76180a54492bb015e405caeacec: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-9f6815b361554c338159f274bd78c0e9 on tablet 9f6815b361554c338159f274bd78c0e9 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:07.383100  1654 tablet_copy_service.cc:342] P 5184c101be204393adf3ac8b31f25ad6: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-d5ccce7ae459467a984bd58d73f701ae received from {username='slave'} at 127.0.0.1:53222
I20260709 13:01:07.383592  1654 tablet_copy_service.cc:434] P 5184c101be204393adf3ac8b31f25ad6: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-d5ccce7ae459467a984bd58d73f701ae on tablet d5ccce7ae459467a984bd58d73f701ae with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:07.475503  2304 raft_consensus.cc:1217] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:07.593796  2304 raft_consensus.cc:1217] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:07.752396  2304 raft_consensus.cc:1217] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:07.848691  2304 raft_consensus.cc:1217] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:07.887182  2542 raft_consensus.cc:1064] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:07.889941  2542 consensus_queue.cc:237] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:07.898706  2304 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:07.899423  1565 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:07.899748  1640 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:07.900381  2539 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:07.902941  2571 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:07.907317  2542 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:07.936275  2539 raft_consensus.cc:2955] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:07.938244  1642 raft_consensus.cc:2955] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:07.943377  1568 raft_consensus.cc:2955] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:07.958425  1465 catalog_manager.cc:5697] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:07.983841  2305 raft_consensus.cc:2955] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.012737  1714 consensus_queue.cc:237] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:08.018100  2305 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:08.018318  1568 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:08.020023  2571 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:08.021924  2571 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:08.028779  2538 raft_consensus.cc:2955] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.035624  2305 raft_consensus.cc:2955] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.038682  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d9ba36df4f014c3a916b0393305ea44a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:08.041042  1568 raft_consensus.cc:2955] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.051301  1467 catalog_manager.cc:5697] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:08.066953  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet d9ba36df4f014c3a916b0393305ea44a with delete_type TABLET_DATA_TOMBSTONED (TS 5184c101be204393adf3ac8b31f25ad6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60370
I20260709 13:01:08.069031  2580 tablet_replica.cc:333] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:01:08.070163  2580 raft_consensus.cc:2243] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.071295  2580 raft_consensus.cc:2272] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.077420  2580 ts_tablet_manager.cc:1916] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:08.094179  2580 ts_tablet_manager.cc:1929] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:08.094719  2580 log.cc:1199] T d9ba36df4f014c3a916b0393305ea44a P 5184c101be204393adf3ac8b31f25ad6: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/wals/d9ba36df4f014c3a916b0393305ea44a
I20260709 13:01:08.103461  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet d9ba36df4f014c3a916b0393305ea44a (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:08.130141  2539 raft_consensus.cc:1064] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:08.132849  2539 consensus_queue.cc:237] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:08.139891  1642 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:08.139891  2305 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:08.141597  2539 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:08.141984  1565 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:08.142889  2571 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:08.144481  2542 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:08.173448  2542 raft_consensus.cc:2955] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.175633  1642 raft_consensus.cc:2955] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.188376  2304 raft_consensus.cc:2955] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.206277  1466 catalog_manager.cc:5697] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:08.224288  1568 raft_consensus.cc:2955] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.236732  2540 raft_consensus.cc:1064] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:08.239374  2540 consensus_queue.cc:237] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:08.253046  1714 consensus_queue.cc:237] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:08.267997  2304 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:08.269913  2542 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:08.271843  1642 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:08.279309  2304 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:08.278393  2571 raft_consensus.cc:2955] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.280598  1714 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:08.281159  2585 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:08.282660  2540 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:08.284574  2304 raft_consensus.cc:2955] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.299180  2538 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:08.300659  1568 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:08.306025  2540 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:08.308723  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d0049d6871cf4f95be599108abda58a5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:08.318956  1467 catalog_manager.cc:5697] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:08.320152  1642 raft_consensus.cc:2955] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.336560  1554 tablet_service.cc:1558] Processing DeleteTablet for tablet d0049d6871cf4f95be599108abda58a5 with delete_type TABLET_DATA_TOMBSTONED (TS 3f64b76180a54492bb015e405caeacec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41262
I20260709 13:01:08.345312  2518 raft_consensus.cc:1064] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:08.348181  2518 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:08.368614  2305 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:08.369311  1709 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:08.370128  2526 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:08.370427  2472 raft_consensus.cc:2955] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.371927  2526 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:08.379070  1640 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:08.380586  2518 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:08.397601  2304 raft_consensus.cc:2955] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.400334  2591 tablet_replica.cc:333] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:01:08.401566  2591 raft_consensus.cc:2243] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.402437  2591 raft_consensus.cc:2272] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.406396  2591 ts_tablet_manager.cc:1916] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:01:08.406571  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 43690.1us
I20260709 13:01:08.410033  1564 raft_consensus.cc:2955] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.423750  1466 catalog_manager.cc:5697] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:08.426314  2591 ts_tablet_manager.cc:1929] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:08.426945  2591 log.cc:1199] T d0049d6871cf4f95be599108abda58a5 P 3f64b76180a54492bb015e405caeacec: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/wals/d0049d6871cf4f95be599108abda58a5
I20260709 13:01:08.430418  1451 catalog_manager.cc:5028] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): tablet d0049d6871cf4f95be599108abda58a5 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
W20260709 13:01:08.433465  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 61926.2us
I20260709 13:01:08.433732  2518 raft_consensus.cc:2955] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.440977  1709 raft_consensus.cc:2955] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.449958  2304 raft_consensus.cc:2955] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.453460  1640 raft_consensus.cc:2955] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.466481  1467 catalog_manager.cc:5697] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:08.470250  1642 consensus_queue.cc:237] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:08.485776  1565 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:08.493687  1643 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:08.496580  2518 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:08.500422  2305 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:08.503134  2303 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:08.504098  2518 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:08.506287  1709 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:08.505643  2526 raft_consensus.cc:2955] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.508591  2472 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:08.509965  1643 raft_consensus.cc:2955] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.512297  2472 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:08.519364  2304 raft_consensus.cc:2955] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.522735  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9f6815b361554c338159f274bd78c0e9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:08.528153  2585 raft_consensus.cc:2955] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.538861  2304 raft_consensus.cc:2955] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.540086  1467 catalog_manager.cc:5697] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:08.556365  1699 tablet_service.cc:1558] Processing DeleteTablet for tablet 9f6815b361554c338159f274bd78c0e9 with delete_type TABLET_DATA_TOMBSTONED (TS 28bc59d37f664ce0a6f5784f2619f0b5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36054
I20260709 13:01:08.556448  1452 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d5ccce7ae459467a984bd58d73f701ae with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:08.568179  1709 raft_consensus.cc:2955] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.569944  1467 catalog_manager.cc:5697] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:08.575740  2598 tablet_replica.cc:333] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:01:08.576995  2598 raft_consensus.cc:2243] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.577665  2598 pending_rounds.cc:70] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5: Trying to abort 1 pending ops.
I20260709 13:01:08.577991  2598 pending_rounds.cc:77] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305634071947845632 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "9f6815b361554c338159f274bd78c0e9" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } } }
I20260709 13:01:08.579329  2598 raft_consensus.cc:2917] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 13:01:08.580247  2598 raft_consensus.cc:2889] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:01:08.580928  2598 raft_consensus.cc:2272] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.585258  1554 tablet_service.cc:1558] Processing DeleteTablet for tablet d5ccce7ae459467a984bd58d73f701ae with delete_type TABLET_DATA_TOMBSTONED (TS 3f64b76180a54492bb015e405caeacec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41262
I20260709 13:01:08.586215  2591 tablet_replica.cc:333] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:01:08.586899  2591 raft_consensus.cc:2243] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:08.587718  2591 raft_consensus.cc:2272] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:08.592495  2591 ts_tablet_manager.cc:1916] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:08.592640  2598 ts_tablet_manager.cc:1916] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:08.608641  2591 ts_tablet_manager.cc:1929] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:08.608641  2598 ts_tablet_manager.cc:1929] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:08.609232  2591 log.cc:1199] T d5ccce7ae459467a984bd58d73f701ae P 3f64b76180a54492bb015e405caeacec: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/wals/d5ccce7ae459467a984bd58d73f701ae
I20260709 13:01:08.609414  2598 log.cc:1199] T 9f6815b361554c338159f274bd78c0e9 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/wals/9f6815b361554c338159f274bd78c0e9
I20260709 13:01:08.610770  1451 catalog_manager.cc:5028] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): tablet d5ccce7ae459467a984bd58d73f701ae (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:08.611563  1451 catalog_manager.cc:5028] TS 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): tablet 9f6815b361554c338159f274bd78c0e9 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:09.653290  1568 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:09.658895  1709 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:09.659562  1640 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:09.659888  1534 consensus_peers.cc:597] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4594cfed297144b39e1a29473c9707d9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:09.660653  2599 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:09.661661  2526 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:09.668716  2601 raft_consensus.cc:2955] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:09.671669  1714 raft_consensus.cc:2955] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:09.671456  1640 raft_consensus.cc:2955] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:09.686997  1467 catalog_manager.cc:5697] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:09.696676  1564 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:09.707435  1714 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:01:09.708400  1534 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 408b4ec829c54266b41334aac8245405. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:09.708920  1643 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:09.709897  2601 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:09.711015  2526 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:09.721724  2601 raft_consensus.cc:2955] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:09.723497  1709 raft_consensus.cc:2955] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:09.739612  1466 catalog_manager.cc:5697] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:09.751865  1568 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:09.755685  1643 raft_consensus.cc:2955] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:09.762524  1714 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:09.763264  1534 consensus_peers.cc:597] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8a4760a45328493e87523c2ee8292599. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:09.765462  1640 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:09.773456  2601 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:09.774490  2526 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:09.789140  2599 raft_consensus.cc:2955] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:09.794998  1714 raft_consensus.cc:2955] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:09.799216  1640 raft_consensus.cc:2955] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:09.808080  1709 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:09.814810  1466 catalog_manager.cc:5697] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:09.849086  1683 consensus_peers.cc:597] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 213d9f9af66a4ebd8c9a7b0163196a57. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:09.849287  1640 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:09.852123  1564 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:09.851608  2589 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:09.854846  2589 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:09.861685  2539 raft_consensus.cc:2955] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:09.863893  1640 raft_consensus.cc:2955] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:09.878042  1564 raft_consensus.cc:2955] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:09.889410  1466 catalog_manager.cc:5697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:10.080318  2618 ts_tablet_manager.cc:933] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
I20260709 13:01:10.088315  2618 tablet_copy_client.cc:323] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:36547
I20260709 13:01:10.090245  1579 tablet_copy_service.cc:140] P 3f64b76180a54492bb015e405caeacec: Received BeginTabletCopySession request for tablet 4594cfed297144b39e1a29473c9707d9 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:35222)
I20260709 13:01:10.090790  1579 tablet_copy_service.cc:161] P 3f64b76180a54492bb015e405caeacec: Beginning new tablet copy session on tablet 4594cfed297144b39e1a29473c9707d9 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:35222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-4594cfed297144b39e1a29473c9707d9
I20260709 13:01:10.096583  1579 tablet_copy_source_session.cc:215] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:10.099704  2618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4594cfed297144b39e1a29473c9707d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.113982  2618 tablet_copy_client.cc:806] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:10.114619  2618 tablet_copy_client.cc:670] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:10.128108  2618 tablet_copy_client.cc:538] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:10.137252  2618 tablet_bootstrap.cc:492] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:10.163640  2618 tablet_bootstrap.cc:492] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:10.164638  2618 tablet_bootstrap.cc:492] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:10.165637  2618 ts_tablet_manager.cc:1403] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.014s
I20260709 13:01:10.169299  2618 raft_consensus.cc:359] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:10.170293  2618 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:10.171140  2618 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:10.174861  2618 ts_tablet_manager.cc:1434] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.001s
I20260709 13:01:10.176842  1579 tablet_copy_service.cc:342] P 3f64b76180a54492bb015e405caeacec: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-4594cfed297144b39e1a29473c9707d9 received from {username='slave'} at 127.0.0.1:35222
I20260709 13:01:10.177294  1579 tablet_copy_service.cc:434] P 3f64b76180a54492bb015e405caeacec: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-4594cfed297144b39e1a29473c9707d9 on tablet 4594cfed297144b39e1a29473c9707d9 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:10.316193  2618 ts_tablet_manager.cc:933] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
I20260709 13:01:10.319176  2618 tablet_copy_client.cc:323] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:36547
I20260709 13:01:10.322570  1579 tablet_copy_service.cc:140] P 3f64b76180a54492bb015e405caeacec: Received BeginTabletCopySession request for tablet 408b4ec829c54266b41334aac8245405 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:35222)
I20260709 13:01:10.323298  1579 tablet_copy_service.cc:161] P 3f64b76180a54492bb015e405caeacec: Beginning new tablet copy session on tablet 408b4ec829c54266b41334aac8245405 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:35222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-408b4ec829c54266b41334aac8245405
I20260709 13:01:10.334444  1579 tablet_copy_source_session.cc:215] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:10.340767  2623 ts_tablet_manager.cc:933] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:10.341578  2618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 408b4ec829c54266b41334aac8245405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.358933  2625 ts_tablet_manager.cc:933] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
I20260709 13:01:10.359404  2623 tablet_copy_client.cc:323] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:10.362709  1727 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet 213d9f9af66a4ebd8c9a7b0163196a57 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:10.363209  1727 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet 213d9f9af66a4ebd8c9a7b0163196a57 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-213d9f9af66a4ebd8c9a7b0163196a57
I20260709 13:01:10.373669  1727 tablet_copy_source_session.cc:215] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:10.376549  2618 tablet_copy_client.cc:806] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:10.379742  2623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 213d9f9af66a4ebd8c9a7b0163196a57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.387035  2625 tablet_copy_client.cc:323] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:36547
I20260709 13:01:10.388736  2618 tablet_copy_client.cc:670] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:10.389269  1579 tablet_copy_service.cc:140] P 3f64b76180a54492bb015e405caeacec: Received BeginTabletCopySession request for tablet 8a4760a45328493e87523c2ee8292599 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:35222)
I20260709 13:01:10.389827  1579 tablet_copy_service.cc:161] P 3f64b76180a54492bb015e405caeacec: Beginning new tablet copy session on tablet 8a4760a45328493e87523c2ee8292599 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:35222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-8a4760a45328493e87523c2ee8292599
I20260709 13:01:10.395960  2618 tablet_copy_client.cc:538] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:10.398968  1579 tablet_copy_source_session.cc:215] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:10.402338  2625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a4760a45328493e87523c2ee8292599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:10.403687  2623 tablet_copy_client.cc:806] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:10.409423  2623 tablet_copy_client.cc:670] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:10.412788  2618 tablet_bootstrap.cc:492] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:10.417711  2623 tablet_copy_client.cc:538] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:10.435535  2623 tablet_bootstrap.cc:492] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:10.435593  2625 tablet_copy_client.cc:806] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:10.436515  2625 tablet_copy_client.cc:670] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:10.440677  2625 tablet_copy_client.cc:538] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:10.451711  2625 tablet_bootstrap.cc:492] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:10.463944  2618 tablet_bootstrap.cc:492] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:10.465451  2618 tablet_bootstrap.cc:492] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:10.466342  2618 ts_tablet_manager.cc:1403] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.054s	user 0.036s	sys 0.007s
I20260709 13:01:10.473335  2618 raft_consensus.cc:359] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:10.474398  2618 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:10.475260  2618 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:10.484233  2618 ts_tablet_manager.cc:1434] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:01:10.486879  1579 tablet_copy_service.cc:342] P 3f64b76180a54492bb015e405caeacec: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-408b4ec829c54266b41334aac8245405 received from {username='slave'} at 127.0.0.1:35222
I20260709 13:01:10.487434  1579 tablet_copy_service.cc:434] P 3f64b76180a54492bb015e405caeacec: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-408b4ec829c54266b41334aac8245405 on tablet 408b4ec829c54266b41334aac8245405 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:10.494293  2623 tablet_bootstrap.cc:492] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:10.505031  2623 tablet_bootstrap.cc:492] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:10.505828  2623 ts_tablet_manager.cc:1403] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.071s	user 0.012s	sys 0.018s
I20260709 13:01:10.503901  2625 tablet_bootstrap.cc:492] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:10.509114  2623 raft_consensus.cc:359] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:10.510027  2625 tablet_bootstrap.cc:492] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:10.510079  2623 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:10.510985  2625 ts_tablet_manager.cc:1403] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.060s	user 0.027s	sys 0.002s
I20260709 13:01:10.511034  2623 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:10.515614  2623 ts_tablet_manager.cc:1434] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:01:10.517396  1727 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-213d9f9af66a4ebd8c9a7b0163196a57 received from {username='slave'} at 127.0.0.1:33280
I20260709 13:01:10.517897  1727 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-213d9f9af66a4ebd8c9a7b0163196a57 on tablet 213d9f9af66a4ebd8c9a7b0163196a57 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:10.517457  2625 raft_consensus.cc:359] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:10.518520  2625 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:10.519340  2625 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:10.522403  2625 ts_tablet_manager.cc:1434] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:01:10.525234  1579 tablet_copy_service.cc:342] P 3f64b76180a54492bb015e405caeacec: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-8a4760a45328493e87523c2ee8292599 received from {username='slave'} at 127.0.0.1:35222
I20260709 13:01:10.525707  1579 tablet_copy_service.cc:434] P 3f64b76180a54492bb015e405caeacec: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-8a4760a45328493e87523c2ee8292599 on tablet 8a4760a45328493e87523c2ee8292599 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:10.542083  2304 raft_consensus.cc:1217] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:10.724278  2304 raft_consensus.cc:1217] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:01:10.939057  2304 raft_consensus.cc:1217] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:10.959358  2304 raft_consensus.cc:1217] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:11.119172  2599 raft_consensus.cc:1064] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:11.121942  2599 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:11.130511  2304 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:11.131383  1709 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:11.131752  1640 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:11.132089  2614 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.133519  2526 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.139209  2614 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:11.178014  1643 raft_consensus.cc:2955] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.177129  2599 raft_consensus.cc:2955] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.185961  1714 raft_consensus.cc:2955] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.191756  2633 raft_consensus.cc:1064] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:11.194479  2633 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:11.216909  2305 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:11.216538  2304 raft_consensus.cc:2955] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.218567  2632 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.225603  1643 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:11.227178  2614 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.232621  1464 catalog_manager.cc:5697] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.237090  1709 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:11.241508  2632 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.274534  2633 raft_consensus.cc:2955] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.277858  1714 raft_consensus.cc:2955] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.291149  1643 raft_consensus.cc:2955] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.302156  2305 raft_consensus.cc:2955] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.325021  1568 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:11.333045  2305 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:11.334342  1710 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:11.336335  2638 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:11.335274  1465 catalog_manager.cc:5697] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.338951  2526 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:11.354394  2614 raft_consensus.cc:2955] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.369918  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4594cfed297144b39e1a29473c9707d9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:11.369571  2304 raft_consensus.cc:2955] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.373735  1565 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:11.380285  2303 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:11.384459  2599 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:11.387581  1643 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:11.391296  2633 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:11.391386  1714 raft_consensus.cc:2955] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.402284  2303 raft_consensus.cc:2955] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.398329  2599 raft_consensus.cc:2955] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.416637  1643 raft_consensus.cc:2955] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.423493  1466 catalog_manager.cc:5697] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.439273  1465 catalog_manager.cc:5697] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.443328  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 408b4ec829c54266b41334aac8245405 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:11.451579  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 4594cfed297144b39e1a29473c9707d9 with delete_type TABLET_DATA_TOMBSTONED (TS 5184c101be204393adf3ac8b31f25ad6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60370
I20260709 13:01:11.463722  1699 tablet_service.cc:1558] Processing DeleteTablet for tablet 408b4ec829c54266b41334aac8245405 with delete_type TABLET_DATA_TOMBSTONED (TS 28bc59d37f664ce0a6f5784f2619f0b5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36054
I20260709 13:01:11.467974  2646 tablet_replica.cc:333] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:01:11.469238  2646 raft_consensus.cc:2243] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:11.470082  2646 raft_consensus.cc:2272] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:11.473665  2646 ts_tablet_manager.cc:1916] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:11.475234  2630 raft_consensus.cc:1064] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:11.478035  2630 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:11.490597  2646 ts_tablet_manager.cc:1929] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:11.491174  2646 log.cc:1199] T 4594cfed297144b39e1a29473c9707d9 P 5184c101be204393adf3ac8b31f25ad6: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/wals/4594cfed297144b39e1a29473c9707d9
I20260709 13:01:11.493880  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet 4594cfed297144b39e1a29473c9707d9 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:11.495703  2647 tablet_replica.cc:333] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:01:11.500247  1640 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:11.500949  2647 raft_consensus.cc:2243] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:11.500068  2303 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:11.501847  2647 raft_consensus.cc:2272] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:11.502130  2542 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.502439  1565 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:11.503937  2589 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.504725  2539 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.507134  2647 ts_tablet_manager.cc:1916] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:11.544066  2647 ts_tablet_manager.cc:1929] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:01:11.544600  2647 log.cc:1199] T 408b4ec829c54266b41334aac8245405 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/wals/408b4ec829c54266b41334aac8245405
I20260709 13:01:11.544821  2630 raft_consensus.cc:2955] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.546344  1451 catalog_manager.cc:5028] TS 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): tablet 408b4ec829c54266b41334aac8245405 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:11.546998  2304 raft_consensus.cc:2955] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.552282  2633 raft_consensus.cc:1064] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:11.551610  1568 raft_consensus.cc:2955] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.557863  2633 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:11.564464  1640 raft_consensus.cc:2955] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.580295  1714 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:11.580001  1467 catalog_manager.cc:5697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.582466  2526 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.587061  1644 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:11.591792  2526 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:11.594173  2304 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:11.607515  2633 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:11.610186  1710 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:11.618185  2303 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:11.618183  1568 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:11.625432  2589 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:11.629035  2589 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:01:11.676806  2303 raft_consensus.cc:2955] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.678004  2630 raft_consensus.cc:2955] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:01:11.679937  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 72984us
I20260709 13:01:11.698237  2305 raft_consensus.cc:2955] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.705013  2599 raft_consensus.cc:2955] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.709987  1564 raft_consensus.cc:2955] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.724943  1714 raft_consensus.cc:2955] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.731226  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 213d9f9af66a4ebd8c9a7b0163196a57 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:11.741250  1464 catalog_manager.cc:5697] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.754770  1465 catalog_manager.cc:5697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.752508  1644 raft_consensus.cc:2955] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.777515  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 213d9f9af66a4ebd8c9a7b0163196a57 with delete_type TABLET_DATA_TOMBSTONED (TS 5184c101be204393adf3ac8b31f25ad6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60370
I20260709 13:01:11.778597  2646 tablet_replica.cc:333] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:01:11.780237  2646 raft_consensus.cc:2243] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:11.781116  2646 raft_consensus.cc:2272] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:11.784593  2646 ts_tablet_manager.cc:1916] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:11.800864  1568 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:11.807603  2305 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:11.809513  2599 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:11.811486  1644 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:11.815006  2599 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:11.815660  2646 ts_tablet_manager.cc:1929] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:11.818702  2646 log.cc:1199] T 213d9f9af66a4ebd8c9a7b0163196a57 P 5184c101be204393adf3ac8b31f25ad6: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/wals/213d9f9af66a4ebd8c9a7b0163196a57
I20260709 13:01:11.818302  2526 raft_consensus.cc:2955] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.820580  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet 213d9f9af66a4ebd8c9a7b0163196a57 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:11.824717  2305 raft_consensus.cc:2955] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.825605  1644 raft_consensus.cc:2955] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.828927  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8a4760a45328493e87523c2ee8292599 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:11.839488  1463 catalog_manager.cc:5697] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:11.856992  1699 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a4760a45328493e87523c2ee8292599 with delete_type TABLET_DATA_TOMBSTONED (TS 28bc59d37f664ce0a6f5784f2619f0b5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36054
I20260709 13:01:11.857925  2647 tablet_replica.cc:333] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:01:11.859074  2647 raft_consensus.cc:2243] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:11.859810  2647 raft_consensus.cc:2272] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:11.864341  2647 ts_tablet_manager.cc:1916] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:11.880142  2647 ts_tablet_manager.cc:1929] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:11.880537  2647 log.cc:1199] T 8a4760a45328493e87523c2ee8292599 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/wals/8a4760a45328493e87523c2ee8292599
I20260709 13:01:11.881827  1451 catalog_manager.cc:5028] TS 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): tablet 8a4760a45328493e87523c2ee8292599 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:12.938674  1714 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:12.946986  1644 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:12.947506  1683 consensus_peers.cc:597] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 344c88e0be9640f583140fb4aab3af58. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:12.947008  1568 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:12.948428  2630 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:12.951440  2663 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:12.978331  2630 raft_consensus.cc:2955] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:12.982106  1644 raft_consensus.cc:2955] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:12.989390  1568 raft_consensus.cc:2955] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:12.993306  1640 consensus_queue.cc:237] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:13.001055  1466 catalog_manager.cc:5697] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:13.023763  1713 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:13.025333  2593 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:13.026197  1609 consensus_peers.cc:597] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 50d8055bc87c4624a96f13638fe667e6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:13.027360  1568 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:13.028792  2593 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.042169  2659 raft_consensus.cc:2955] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:13.043828  1568 raft_consensus.cc:2955] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:13.060967  1713 raft_consensus.cc:2955] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:13.062301  1640 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:13.076687  1466 catalog_manager.cc:5697] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:13.098425  1609 consensus_peers.cc:597] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 33657ac7590d4739b9bda3f7bb345c7b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:13.099820  1713 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:13.100600  1568 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:13.101269  2659 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.103052  2674 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.111878  2674 raft_consensus.cc:2955] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:13.113328  1713 raft_consensus.cc:2955] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:13.113533  1568 raft_consensus.cc:2955] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:13.130425  1466 catalog_manager.cc:5697] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:13.131182  1568 consensus_queue.cc:237] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:13.141347  1713 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.143154  1640 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:13.146273  2638 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:13.153254  1534 consensus_peers.cc:597] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 768b897f18a644e8b1e8f8ed1c8b616f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:13.153936  2633 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:13.167629  2638 raft_consensus.cc:2955] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:13.176456  1713 raft_consensus.cc:2955] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:13.176714  1640 raft_consensus.cc:2955] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:13.197882  1465 catalog_manager.cc:5697] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:13.488092  2681 ts_tablet_manager.cc:933] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:13.492362  2681 tablet_copy_client.cc:323] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:40023
I20260709 13:01:13.493921  2683 ts_tablet_manager.cc:933] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:13.494658  1654 tablet_copy_service.cc:140] P 5184c101be204393adf3ac8b31f25ad6: Received BeginTabletCopySession request for tablet 50d8055bc87c4624a96f13638fe667e6 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:53222)
I20260709 13:01:13.495241  1654 tablet_copy_service.cc:161] P 5184c101be204393adf3ac8b31f25ad6: Beginning new tablet copy session on tablet 50d8055bc87c4624a96f13638fe667e6 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:53222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-50d8055bc87c4624a96f13638fe667e6
I20260709 13:01:13.496909  2683 tablet_copy_client.cc:323] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:13.498615  1727 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet 344c88e0be9640f583140fb4aab3af58 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:13.499151  1727 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet 344c88e0be9640f583140fb4aab3af58 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-344c88e0be9640f583140fb4aab3af58
I20260709 13:01:13.501672  1654 tablet_copy_source_session.cc:215] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:13.504969  2681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50d8055bc87c4624a96f13638fe667e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:13.506770  1727 tablet_copy_source_session.cc:215] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:13.510648  2683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 344c88e0be9640f583140fb4aab3af58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:13.525453  2681 tablet_copy_client.cc:806] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:13.526135  2681 tablet_copy_client.cc:670] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:13.530360  2681 tablet_copy_client.cc:538] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:13.546909  2683 tablet_copy_client.cc:806] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:13.548594  2683 tablet_copy_client.cc:670] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:13.549911  2685 ts_tablet_manager.cc:933] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:13.555845  2683 tablet_copy_client.cc:538] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:13.556443  2681 tablet_bootstrap.cc:492] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:13.563705  2685 tablet_copy_client.cc:323] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:40023
I20260709 13:01:13.611202  1654 tablet_copy_service.cc:140] P 5184c101be204393adf3ac8b31f25ad6: Received BeginTabletCopySession request for tablet 33657ac7590d4739b9bda3f7bb345c7b from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:53222)
I20260709 13:01:13.611756  1654 tablet_copy_service.cc:161] P 5184c101be204393adf3ac8b31f25ad6: Beginning new tablet copy session on tablet 33657ac7590d4739b9bda3f7bb345c7b from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:53222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-33657ac7590d4739b9bda3f7bb345c7b
I20260709 13:01:13.619714  1654 tablet_copy_source_session.cc:215] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:13.629861  2685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33657ac7590d4739b9bda3f7bb345c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:13.634934  2683 tablet_bootstrap.cc:492] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:13.680034  2692 ts_tablet_manager.cc:933] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
I20260709 13:01:13.716681  2681 tablet_bootstrap.cc:492] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:13.717913  2681 tablet_bootstrap.cc:492] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:13.718797  2681 ts_tablet_manager.cc:1403] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.163s	user 0.022s	sys 0.023s
I20260709 13:01:13.733661  2692 tablet_copy_client.cc:323] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:36547
I20260709 13:01:13.735805  1579 tablet_copy_service.cc:140] P 3f64b76180a54492bb015e405caeacec: Received BeginTabletCopySession request for tablet 768b897f18a644e8b1e8f8ed1c8b616f from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:35222)
I20260709 13:01:13.736349  1579 tablet_copy_service.cc:161] P 3f64b76180a54492bb015e405caeacec: Beginning new tablet copy session on tablet 768b897f18a644e8b1e8f8ed1c8b616f from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:35222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-768b897f18a644e8b1e8f8ed1c8b616f
I20260709 13:01:13.748024  1579 tablet_copy_source_session.cc:215] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:13.750950  2681 raft_consensus.cc:359] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:13.752571  2681 raft_consensus.cc:740] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:13.753747  2685 tablet_copy_client.cc:806] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:13.754077  2692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 768b897f18a644e8b1e8f8ed1c8b616f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:13.753494  2681 consensus_queue.cc:260] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:13.754438  2685 tablet_copy_client.cc:670] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:13.759498  2685 tablet_copy_client.cc:538] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:13.765695  2683 tablet_bootstrap.cc:492] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:13.768018  2683 tablet_bootstrap.cc:492] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:13.769316  2683 ts_tablet_manager.cc:1403] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.135s	user 0.017s	sys 0.021s
I20260709 13:01:13.770792  2681 ts_tablet_manager.cc:1434] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.006s
I20260709 13:01:13.770888  2692 tablet_copy_client.cc:806] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:13.771867  2692 tablet_copy_client.cc:670] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:13.773554  2683 raft_consensus.cc:359] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:13.774487  2683 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:13.775537  2683 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:13.777745  2692 tablet_copy_client.cc:538] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:13.780059  2685 tablet_bootstrap.cc:492] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:13.797644  2692 tablet_bootstrap.cc:492] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:13.800705  1654 tablet_copy_service.cc:342] P 5184c101be204393adf3ac8b31f25ad6: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-50d8055bc87c4624a96f13638fe667e6 received from {username='slave'} at 127.0.0.1:53222
I20260709 13:01:13.808694  1654 tablet_copy_service.cc:434] P 5184c101be204393adf3ac8b31f25ad6: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-50d8055bc87c4624a96f13638fe667e6 on tablet 50d8055bc87c4624a96f13638fe667e6 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:13.814999  2683 ts_tablet_manager.cc:1434] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.044s	user 0.011s	sys 0.007s
I20260709 13:01:13.817449  1727 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-344c88e0be9640f583140fb4aab3af58 received from {username='slave'} at 127.0.0.1:33280
I20260709 13:01:13.817929  1727 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-344c88e0be9640f583140fb4aab3af58 on tablet 344c88e0be9640f583140fb4aab3af58 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:13.845305  2692 tablet_bootstrap.cc:492] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:13.846396  2685 tablet_bootstrap.cc:492] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:13.856473  2685 tablet_bootstrap.cc:492] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:13.857223  2685 ts_tablet_manager.cc:1403] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.077s	user 0.022s	sys 0.002s
I20260709 13:01:13.860402  2685 raft_consensus.cc:359] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:13.846463  2692 tablet_bootstrap.cc:492] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:13.861366  2685 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:13.861990  2692 ts_tablet_manager.cc:1403] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.065s	user 0.025s	sys 0.000s
I20260709 13:01:13.862088  2685 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:13.865443  2685 ts_tablet_manager.cc:1434] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:01:13.865083  2692 raft_consensus.cc:359] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:13.866652  2692 raft_consensus.cc:740] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:13.867838  1654 tablet_copy_service.cc:342] P 5184c101be204393adf3ac8b31f25ad6: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-33657ac7590d4739b9bda3f7bb345c7b received from {username='slave'} at 127.0.0.1:53222
I20260709 13:01:13.867415  2692 consensus_queue.cc:260] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:13.868358  1654 tablet_copy_service.cc:434] P 5184c101be204393adf3ac8b31f25ad6: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-33657ac7590d4739b9bda3f7bb345c7b on tablet 33657ac7590d4739b9bda3f7bb345c7b with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:13.870565  2692 ts_tablet_manager.cc:1434] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:01:13.872548  1579 tablet_copy_service.cc:342] P 3f64b76180a54492bb015e405caeacec: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-768b897f18a644e8b1e8f8ed1c8b616f received from {username='slave'} at 127.0.0.1:35222
I20260709 13:01:13.872972  1579 tablet_copy_service.cc:434] P 3f64b76180a54492bb015e405caeacec: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-768b897f18a644e8b1e8f8ed1c8b616f on tablet 768b897f18a644e8b1e8f8ed1c8b616f with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:13.970100  2305 raft_consensus.cc:1217] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:14.005687  2305 raft_consensus.cc:1217] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:14.047775  2305 raft_consensus.cc:1217] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:14.179420  2305 raft_consensus.cc:1217] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:01:14.570406  2659 raft_consensus.cc:1064] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:14.573112  2659 consensus_queue.cc:237] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:14.582280  2691 raft_consensus.cc:1064] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:14.584914  2691 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:14.590175  2305 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:14.592084  2593 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:14.598775  1714 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:14.604967  1568 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:14.610754  2688 raft_consensus.cc:1064] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:14.613335  2688 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:14.620574  2593 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:01:14.624428  2690 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:14.632047  1709 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:14.634105  2691 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:14.638269  2305 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:14.640098  2705 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:14.643940  2303 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:14.645421  1644 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:14.647218  1565 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:14.647635  2661 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:14.649575  2661 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:14.650274  1565 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:14.651121  2659 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:14.655081  2689 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:14.666051  2638 raft_consensus.cc:1064] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:14.668762  2638 consensus_queue.cc:237] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:14.669750  2705 raft_consensus.cc:2955] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:14.675288  1714 raft_consensus.cc:2955] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:14.701550  2689 raft_consensus.cc:2955] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:14.713785  2659 raft_consensus.cc:2955] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:14.722880  1463 catalog_manager.cc:5697] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:14.731954  1564 raft_consensus.cc:2955] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:14.743566  2305 raft_consensus.cc:2955] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:14.741439  1644 raft_consensus.cc:2955] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:14.756898  1714 raft_consensus.cc:2955] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:14.785678  2304 raft_consensus.cc:2955] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:14.793799  2303 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:14.795056  1713 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:14.801682  1643 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:14.811189  2303 raft_consensus.cc:2955] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:14.826503  2678 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:01:14.877816  1465 catalog_manager.cc:5697] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:01:14.905572  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 74661.3us
W20260709 13:01:14.905467  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 88337.9us
I20260709 13:01:14.917194  2720 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 13:01:14.957798  2633 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:01:14.970825  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 110852us
I20260709 13:01:15.070853  2727 raft_consensus.cc:2955] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.077698  2304 raft_consensus.cc:2955] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:01:15.106525  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 277917us
I20260709 13:01:15.106190  1644 raft_consensus.cc:2955] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:01:15.108368  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 191711us
I20260709 13:01:15.115285  1564 raft_consensus.cc:2955] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.125414  1709 raft_consensus.cc:2955] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.186848  1567 raft_consensus.cc:2955] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.207122  1465 catalog_manager.cc:5697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.208702  1465 catalog_manager.cc:5697] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.252866  1643 consensus_queue.cc:237] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:15.257386  1710 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:15.257345  1644 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:15.268688  1642 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:15.270991  2717 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:15.275679  2304 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:15.276670  1564 consensus_queue.cc:237] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:15.278502  2717 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:15.279908  2304 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:15.283416  2690 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:15.286365  2733 raft_consensus.cc:2955] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.294446  1568 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:15.297534  2735 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:15.308550  1709 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:15.295277  2302 raft_consensus.cc:2955] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.312326  2304 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:15.294377  1642 raft_consensus.cc:2955] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.317540  2659 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:15.319617  2659 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:15.323313  2303 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:15.323017  2726 raft_consensus.cc:2955] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.325053  2633 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:15.329316  2303 raft_consensus.cc:2955] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.330598  1710 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:15.332314  2633 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:15.335296  2727 raft_consensus.cc:2955] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.338798  1466 catalog_manager.cc:5697] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.340385  2691 raft_consensus.cc:2955] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.341122  2304 raft_consensus.cc:2955] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.342571  1709 raft_consensus.cc:2955] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.350346  2305 raft_consensus.cc:2955] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.356683  1452 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 33657ac7590d4739b9bda3f7bb345c7b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:15.363583  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 344c88e0be9640f583140fb4aab3af58 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:15.389566  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 768b897f18a644e8b1e8f8ed1c8b616f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:15.393338  1452 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 50d8055bc87c4624a96f13638fe667e6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:15.394220  1465 catalog_manager.cc:5697] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:15.407738  1554 tablet_service.cc:1558] Processing DeleteTablet for tablet 344c88e0be9640f583140fb4aab3af58 with delete_type TABLET_DATA_TOMBSTONED (TS 3f64b76180a54492bb015e405caeacec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41262
I20260709 13:01:15.390516  1564 raft_consensus.cc:2955] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.414706  1709 raft_consensus.cc:2955] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.443378  2743 tablet_replica.cc:333] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:01:15.444666  2743 raft_consensus.cc:2243] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.447153  1463 catalog_manager.cc:5697] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.452626  2743 raft_consensus.cc:2272] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.452852  1463 catalog_manager.cc:5697] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:15.448547  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 768b897f18a644e8b1e8f8ed1c8b616f with delete_type TABLET_DATA_TOMBSTONED (TS 5184c101be204393adf3ac8b31f25ad6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60370
I20260709 13:01:15.459236  2743 ts_tablet_manager.cc:1916] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:15.459553  2744 tablet_replica.cc:333] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:01:15.466446  2744 raft_consensus.cc:2243] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.467276  2744 raft_consensus.cc:2272] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.468448  1554 tablet_service.cc:1558] Processing DeleteTablet for tablet 50d8055bc87c4624a96f13638fe667e6 with delete_type TABLET_DATA_TOMBSTONED (TS 3f64b76180a54492bb015e405caeacec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41262
I20260709 13:01:15.470028  1699 tablet_service.cc:1558] Processing DeleteTablet for tablet 33657ac7590d4739b9bda3f7bb345c7b with delete_type TABLET_DATA_TOMBSTONED (TS 28bc59d37f664ce0a6f5784f2619f0b5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36054
I20260709 13:01:15.475677  2744 ts_tablet_manager.cc:1916] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:15.484575  2746 tablet_replica.cc:333] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:01:15.485703  2746 raft_consensus.cc:2243] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.486402  2746 raft_consensus.cc:2272] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.489068  2743 ts_tablet_manager.cc:1929] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:15.489545  2743 log.cc:1199] T 344c88e0be9640f583140fb4aab3af58 P 3f64b76180a54492bb015e405caeacec: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/wals/344c88e0be9640f583140fb4aab3af58
I20260709 13:01:15.491444  2743 tablet_replica.cc:333] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:01:15.492180  2746 ts_tablet_manager.cc:1916] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:15.492515  2743 raft_consensus.cc:2243] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:15.493346  2743 raft_consensus.cc:2272] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:15.497956  2743 ts_tablet_manager.cc:1916] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:15.500576  2744 ts_tablet_manager.cc:1929] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:01:15.501258  2744 log.cc:1199] T 768b897f18a644e8b1e8f8ed1c8b616f P 5184c101be204393adf3ac8b31f25ad6: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/wals/768b897f18a644e8b1e8f8ed1c8b616f
I20260709 13:01:15.532173  1451 catalog_manager.cc:5028] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): tablet 344c88e0be9640f583140fb4aab3af58 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:15.618573  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet 768b897f18a644e8b1e8f8ed1c8b616f (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:15.685956  2746 ts_tablet_manager.cc:1929] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:15.701449  2746 log.cc:1199] T 33657ac7590d4739b9bda3f7bb345c7b P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/wals/33657ac7590d4739b9bda3f7bb345c7b
I20260709 13:01:15.714195  1451 catalog_manager.cc:5028] TS 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): tablet 33657ac7590d4739b9bda3f7bb345c7b (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:15.733194  2743 ts_tablet_manager.cc:1929] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:15.733803  2743 log.cc:1199] T 50d8055bc87c4624a96f13638fe667e6 P 3f64b76180a54492bb015e405caeacec: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/wals/50d8055bc87c4624a96f13638fe667e6
I20260709 13:01:15.752111  1451 catalog_manager.cc:5028] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): tablet 50d8055bc87c4624a96f13638fe667e6 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:15.767849   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:15.847757  1642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:01:15.848610  1642 raft_consensus.cc:606] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:15.864127  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:15.864684  1710 raft_consensus.cc:606] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:01:15.869081  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:15.869654  1710 raft_consensus.cc:606] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:15.871934  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:15.872498  1710 raft_consensus.cc:606] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:01:15.875113  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:15.875746  1710 raft_consensus.cc:606] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:01:15.878509  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:15.879104  1710 raft_consensus.cc:606] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:01:15.880782   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:01:15.881875   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:16.015161  2731 raft_consensus.cc:993] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:16.016033  2661 raft_consensus.cc:1081] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:16.018043  1565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:41274
I20260709 13:01:16.018734  1565 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:16.019164  1565 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.024793  2659 raft_consensus.cc:993] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:16.025581  2748 raft_consensus.cc:1081] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:16.026417  1565 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:16.027678  2303 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53446
I20260709 13:01:16.028264  2303 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:16.028612  2303 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.029439  1565 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:16.030747  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:16.031502  1710 raft_consensus.cc:3055] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:16.032135  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:16.032589  1710 raft_consensus.cc:740] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:16.033664  1710 consensus_queue.cc:260] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:16.032768  1642 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.037412  2303 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:16.041319  1642 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:16.034845  1710 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.043666  2303 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:16.043854  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:16.044524  1564 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.045724  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:16.046828  2678 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:16.047402  2678 raft_consensus.cc:697] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:16.048590  2678 consensus_queue.cc:237] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:16.050297  1710 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:16.051571  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:16.052330  1642 raft_consensus.cc:3055] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:16.052783  1642 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:16.053727  1642 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:16.056712  1642 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.058367  1564 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:16.059293  1565 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:16.070268  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:16.070607  1642 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:16.075966  2753 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:16.075461  1466 catalog_manager.cc:5697] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:16.087628  2753 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:16.088728  2753 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:16.104545  1464 catalog_manager.cc:5697] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 2, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:16.141780  2731 raft_consensus.cc:993] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:16.142202  2661 raft_consensus.cc:1081] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:16.143853  1564 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:41274
I20260709 13:01:16.144621  1564 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:16.145258  1564 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.148381  2661 raft_consensus.cc:993] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:16.149078  2732 raft_consensus.cc:1081] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:16.150666  1565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:41274
I20260709 13:01:16.151224  1565 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:16.151547  1565 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.152300  1564 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:16.155225  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:16.155273  1564 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:16.155483  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:16.155818  1642 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.156056  1710 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:16.156461  1710 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:16.157328  1710 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:16.158454  1710 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.160442  1565 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:16.165823  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:16.166446  1644 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.166841  1642 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:16.167726  1710 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:16.169869  1532 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:16.170755  2678 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:16.171378  2678 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:16.172394  1565 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:16.173153  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:16.173225  2678 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:16.173919  1710 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:16.174417  1710 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:16.175946  1710 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:16.176898  1644 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:16.177592  1710 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.178010  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:16.179105  2633 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:16.179687  2633 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:16.180697  2633 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:16.188299  1710 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:16.190181  1464 catalog_manager.cc:5697] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:16.206506  1464 catalog_manager.cc:5697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:16.220939  2661 raft_consensus.cc:993] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:16.222359  2731 raft_consensus.cc:1081] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:16.224160  1565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:41274
I20260709 13:01:16.224651  1565 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:16.224952  1565 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.229395  1565 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:16.230832  1565 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:16.231774  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:16.231782  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:16.232578  1710 raft_consensus.cc:3055] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:16.232573  1642 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.233022  1710 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:16.233867  1710 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:16.235102  1710 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.240309  1642 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:16.241386  1710 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:16.241775  1533 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:16.243765  2633 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:16.244486  2633 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:16.245332  2633 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:16.255478  1464 catalog_manager.cc:5697] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:16.336826  2731 raft_consensus.cc:993] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:16.337360  2732 raft_consensus.cc:1081] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:16.338994  2303 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53448
I20260709 13:01:16.339713  2303 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:16.340212  2303 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.345198  2303 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:16.347622  2303 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:16.348003  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:16.348202  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:16.348680  1642 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.348840  1710 raft_consensus.cc:3055] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:16.349275  1710 raft_consensus.cc:740] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:16.350101  1710 consensus_queue.cc:260] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:16.351246  1710 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:16.354084  1642 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:16.355260  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:16.356120  2753 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:16.356586  2753 raft_consensus.cc:697] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:16.357386  1710 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:16.357527  2753 consensus_queue.cc:237] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:16.367715  1464 catalog_manager.cc:5697] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:16.548218  1565 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:16.560890  1710 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:16.562227  1534 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 403bcba338a042c1b5fecfe8ac430922. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:16.561976  1642 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:16.563200  2633 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:16.564728  2638 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:16.577231  1709 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:16.579159  2633 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:16.584836  2638 raft_consensus.cc:2955] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:16.591907  1642 raft_consensus.cc:2955] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:16.609040  1709 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:16.613548  1564 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:16.614014  1710 raft_consensus.cc:2955] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:16.616271  2753 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:16.619831  1644 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:16.623651  2678 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:16.633608  1644 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:16.640273  2771 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:16.648528  1464 catalog_manager.cc:5697] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:16.664023  1568 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:16.665652  1644 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:16.673342  2661 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:16.685501  2732 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:16.696833  1683 consensus_peers.cc:597] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37b565cd27a64e96addb09389c2c7d5e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:16.706616  1709 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:16.708741  1640 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:16.708973  2771 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:16.710839  2753 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:16.717669  1643 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:16.719683  2769 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:16.725104  1644 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:16.727712  2771 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:16.727119  2661 raft_consensus.cc:2955] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:16.731724  1565 raft_consensus.cc:2955] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:16.744743  1713 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:16.747728  2769 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:16.771788  1642 raft_consensus.cc:2955] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:16.800760  1710 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:16.803086  1709 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:16.804991  2769 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:16.812925  2754 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:16.814586  1642 consensus_queue.cc:237] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:16.854387  1465 catalog_manager.cc:5697] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:16.888439   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:16.888928  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 55160.9us
I20260709 13:01:16.924012  1640 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:16.927168  2753 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:16.950379  1609 consensus_peers.cc:597] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 92a2b88a149f424591686a115bc1f0d6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:16.962304  1714 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:16.965839  1565 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:16.967363  2763 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:16.978812  2763 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:17.005183  2659 raft_consensus.cc:2955] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:17.007506  1565 raft_consensus.cc:2955] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:17.025250  1714 raft_consensus.cc:2955] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:17.027331  1709 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:17.027364  1464 catalog_manager.cc:5697] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:17.028119   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:17.051307  1565 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:17.051885  1683 consensus_peers.cc:597] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac4c23f0e4f742e38ca7a5ebd7778dfa. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:17.052597  1640 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:17.052944  2788 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:17.054554  2778 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:17.079388  2788 raft_consensus.cc:2955] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:17.084070  1640 raft_consensus.cc:2955] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:17.091122  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:17.091789  1709 raft_consensus.cc:606] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:17.094579  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:17.095184  1709 raft_consensus.cc:606] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:17.098287  1714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:17.098881  1714 raft_consensus.cc:606] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:01:17.102494  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "520f39ecc282462782487a3651e28b44"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:17.103116  1709 raft_consensus.cc:606] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:17.107097  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:17.107714  1709 raft_consensus.cc:606] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:17.121338  1466 catalog_manager.cc:5697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:17.149858  1565 raft_consensus.cc:2955] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:17.172580  2802 ts_tablet_manager.cc:933] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
I20260709 13:01:17.180807  2792 raft_consensus.cc:993] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:17.181401  2788 raft_consensus.cc:1081] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:17.183074  1644 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "520f39ecc282462782487a3651e28b44"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60390
I20260709 13:01:17.183662  1644 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:17.183987  1644 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.195005  1644 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:17.197103  1644 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:17.198331  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:17.199201  1565 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.199915  1564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:17.200639  1564 raft_consensus.cc:606] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:17.201858  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:17.202441  1709 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:17.202733  1709 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:17.203646  1709 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:17.205228  1709 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.209800  2802 tablet_copy_client.cc:323] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:36547
I20260709 13:01:17.212553  1709 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:17.214457  1565 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:17.214613  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:17.215121  1644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:01:17.215946  1644 raft_consensus.cc:606] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:17.217736   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:01:17.219107   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:17.219707  2748 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:17.220191  2748 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:17.221176  2748 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:17.221387  1579 tablet_copy_service.cc:140] P 3f64b76180a54492bb015e405caeacec: Received BeginTabletCopySession request for tablet 403bcba338a042c1b5fecfe8ac430922 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:35222)
I20260709 13:01:17.221980  1579 tablet_copy_service.cc:161] P 3f64b76180a54492bb015e405caeacec: Beginning new tablet copy session on tablet 403bcba338a042c1b5fecfe8ac430922 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:35222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-403bcba338a042c1b5fecfe8ac430922
I20260709 13:01:17.229494  2788 raft_consensus.cc:993] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:17.230326  2788 raft_consensus.cc:1081] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:17.231580  1579 tablet_copy_source_session.cc:215] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:17.232573  2303 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53448
I20260709 13:01:17.233171  2303 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:17.233551  2303 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.243057  2802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 403bcba338a042c1b5fecfe8ac430922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:17.245041  1464 catalog_manager.cc:5697] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:17.248931  2303 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:17.251106  2303 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:17.252727  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:17.253374  1565 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.259162  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:17.260443  1709 raft_consensus.cc:3055] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:17.260812  1709 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:17.261526  2802 tablet_copy_client.cc:806] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:17.262185  2802 tablet_copy_client.cc:670] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:17.263806  1709 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:17.267472  1709 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.267870  2802 tablet_copy_client.cc:538] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:17.272680  1565 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:17.274127  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:17.275112  2753 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:17.275609  2753 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:17.276137  1709 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:17.276521  2753 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:17.281294  2802 tablet_bootstrap.cc:492] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:17.292398  1464 catalog_manager.cc:5697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:17.306645  2802 tablet_bootstrap.cc:492] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:17.307636  2802 tablet_bootstrap.cc:492] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:17.308517  2802 ts_tablet_manager.cc:1403] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.027s	user 0.022s	sys 0.005s
I20260709 13:01:17.315214  2807 ts_tablet_manager.cc:933] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:17.318763  2802 raft_consensus.cc:359] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:17.319875  2802 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:17.320947  2802 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:17.325655  2802 ts_tablet_manager.cc:1434] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:01:17.328243  1579 tablet_copy_service.cc:342] P 3f64b76180a54492bb015e405caeacec: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-403bcba338a042c1b5fecfe8ac430922 received from {username='slave'} at 127.0.0.1:35222
I20260709 13:01:17.328971  1579 tablet_copy_service.cc:434] P 3f64b76180a54492bb015e405caeacec: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-403bcba338a042c1b5fecfe8ac430922 on tablet 403bcba338a042c1b5fecfe8ac430922 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:17.334204  2807 tablet_copy_client.cc:323] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:17.337991  1727 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet 37b565cd27a64e96addb09389c2c7d5e from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:17.338631  1727 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet 37b565cd27a64e96addb09389c2c7d5e from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-37b565cd27a64e96addb09389c2c7d5e
I20260709 13:01:17.352501  1727 tablet_copy_source_session.cc:215] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:17.356391  2807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37b565cd27a64e96addb09389c2c7d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:17.366870  2661 raft_consensus.cc:993] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:17.367522  2661 raft_consensus.cc:1081] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:17.369694  1565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:41274
I20260709 13:01:17.370507  1565 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:17.370898  1565 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.377038  2769 raft_consensus.cc:993] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:17.377635  2638 raft_consensus.cc:1081] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Signalling peer 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:17.378844  2748 raft_consensus.cc:993] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:17.379424  2791 raft_consensus.cc:1081] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:17.379860  1709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:36066
I20260709 13:01:17.380435  1709 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:17.380765  1709 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.383399  2303 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53446
I20260709 13:01:17.383973  1565 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:17.384016  2303 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:17.387013  2303 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.387933  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:17.388595  1713 raft_consensus.cc:3055] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:17.388931  1713 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:17.389741  1713 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:17.391800  1713 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.398988  2303 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:17.399518  2807 tablet_copy_client.cc:806] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:17.400101  2807 tablet_copy_client.cc:670] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:17.402746  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:17.403393  1714 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.405887  1713 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:17.407588  1532 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:17.408593  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:17.409276  1644 raft_consensus.cc:3055] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:17.409575  1644 raft_consensus.cc:740] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:17.410344  1644 consensus_queue.cc:260] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:17.391986  1565 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:17.394187  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:17.412470  1640 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.411590  1644 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.417768  2778 raft_consensus.cc:993] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:17.418386  2792 raft_consensus.cc:1081] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:17.418697  2769 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:17.422343  2302 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53448
I20260709 13:01:17.423120  1640 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:17.422405  2769 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:17.423606  2302 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:17.424033  2302 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.424467  2769 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:17.391153  1709 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:17.428387  2303 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:17.431427  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:17.432017  1565 raft_consensus.cc:3055] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:17.422823  2807 tablet_copy_client.cc:538] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:17.432721  1565 raft_consensus.cc:740] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:17.434544  1565 consensus_queue.cc:260] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:17.435954  1565 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.438006  2302 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:17.411561  1714 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:17.440611  2268 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:17.441437  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:17.441857  2754 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:17.442044  1642 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.442417  2754 raft_consensus.cc:697] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:17.443408  2754 consensus_queue.cc:237] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:17.450134  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:17.450805  1714 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:17.451202  1714 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:17.452140  1714 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:17.420571  1644 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:17.440693  2302 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:17.457944  1714 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.450553  1642 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:17.464715  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:17.465610  2753 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:17.466872  1464 catalog_manager.cc:5697] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:17.469218  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:17.469811  1642 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.465191  1565 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:17.473651  2802 ts_tablet_manager.cc:933] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:17.474701  1681 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:17.476147  2661 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:17.476775  2661 raft_consensus.cc:697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:17.477849  2661 consensus_queue.cc:237] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:17.467528  1709 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:17.498586  2802 tablet_copy_client.cc:323] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:40023
I20260709 13:01:17.500782  1654 tablet_copy_service.cc:140] P 5184c101be204393adf3ac8b31f25ad6: Received BeginTabletCopySession request for tablet 92a2b88a149f424591686a115bc1f0d6 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:53222)
I20260709 13:01:17.501361  1654 tablet_copy_service.cc:161] P 5184c101be204393adf3ac8b31f25ad6: Beginning new tablet copy session on tablet 92a2b88a149f424591686a115bc1f0d6 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:53222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-92a2b88a149f424591686a115bc1f0d6
I20260709 13:01:17.500609  1464 catalog_manager.cc:5697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 2, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:17.503865  2753 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:17.505019  2753 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:17.515769  1654 tablet_copy_source_session.cc:215] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:17.519658  2802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92a2b88a149f424591686a115bc1f0d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:17.526882  1714 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:17.530210  2661 raft_consensus.cc:993] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:17.530730  2661 raft_consensus.cc:1081] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:17.532696  2303 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53448
I20260709 13:01:17.533152  2807 tablet_bootstrap.cc:492] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:17.533416  2303 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:17.533839  2303 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.526454  1464 catalog_manager.cc:5697] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 2, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:17.545358  2303 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:17.546658  1642 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:17.548970  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:17.550318  2303 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:17.550604  1565 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.552488  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:17.553187  1714 raft_consensus.cc:3055] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:17.553555  1714 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:17.554399  1714 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:17.557976  1464 catalog_manager.cc:5697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:17.560702  1565 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:17.555815  1714 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:17.565754  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:17.566640  2753 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:17.567178  2753 raft_consensus.cc:697] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:17.568122  2753 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:17.582100  1714 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:17.587922  2802 tablet_copy_client.cc:806] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:17.588660  2802 tablet_copy_client.cc:670] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:17.588045  2807 tablet_bootstrap.cc:492] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:17.596155  2807 tablet_bootstrap.cc:492] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:17.596997  2807 ts_tablet_manager.cc:1403] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.064s	user 0.027s	sys 0.004s
I20260709 13:01:17.596534  1464 catalog_manager.cc:5697] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:17.599807  2802 tablet_copy_client.cc:538] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:17.600509  2807 raft_consensus.cc:359] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:17.605942  2807 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:17.606745  2807 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:17.623935  2807 ts_tablet_manager.cc:1434] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.002s
I20260709 13:01:17.623935  2818 ts_tablet_manager.cc:933] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:17.626607  2802 tablet_bootstrap.cc:492] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:17.634609  1727 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-37b565cd27a64e96addb09389c2c7d5e received from {username='slave'} at 127.0.0.1:33280
I20260709 13:01:17.635200  1727 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-37b565cd27a64e96addb09389c2c7d5e on tablet 37b565cd27a64e96addb09389c2c7d5e with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:17.637905  2818 tablet_copy_client.cc:323] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:17.640090  1727 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet ac4c23f0e4f742e38ca7a5ebd7778dfa from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:17.640667  1727 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet ac4c23f0e4f742e38ca7a5ebd7778dfa from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-ac4c23f0e4f742e38ca7a5ebd7778dfa
I20260709 13:01:17.648036  2303 raft_consensus.cc:1217] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:17.649454  1727 tablet_copy_source_session.cc:215] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:17.653743  2818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac4c23f0e4f742e38ca7a5ebd7778dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:17.671572  1714 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:17.673954  2753 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:17.715567  1565 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:17.717214  2753 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:17.731371  2818 tablet_copy_client.cc:806] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:17.732112  2818 tablet_copy_client.cc:670] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:17.742559  2802 tablet_bootstrap.cc:492] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:17.743975  2818 tablet_copy_client.cc:538] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:17.743958  2802 tablet_bootstrap.cc:492] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:17.757998  2802 ts_tablet_manager.cc:1403] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.132s	user 0.016s	sys 0.040s
I20260709 13:01:17.760767  1714 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:17.762332  2763 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:17.765405  2818 tablet_bootstrap.cc:492] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:17.772693  2802 raft_consensus.cc:359] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:17.773609  2802 raft_consensus.cc:740] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:17.774250  2802 consensus_queue.cc:260] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:17.783151  2802 ts_tablet_manager.cc:1434] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:01:17.789630  1654 tablet_copy_service.cc:342] P 5184c101be204393adf3ac8b31f25ad6: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-92a2b88a149f424591686a115bc1f0d6 received from {username='slave'} at 127.0.0.1:53222
I20260709 13:01:17.790275  1654 tablet_copy_service.cc:434] P 5184c101be204393adf3ac8b31f25ad6: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-92a2b88a149f424591686a115bc1f0d6 on tablet 92a2b88a149f424591686a115bc1f0d6 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:17.808267  1565 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:17.811995  2748 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:17.817174  2818 tablet_bootstrap.cc:492] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:17.818233  2818 tablet_bootstrap.cc:492] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:17.819011  2818 ts_tablet_manager.cc:1403] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.054s	user 0.024s	sys 0.005s
I20260709 13:01:17.822690  2818 raft_consensus.cc:359] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:17.823854  2818 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:17.824781  2818 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:17.827782  2818 ts_tablet_manager.cc:1434] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.006s
I20260709 13:01:17.829205  1714 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:17.834457  1727 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-ac4c23f0e4f742e38ca7a5ebd7778dfa received from {username='slave'} at 127.0.0.1:33280
I20260709 13:01:17.835134  1727 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-ac4c23f0e4f742e38ca7a5ebd7778dfa on tablet ac4c23f0e4f742e38ca7a5ebd7778dfa with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:17.834419  2753 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:17.849572  1640 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:17.851197  2768 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:17.865957  1642 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:17.867882  2754 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:17.865957  1709 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:17.885883  2303 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Advancing to term 2
I20260709 13:01:17.890877  2772 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:17.900660  2303 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:17.902261  2772 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:17.935958  1709 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:17.942261  2753 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:17.946444  2769 raft_consensus.cc:1064] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:17.952335  2769 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:17.978223  2302 raft_consensus.cc:1217] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:17.983142  2303 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:17.987068  2768 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:17.999557  1640 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:18.007591  1565 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:18.009346  2788 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:18.019301  2303 raft_consensus.cc:1217] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:18.007673  2768 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:01:18.043782  1713 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:18.045226  1640 raft_consensus.cc:2955] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:18.046288  2772 raft_consensus.cc:2955] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:01:18.064323  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 50618.6us
I20260709 13:01:18.077144  1642 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:18.084481  2788 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:18.092880  1644 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:18.094735  2753 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:18.112931  1713 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:18.168596  2768 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:18.231678   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:18.134605  2753 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
W20260709 13:01:18.245400  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 69801.6us
W20260709 13:01:18.240872  2831 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:01:18.314112  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 55730.6us
W20260709 13:01:18.332140  2837 log.cc:927] Time spent T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:01:18.334779  1568 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:18.339299  2753 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:18.349287  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 226539us
I20260709 13:01:18.388978  1713 raft_consensus.cc:2955] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:18.414142  2303 raft_consensus.cc:2955] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:18.460568  2769 raft_consensus.cc:1064] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:18.460641  1463 catalog_manager.cc:5697] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 2 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:18.464118  2769 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:18.472798  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 150860us
I20260709 13:01:18.498131  2778 raft_consensus.cc:1064] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:18.501492  2778 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:18.524549  2853 raft_consensus.cc:1064] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:18.525118  1640 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:18.527333  2853 consensus_queue.cc:237] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:18.530804  2305 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:18.532955  2661 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:18.535004  2303 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:18.537235  1713 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:18.537068  1640 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:18.542773  2788 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:18.537742  2769 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:18.544041  2768 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:18.545882  2772 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:18.546414  1568 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:18.549607  2661 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:18.551848  2305 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:18.553375  2748 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:18.577593  1565 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:18.579490  2850 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:01:18.596617  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 46376.1us
I20260709 13:01:18.606891  1709 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:18.707535  2852 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:01:18.747886   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:18.751350  1640 raft_consensus.cc:2955] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:18.754956  2778 raft_consensus.cc:2955] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:18.757131  1643 raft_consensus.cc:2955] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:18.768846  1713 raft_consensus.cc:2955] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:18.767311  2769 raft_consensus.cc:2955] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:18.844882  1565 raft_consensus.cc:2955] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:18.854426  1464 catalog_manager.cc:5697] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:18.936067  1643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:01:18.936661  1643 raft_consensus.cc:606] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:01:18.946902  2305 raft_consensus.cc:2955] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:01:18.970441  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 50589us
I20260709 13:01:18.985292  1565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:18.985960  1565 raft_consensus.cc:606] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:18.991206  2871 raft_consensus.cc:2955] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.044134  1567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a4760a45328493e87523c2ee8292599"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:19.044860  1567 raft_consensus.cc:606] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:19.024803  1462 catalog_manager.cc:5697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:01:19.053081  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 98274.2us
I20260709 13:01:19.079900  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:19.080588  1710 raft_consensus.cc:606] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:19.089972  1565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:19.090874  1565 raft_consensus.cc:606] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:19.095847  1565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:19.095587  2302 raft_consensus.cc:2955] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.096832  1565 raft_consensus.cc:606] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:19.098484   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:01:19.100138   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:19.103127  2876 raft_consensus.cc:993] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: : Instructing follower 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:19.110680  2874 raft_consensus.cc:993] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:19.113591  2874 raft_consensus.cc:1081] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:19.118750  2871 raft_consensus.cc:1081] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Signalling peer 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:19.118170  1714 raft_consensus.cc:2955] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.133916  2303 raft_consensus.cc:2955] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.137359  2305 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a4760a45328493e87523c2ee8292599"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53464
I20260709 13:01:19.138100  2305 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:19.138573  2305 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:19.165506  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 47096.6us
I20260709 13:01:19.179140  1564 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:41282
I20260709 13:01:19.179888  1564 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:19.180500  1564 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.179665  1563 raft_consensus.cc:2955] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.200420  1714 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:19.204669  1564 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:19.206668  2788 raft_consensus.cc:993] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:19.170715  2305 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:19.231040  2304 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:01:19.216470  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 46375us
I20260709 13:01:19.221720  2788 raft_consensus.cc:1081] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
W20260709 13:01:19.235973  1451 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leader transfer in progress [suppressed 2 similar messages]
I20260709 13:01:19.236907  2883 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:19.238588  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:19.239442  1565 raft_consensus.cc:3055] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:19.239864  1565 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:19.242103  1565 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:19.227705  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:19.251152  2302 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:19.225390  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 67529.9us
I20260709 13:01:19.247340  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:19.252879  1642 raft_consensus.cc:3055] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:19.253234  1642 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:19.242713  2769 raft_consensus.cc:993] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: : Instructing follower 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:19.255823  1643 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:19.247327  2883 raft_consensus.cc:2955] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.254618  1465 catalog_manager.cc:5697] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.264003  1642 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:19.265836  1642 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.261011  1567 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:19.260234  2661 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:19.268226  2305 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:19.268119  2303 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53448
I20260709 13:01:19.268971  2303 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:19.264104  1565 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.269732  2303 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.269456  2874 raft_consensus.cc:1081] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Signalling peer 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:19.254660  1564 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:19.277254  1713 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:36066
I20260709 13:01:19.278033  1644 raft_consensus.cc:2955] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.286007  1713 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:19.286490  1713 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.274350  2301 raft_consensus.cc:2955] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.289142  2303 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:19.332103  1642 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:19.321151  2843 raft_consensus.cc:993] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:19.340758  2843 raft_consensus.cc:1081] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:19.338833  1464 catalog_manager.cc:5697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.342479  2305 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53464
I20260709 13:01:19.343034  2305 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:19.343367  2305 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.339898  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:19.346815  1643 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.289563  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:19.322568  1713 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:19.351137  1640 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.351233  1644 consensus_queue.cc:237] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:19.351251  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:19.352785  1713 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:19.353183  1564 raft_consensus.cc:3055] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:19.341398  1533 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:19.354050  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:19.354144  1565 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:19.354511  2303 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:19.353569  1564 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:01:19.355746  1451 catalog_manager.cc:4755] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): ChangeConfig:REMOVE_PEER RPC failed for tablet 37b565cd27a64e96addb09389c2c7d5e: Remote error: Service unavailable: leader transfer in progress
I20260709 13:01:19.356786  2772 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:19.357286  2772 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:19.357859  1567 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:19.358211  2772 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:19.362365  2305 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:19.364158  1643 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:19.354866  1642 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.363447  2268 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:19.355707  1564 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:19.371253  1640 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:19.377449  2305 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:19.385589  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:19.386509  1565 raft_consensus.cc:3055] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:19.386624  2753 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:19.386888  1565 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:19.387578  1564 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.388729  2269 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:19.355645  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:19.413826  1716 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:19.414178  1716 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:19.388887  2303 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:19.389245  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:19.416038  1713 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.390036  2753 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:19.391503  2305 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:19.419610  1640 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:19.422552  2753 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:19.423858  2769 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:19.413477  2754 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:19.427207  2754 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:19.425807  2874 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:19.419610  1564 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:19.408713  1565 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:19.433699  1642 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:19.443004  1565 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.444933  1713 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:19.451915  1554 tablet_service.cc:1558] Processing DeleteTablet for tablet ac4c23f0e4f742e38ca7a5ebd7778dfa with delete_type TABLET_DATA_TOMBSTONED (TS 3f64b76180a54492bb015e405caeacec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41262
I20260709 13:01:19.432525  1681 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:19.428220  2754 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:19.461402  2302 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:19.474572  2851 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:19.486212  2851 raft_consensus.cc:697] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:19.431732  2763 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:01:19.510116  1713 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:19.428860  1716 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:19.514577  1565 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:19.515374  2748 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:19.487744  2851 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:19.515864  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:01:19.688086  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.688225  1451 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 37b565cd27a64e96addb09389c2c7d5e with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3f64b76180a54492bb015e405caeacec is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.685343  1464 catalog_manager.cc:5697] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 2, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:19.506517  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 125611us
I20260709 13:01:19.675074  2886 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:19.717329  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:19.718225  1644 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:19.726356  2888 tablet_replica.cc:333] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
W20260709 13:01:19.730785  2887 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.090s	user 0.003s	sys 0.000s
I20260709 13:01:19.706676  1716 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:19.780649  2844 raft_consensus.cc:2955] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.794322  1462 catalog_manager.cc:5697] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 2, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:19.845577  2748 raft_consensus.cc:2955] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.847597  2301 raft_consensus.cc:2955] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.872521  1644 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:19.888469  1716 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:19.885744  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ac4c23f0e4f742e38ca7a5ebd7778dfa with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:19.916961  2305 raft_consensus.cc:2955] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:19.941895  2303 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:01:19.942811  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 253363us
I20260709 13:01:19.949652  1464 catalog_manager.cc:5697] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 2, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:19.923069  2886 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:01:19.980121  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 88216.9us
I20260709 13:01:20.060527  2886 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:20.065196  2879 raft_consensus.cc:993] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:20.065759  2879 raft_consensus.cc:1081] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Signalling peer 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:20.069319  2888 raft_consensus.cc:2243] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:20.071507  1709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:36066
I20260709 13:01:20.072184  1709 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:20.072574  1709 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:20.076042  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 73587.7us
I20260709 13:01:20.083340  2888 raft_consensus.cc:2272] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:20.093479  1643 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:20.097456  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 403bcba338a042c1b5fecfe8ac430922 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:01:20.112921  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 131488us
W20260709 13:01:20.116411  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 45622.2us
W20260709 13:01:20.120172  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 164855us
W20260709 13:01:20.119621  2867 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:01:20.153088  1709 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:20.155741  1709 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:20.168416  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 190725us
I20260709 13:01:20.159430  2905 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
W20260709 13:01:20.174010  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 150673us
I20260709 13:01:20.174731  1452 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 92a2b88a149f424591686a115bc1f0d6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:20.178015  1565 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:20.181779  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:20.189687  1644 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:20.191807  2897 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:01:20.196278  1642 raft_consensus.cc:2955] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:20.217641  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:20.218529  1644 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:20.221131  2838 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:01:20.121227   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:20.202365  2753 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:01:20.286612  2888 ts_tablet_manager.cc:1916] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:20.291267  1709 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:20.324191  1640 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:01:20.342729  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 165422us
I20260709 13:01:20.337726  1644 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:20.345734  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:20.355573  1714 raft_consensus.cc:2955] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:20.376278  2918 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:01:20.381644  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 182959us
I20260709 13:01:20.392426  2918 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:20.408111  2918 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:20.391388  2753 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:01:20.449854  1462 catalog_manager.cc:5697] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:20.459545  1643 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:20.463950  2844 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:20.471992  2754 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:20.480369  1565 raft_consensus.cc:3055] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:20.480841  1565 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:20.481590  1565 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:20.482672  1565 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:20.502877  1642 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:01:20.507478  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 91082.2us
I20260709 13:01:20.512399  1642 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.518782  1565 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:20.518888  2661 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:20.525614  2844 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:01:20.543883  2890 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
W20260709 13:01:20.589686  2923 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:01:20.620204  1709 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:20.663030  1699 tablet_service.cc:1558] Processing DeleteTablet for tablet 403bcba338a042c1b5fecfe8ac430922 with delete_type TABLET_DATA_TOMBSTONED (TS 28bc59d37f664ce0a6f5784f2619f0b5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36054
I20260709 13:01:20.663708  2888 ts_tablet_manager.cc:1929] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:20.664209  2888 log.cc:1199] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 3f64b76180a54492bb015e405caeacec: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/wals/ac4c23f0e4f742e38ca7a5ebd7778dfa
I20260709 13:01:20.675004   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:20.712548  1718 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:20.720491  2753 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:20.716840  2888 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41262 (request call id 78) took 1275 ms. Trace:
I20260709 13:01:20.729948  2888 rpcz_store.cc:276] 0709 13:01:19.430714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.451555 (+ 20841us) service_pool.cc:224] Handling call
0709 13:01:19.725926 (+274371us) ts_tablet_manager.cc:1151] Deleting tablet ac4c23f0e4f742e38ca7a5ebd7778dfa
0709 13:01:19.726026 (+   100us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:01:20.462432 (+736406us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:20.663557 (+201125us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:20.705624 (+ 42067us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":170533,"tablet-delete.queue_time_us":264702,"thread_start_us":221215,"threads_started":1}
I20260709 13:01:20.774888  1451 catalog_manager.cc:5028] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): tablet ac4c23f0e4f742e38ca7a5ebd7778dfa (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:20.783411  1462 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:20.824779  2842 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:20.825428  2842 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:20.832355  2932 tablet_replica.cc:333] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:01:20.840271  1568 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:20.848155  1565 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:20.818274  1462 catalog_manager.cc:5697] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 2 to 3, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 3 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:20.851070  2838 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:20.851706  2838 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:20.863804  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:20.866640  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:20.863804  1467 catalog_manager.cc:5697] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 3 to 4, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:20.868822  1533 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:20.869242  2838 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:20.875614  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:20.877683  2906 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:20.903648  2932 raft_consensus.cc:2243] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:20.904685  2842 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:20.907758  2932 raft_consensus.cc:2272] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:20.908892  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:20.916205  1683 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:20.929137  2932 ts_tablet_manager.cc:1916] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:20.929386  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:20.944984  1451 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 37b565cd27a64e96addb09389c2c7d5e with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3f64b76180a54492bb015e405caeacec is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:20.974761  1554 tablet_service.cc:1558] Processing DeleteTablet for tablet 92a2b88a149f424591686a115bc1f0d6 with delete_type TABLET_DATA_TOMBSTONED (TS 3f64b76180a54492bb015e405caeacec not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41262
I20260709 13:01:20.981516  2901 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:20.954141  2778 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:01:20.962628  2753 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:01:20.997303  1464 catalog_manager.cc:5697] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 3, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:20.982471  2901 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:21.025281  2901 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:21.025974  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:21.028954  2888 tablet_replica.cc:333] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:01:21.030165  2888 raft_consensus.cc:2243] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.036909  1566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:21.037577  1566 raft_consensus.cc:606] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:21.037930  1566 raft_consensus.cc:614] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:21.039637  1566 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:21.047706  1641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:01:21.048295  1641 raft_consensus.cc:606] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
W20260709 13:01:21.050400  2938 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:01:21.050259  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:21.050879  2778 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:21.053696  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:21.054350  1710 raft_consensus.cc:606] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:21.058182  2888 raft_consensus.cc:2272] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.062947  1533 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:01:21.063766  1533 raft_consensus.cc:2686] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:01:21.066185  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:21.070981  2888 ts_tablet_manager.cc:1916] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:21.076880  2932 ts_tablet_manager.cc:1929] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:21.077368  2932 log.cc:1199] T 403bcba338a042c1b5fecfe8ac430922 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/wals/403bcba338a042c1b5fecfe8ac430922
I20260709 13:01:21.080441  1642 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:21.082099  2882 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:21.079439  1451 catalog_manager.cc:5028] TS 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): tablet 403bcba338a042c1b5fecfe8ac430922 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:21.091531  1566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:21.092720  1566 raft_consensus.cc:606] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:21.093883   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:01:21.096520   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:21.176589  2918 raft_consensus.cc:993] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:21.177362  2842 raft_consensus.cc:1081] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:21.180398  2303 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53448
I20260709 13:01:21.181223  2303 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:21.181689  2303 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:21.215149  2303 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:21.217152  2303 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:21.218585  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:21.229477  1710 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:21.229890  1710 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:21.230702  1710 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:21.231839  1710 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:21.231750  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:21.245625  1568 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:21.245702  2768 raft_consensus.cc:993] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:21.256623  2768 raft_consensus.cc:1081] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Signalling peer 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:21.258581  1712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:36066
I20260709 13:01:21.259233  1712 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:21.259616  1712 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:21.253989  1710 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:21.262902  1568 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:21.264189  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:21.264947  2897 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:21.265450  2897 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:21.266287  2897 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:21.268435  1712 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:21.270707  1712 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:21.272569  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:21.273351  1641 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:21.275251  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:21.275842  1568 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:21.276126  1568 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:21.277055  1568 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:21.278298  1568 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:21.284159  1641 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:21.286502  1568 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:21.287840  1681 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:21.291108  2778 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:21.291635  2778 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:21.291098  2888 ts_tablet_manager.cc:1929] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:21.292799  2888 log.cc:1199] T 92a2b88a149f424591686a115bc1f0d6 P 3f64b76180a54492bb015e405caeacec: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/wals/92a2b88a149f424591686a115bc1f0d6
I20260709 13:01:21.292714  2778 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:21.294497  1451 catalog_manager.cc:5028] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): tablet 92a2b88a149f424591686a115bc1f0d6 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:21.293912  1464 catalog_manager.cc:5697] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:21.305526  2895 raft_consensus.cc:993] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6: : Instructing follower 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:21.306070  2880 raft_consensus.cc:1081] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Signalling peer 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:21.307826  1563 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:41282
I20260709 13:01:21.308418  1563 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:21.308828  1563 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:21.309711  1465 catalog_manager.cc:5697] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 3, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:21.318336  1563 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:21.321177  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:21.321790  1643 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:21.322121  1643 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:21.322830  1643 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:21.323906  1643 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:21.330165  1563 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:21.332749  1643 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:21.333033  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:21.333746  1712 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:21.334190  1533 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:21.335045  2768 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:21.335558  2768 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:21.336498  2768 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:21.341105  1712 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:21.346765  1465 catalog_manager.cc:5697] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 2, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:21.491258  2301 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:21.500067  1568 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:21.500310  1712 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:21.502185  2886 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:21.508497  2897 raft_consensus.cc:2955] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 4 to 6, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:21.516029  2902 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:21.518604  1712 raft_consensus.cc:2955] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 4 to 6, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:21.534428  1453 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 37b565cd27a64e96addb09389c2c7d5e with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 7)
I20260709 13:01:21.534947  1566 raft_consensus.cc:2955] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 4 to 6, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:21.542646  1464 catalog_manager.cc:5697] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 4 to 6, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New cstate: current_term: 3 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:21.562489  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 37b565cd27a64e96addb09389c2c7d5e with delete_type TABLET_DATA_TOMBSTONED (TS 5184c101be204393adf3ac8b31f25ad6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60370
I20260709 13:01:21.583330  2949 tablet_replica.cc:333] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:01:21.584291  2949 raft_consensus.cc:2243] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:21.585139  2949 raft_consensus.cc:2272] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:21.587877  2949 ts_tablet_manager.cc:1916] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:21.601212  2949 ts_tablet_manager.cc:1929] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:01:21.601528  2949 log.cc:1199] T 37b565cd27a64e96addb09389c2c7d5e P 5184c101be204393adf3ac8b31f25ad6: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/wals/37b565cd27a64e96addb09389c2c7d5e
I20260709 13:01:21.603156  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet 37b565cd27a64e96addb09389c2c7d5e (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:21.761912  1710 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:21.763975  1568 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:21.764297  2902 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:21.765462  2897 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:21.805027  1710 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:21.806696  2768 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:21.833449  1641 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:21.836318  2901 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:21.857797  1568 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:21.860008  2922 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:21.887496  1641 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:21.894248  2778 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:22.097656   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:22.146420   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:22.187692  1566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:22.188354  1566 raft_consensus.cc:606] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:22.191040  2901 raft_consensus.cc:993] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:22.191602  2901 raft_consensus.cc:1081] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Signalling peer 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:22.194288  1641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:01:22.194455  1710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:36066
I20260709 13:01:22.194967  1641 raft_consensus.cc:606] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:01:22.195266  1710 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:22.195816  1710 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.205030  1710 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:22.205547  1712 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:22.206290  1712 raft_consensus.cc:606] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:22.209677   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:01:22.210538  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:22.211165  1566 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:22.211470  1566 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:22.211596  1710 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:22.212213  1566 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:22.213654  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:22.213879  1566 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.214272  1643 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.214486   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:22.220587  1566 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:22.221694  1681 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:22.222587  2922 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:22.222837  1643 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:22.223073  2922 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:22.224150  2922 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:22.234164  1462 catalog_manager.cc:5697] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 2, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:22.467873  2778 raft_consensus.cc:993] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:22.468425  2967 raft_consensus.cc:1081] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:22.470028  1643 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60390
I20260709 13:01:22.470634  1643 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:22.470981  1643 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.476459  1643 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:22.478358  1643 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:22.480211  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:22.480723  2303 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.482059  2763 raft_consensus.cc:993] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: : Instructing follower 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:22.482545  2895 raft_consensus.cc:1081] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Signalling peer 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:22.484185  1566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:41282
I20260709 13:01:22.484607  1566 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:22.484834  1566 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.486096  2303 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:22.487394  1609 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:22.488255  2895 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:22.488775  2895 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:22.489809  2895 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:22.488674  2301 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:22.493973  2922 consensus_queue.cc:1059] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:22.494959  2778 raft_consensus.cc:3055] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:22.495329  2778 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:22.496146  2778 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:22.497416  2778 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.497620  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:22.498481  1566 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:22.502482  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:22.503163  1643 raft_consensus.cc:3055] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:22.503481  1643 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:22.504314  1566 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:22.505616  1643 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:22.506719  1643 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.511080  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:22.511674  1710 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:22.514089  1643 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:22.515260  1533 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:22.516057  2966 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:22.516057  1712 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:22.516652  2966 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:22.517504  2966 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:22.522037  1710 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:22.522822  1462 catalog_manager.cc:5697] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:22.532902  1464 catalog_manager.cc:5697] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 2, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:22.634181  1563 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:22.640995  1641 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:01:22.641551  1534 consensus_peers.cc:597] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70a0a5f8de344870be97d83cec17c047. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:22.641568  1710 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:22.642382  2768 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:22.643275  2966 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:22.650350  2966 raft_consensus.cc:2955] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:22.652433  1710 raft_consensus.cc:2955] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:22.652640  1641 raft_consensus.cc:2955] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:22.670334  1643 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:22.669862  1464 catalog_manager.cc:5697] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:22.696633  1712 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:22.697510  1563 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:22.699164  2895 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:22.700865  2763 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:01:22.703305  1609 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c5eae102421c4bc4b19781bf4470a0b0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:22.717525  1643 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:22.722532  1568 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:22.719475  2763 raft_consensus.cc:2955] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:22.733767  2967 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:22.734553  1563 raft_consensus.cc:2955] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:22.741806  1712 raft_consensus.cc:2955] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:22.746927  2922 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:22.775460  1465 catalog_manager.cc:5697] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:22.815048  1568 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:22.824898  1712 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:22.826709  2901 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:01:22.828037  1534 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e62e41b6e9a54f26a1c7c1538bdf5334. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:22.831529  1641 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:22.833473  2901 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:22.842360  2768 raft_consensus.cc:2955] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:22.847524  1712 raft_consensus.cc:2955] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:22.856055  1641 raft_consensus.cc:2955] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
W20260709 13:01:22.861850  1515 auto_rebalancer.cc:629] Failed to schedule move for tablet 5a08948b1fac416d8d27be0561a00db4: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:01:22.863108  1464 catalog_manager.cc:5697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:23.003571  1643 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:23.005323  2966 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:23.025612  1712 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:23.031198  2966 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:23.049002  2303 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:23.053186  2973 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:23.075901  1712 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:23.077989  2895 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:23.113515  2989 ts_tablet_manager.cc:933] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
I20260709 13:01:23.115732  2989 tablet_copy_client.cc:323] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:36547
I20260709 13:01:23.117579  1579 tablet_copy_service.cc:140] P 3f64b76180a54492bb015e405caeacec: Received BeginTabletCopySession request for tablet 70a0a5f8de344870be97d83cec17c047 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:35222)
I20260709 13:01:23.118193  1579 tablet_copy_service.cc:161] P 3f64b76180a54492bb015e405caeacec: Beginning new tablet copy session on tablet 70a0a5f8de344870be97d83cec17c047 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:35222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-70a0a5f8de344870be97d83cec17c047
I20260709 13:01:23.127511  1579 tablet_copy_source_session.cc:215] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:23.130980  2989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70a0a5f8de344870be97d83cec17c047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.147116  2989 tablet_copy_client.cc:806] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:23.148237  2989 tablet_copy_client.cc:670] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:23.152536  2989 tablet_copy_client.cc:538] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:23.168602  2989 tablet_bootstrap.cc:492] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:23.181509  2991 ts_tablet_manager.cc:933] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:23.197422  2991 tablet_copy_client.cc:323] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:40023
I20260709 13:01:23.199409  1654 tablet_copy_service.cc:140] P 5184c101be204393adf3ac8b31f25ad6: Received BeginTabletCopySession request for tablet c5eae102421c4bc4b19781bf4470a0b0 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:53222)
I20260709 13:01:23.199978  1654 tablet_copy_service.cc:161] P 5184c101be204393adf3ac8b31f25ad6: Beginning new tablet copy session on tablet c5eae102421c4bc4b19781bf4470a0b0 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:53222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-c5eae102421c4bc4b19781bf4470a0b0
I20260709 13:01:23.206766  2989 tablet_bootstrap.cc:492] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:23.208001  2989 tablet_bootstrap.cc:492] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:23.208115  1654 tablet_copy_source_session.cc:215] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:23.208833  2989 ts_tablet_manager.cc:1403] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.040s	user 0.027s	sys 0.009s
I20260709 13:01:23.211993  2991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5eae102421c4bc4b19781bf4470a0b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.212040  2989 raft_consensus.cc:359] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:23.213078  2989 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:23.213692  2989 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:23.215824   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:23.216682  2989 ts_tablet_manager.cc:1434] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:01:23.224903  1579 tablet_copy_service.cc:342] P 3f64b76180a54492bb015e405caeacec: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-70a0a5f8de344870be97d83cec17c047 received from {username='slave'} at 127.0.0.1:35222
I20260709 13:01:23.225463  1579 tablet_copy_service.cc:434] P 3f64b76180a54492bb015e405caeacec: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-70a0a5f8de344870be97d83cec17c047 on tablet 70a0a5f8de344870be97d83cec17c047 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:23.233716  2991 tablet_copy_client.cc:806] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:23.234601  2991 tablet_copy_client.cc:670] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:23.247023  2991 tablet_copy_client.cc:538] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:23.252707  2989 ts_tablet_manager.cc:933] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
I20260709 13:01:23.255102  2989 tablet_copy_client.cc:323] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:36547
I20260709 13:01:23.257032  1579 tablet_copy_service.cc:140] P 3f64b76180a54492bb015e405caeacec: Received BeginTabletCopySession request for tablet e62e41b6e9a54f26a1c7c1538bdf5334 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:35222)
I20260709 13:01:23.257608  1579 tablet_copy_service.cc:161] P 3f64b76180a54492bb015e405caeacec: Beginning new tablet copy session on tablet e62e41b6e9a54f26a1c7c1538bdf5334 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:35222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-e62e41b6e9a54f26a1c7c1538bdf5334
I20260709 13:01:23.258859  2991 tablet_bootstrap.cc:492] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:23.289279  1579 tablet_copy_source_session.cc:215] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:23.299499  2989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e62e41b6e9a54f26a1c7c1538bdf5334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:23.309226  2991 tablet_bootstrap.cc:492] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:23.313516  2991 tablet_bootstrap.cc:492] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:23.314302  2991 ts_tablet_manager.cc:1403] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.056s	user 0.021s	sys 0.008s
I20260709 13:01:23.325892  2991 raft_consensus.cc:359] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:23.326823  2991 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:23.327571  2991 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:23.331431  2991 ts_tablet_manager.cc:1434] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.009s	user 0.000s	sys 0.008s
I20260709 13:01:23.333357  2989 tablet_copy_client.cc:806] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:23.334801  2989 tablet_copy_client.cc:670] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:23.339780  2989 tablet_copy_client.cc:538] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:23.342447  1654 tablet_copy_service.cc:342] P 5184c101be204393adf3ac8b31f25ad6: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-c5eae102421c4bc4b19781bf4470a0b0 received from {username='slave'} at 127.0.0.1:53222
I20260709 13:01:23.343120  1654 tablet_copy_service.cc:434] P 5184c101be204393adf3ac8b31f25ad6: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-c5eae102421c4bc4b19781bf4470a0b0 on tablet c5eae102421c4bc4b19781bf4470a0b0 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:23.351348  2989 tablet_bootstrap.cc:492] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:23.383127  2989 tablet_bootstrap.cc:492] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:23.383916  2989 tablet_bootstrap.cc:492] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:23.384514  2989 ts_tablet_manager.cc:1403] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.033s	user 0.026s	sys 0.004s
I20260709 13:01:23.387239  2989 raft_consensus.cc:359] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:23.388010  2989 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:23.388648  2989 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:23.390129   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:23.391536  2989 ts_tablet_manager.cc:1434] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:01:23.397658  1579 tablet_copy_service.cc:342] P 3f64b76180a54492bb015e405caeacec: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-e62e41b6e9a54f26a1c7c1538bdf5334 received from {username='slave'} at 127.0.0.1:35222
I20260709 13:01:23.398149  1579 tablet_copy_service.cc:434] P 3f64b76180a54492bb015e405caeacec: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-e62e41b6e9a54f26a1c7c1538bdf5334 on tablet e62e41b6e9a54f26a1c7c1538bdf5334 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:23.459120  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:23.459739  1710 raft_consensus.cc:606] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:23.463084  1568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:23.463693  1568 raft_consensus.cc:606] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:23.466758  1641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:01:23.467370  1641 raft_consensus.cc:606] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:01:23.468823   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:01:23.471928   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:23.486887  2301 raft_consensus.cc:1217] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:01:23.535912  2778 raft_consensus.cc:993] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:23.536857  2778 raft_consensus.cc:1081] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:23.538954  1641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60390
I20260709 13:01:23.539561  1641 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:23.539948  1641 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.546439  1641 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:23.548945  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:23.549479  1710 raft_consensus.cc:3055] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:23.549842  1710 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:23.550706  1710 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:23.551278  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:23.551962  1568 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.555933  1710 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.548069  1641 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:23.568048  1710 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:23.568178  1568 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:23.569290  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:23.570505  2763 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:23.571044  2763 raft_consensus.cc:697] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:23.571914  2763 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:23.590023  1464 catalog_manager.cc:5697] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:23.637326  2768 raft_consensus.cc:993] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:23.637826  2768 raft_consensus.cc:1081] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Signalling peer 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:23.642539  1710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:36066
I20260709 13:01:23.644152  1710 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:23.644665  1710 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.649371  2301 raft_consensus.cc:1217] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:01:23.655082  2301 raft_consensus.cc:1217] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:23.669505  1710 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:23.672333  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:23.672356  1710 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:23.672571  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:23.673002  1568 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:23.673377  1568 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:23.673414  1643 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.674225  1568 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:23.675823  1568 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:23.681464  1568 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:23.683280  1681 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:23.684157  2922 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:23.684659  2922 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:23.685582  2922 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:23.691835  1643 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:23.699673  1462 catalog_manager.cc:5697] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 3, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:23.947080  2895 raft_consensus.cc:993] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6: : Instructing follower 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:23.947588  2763 raft_consensus.cc:1081] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Signalling peer 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:23.948969  1566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:41282
I20260709 13:01:23.949483  1566 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:23.949843  1566 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.955511  1566 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:23.959942  1566 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:23.960438  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:23.961038  1710 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.961246  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:23.961863  1641 raft_consensus.cc:3055] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:23.962175  1641 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:23.962944  1641 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:23.964285  1641 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:23.971413  1641 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:23.972697  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:23.971668  1710 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:23.973752  2997 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:23.976976  2997 raft_consensus.cc:697] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:23.977910  2997 consensus_queue.cc:237] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:23.983297  1710 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:23.986258  2895 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:24.007289  1568 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:24.010608  2973 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:24.012934  1462 catalog_manager.cc:5697] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 2, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:24.029744  2901 raft_consensus.cc:1064] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:24.032780  2901 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:24.060667  2301 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:24.063007  1643 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:24.063060  1710 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:24.065598  2966 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:24.067843  2768 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:24.076671  2901 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:24.087069  1568 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:24.089507  2922 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:24.106539  2768 raft_consensus.cc:2955] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.114177  1712 raft_consensus.cc:2955] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.114022  1643 raft_consensus.cc:2955] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.138252  3007 raft_consensus.cc:1064] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:24.141810  3007 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:24.143463  2303 raft_consensus.cc:2955] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.158842  2301 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:24.159330  1465 catalog_manager.cc:5697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.160998  2302 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Advancing to term 3
I20260709 13:01:24.164500  3007 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:01:24.168081  1641 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:24.169744  2778 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:24.176035  1712 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:24.177953  2763 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:24.187107  1563 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:24.195473  2763 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:01:24.223369  3015 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:01:24.237018  2973 raft_consensus.cc:2955] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.250118  1566 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:24.251505  2301 raft_consensus.cc:2955] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.263646  1640 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:24.263631  2303 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:24.265594  2768 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:24.267347  2966 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:24.273769  2997 raft_consensus.cc:2955] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.275063  2302 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:24.276643  2778 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:24.276600  1640 raft_consensus.cc:2955] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:01:24.386438  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 83763.6us
I20260709 13:01:24.390249  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e62e41b6e9a54f26a1c7c1538bdf5334 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:24.394655  1465 catalog_manager.cc:5697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 4 to 5, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:24.407792  2967 raft_consensus.cc:1064] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:24.411164  2967 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:24.416460  1699 tablet_service.cc:1558] Processing DeleteTablet for tablet e62e41b6e9a54f26a1c7c1538bdf5334 with delete_type TABLET_DATA_TOMBSTONED (TS 28bc59d37f664ce0a6f5784f2619f0b5 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:36054
I20260709 13:01:24.417805  1462 catalog_manager.cc:5697] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.421883  3029 tablet_replica.cc:333] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:01:24.423939  3029 raft_consensus.cc:2243] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:01:24.428771  3019 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:01:24.427887  2302 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:24.434556  1568 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:24.425541  3029 raft_consensus.cc:2272] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:24.443572  1712 raft_consensus.cc:2955] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.447885  3029 ts_tablet_manager.cc:1916] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:01:24.458642  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 163881us
I20260709 13:01:24.462455  2303 raft_consensus.cc:2955] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.465147  1639 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:01:24.480886  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 99566.5us
I20260709 13:01:24.433746  1641 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:24.497956  2922 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:24.501750   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:24.511590  3021 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:24.522116  1563 raft_consensus.cc:2955] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.534895  3029 ts_tablet_manager.cc:1929] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:01:24.537441  3029 log.cc:1199] T e62e41b6e9a54f26a1c7c1538bdf5334 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/wals/e62e41b6e9a54f26a1c7c1538bdf5334
I20260709 13:01:24.546139  1451 catalog_manager.cc:5028] TS 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): tablet e62e41b6e9a54f26a1c7c1538bdf5334 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:24.575780  2778 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:24.584615  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 88274.1us
I20260709 13:01:24.586860  3030 raft_consensus.cc:2955] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 3 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.590161  2302 raft_consensus.cc:2955] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.593685  1640 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:24.598937  2778 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:24.606165  2303 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:24.608065  1568 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:24.609776  2895 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:24.611398  3027 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:24.611510  1566 raft_consensus.cc:2955] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.618304  2895 raft_consensus.cc:2955] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.621830  1709 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:24.620323  2303 raft_consensus.cc:2955] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.623988  1563 raft_consensus.cc:2955] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.625216  3028 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:24.638192  1640 raft_consensus.cc:2955] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.661787  1462 catalog_manager.cc:5697] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:24.695874  1452 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c5eae102421c4bc4b19781bf4470a0b0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:24.708534  1699 tablet_service.cc:1558] Processing DeleteTablet for tablet c5eae102421c4bc4b19781bf4470a0b0 with delete_type TABLET_DATA_TOMBSTONED (TS 28bc59d37f664ce0a6f5784f2619f0b5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36054
I20260709 13:01:24.708212  1467 catalog_manager.cc:5697] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 3 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:24.709553  3029 tablet_replica.cc:333] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:01:24.729193  3029 raft_consensus.cc:2243] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:24.730055  3029 raft_consensus.cc:2272] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:24.745124   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:24.746666  3029 ts_tablet_manager.cc:1916] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:24.786396  3029 ts_tablet_manager.cc:1929] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:24.787073  3029 log.cc:1199] T c5eae102421c4bc4b19781bf4470a0b0 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/wals/c5eae102421c4bc4b19781bf4470a0b0
I20260709 13:01:24.789359  1451 catalog_manager.cc:5028] TS 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): tablet c5eae102421c4bc4b19781bf4470a0b0 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:24.829083  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:24.829799  1709 raft_consensus.cc:606] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:01:24.833137  1640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:01:24.833823  1640 raft_consensus.cc:606] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:24.837669  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:24.838338  1709 raft_consensus.cc:606] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:24.842133  1563 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:24.842751  1563 raft_consensus.cc:606] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:24.845116   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:01:24.847136   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:24.901496  3027 raft_consensus.cc:993] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6: : Instructing follower 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:24.902052  2895 raft_consensus.cc:1081] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Signalling peer 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:24.903806  1709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:36078
I20260709 13:01:24.904362  1709 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:24.904721  1709 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.911491  1709 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:24.913646  1709 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:24.914678  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:24.915319  1563 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.916909  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:24.917495  1640 raft_consensus.cc:3055] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:24.917809  1640 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:24.918516  1640 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:24.919574  1640 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:24.922943  1563 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:24.924525  1681 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:24.925410  2922 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:24.926465  2922 raft_consensus.cc:697] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:24.927281  2922 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:24.930783  1640 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:24.943575  1462 catalog_manager.cc:5697] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 3, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.059078  2768 raft_consensus.cc:993] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:25.059592  2768 raft_consensus.cc:1081] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:25.061165  2302 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53464
I20260709 13:01:25.061708  2302 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:25.062050  2302 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.084949  2302 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:25.088238  2302 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:25.088186  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:25.088482  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:25.089038  1566 raft_consensus.cc:3055] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:25.089342  1640 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.089361  1566 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:25.090370  1566 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:25.091617  1566 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.098739  1640 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:25.099943  2269 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:25.100898  3035 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:25.101389  3035 raft_consensus.cc:697] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:25.103276  3035 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:25.103742  1566 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:25.113372  1462 catalog_manager.cc:5697] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 2, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.176172  1709 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:25.182607  2302 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:25.184990  2922 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:25.191229  1566 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:25.192795  3031 raft_consensus.cc:2955] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:25.194854  2302 raft_consensus.cc:2955] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:25.198434  2778 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:25.209010  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 70a0a5f8de344870be97d83cec17c047 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:25.205435  1566 raft_consensus.cc:2955] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:25.216238  1462 catalog_manager.cc:5697] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 5 to 6, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.231278  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 70a0a5f8de344870be97d83cec17c047 with delete_type TABLET_DATA_TOMBSTONED (TS 5184c101be204393adf3ac8b31f25ad6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60370
I20260709 13:01:25.236485  3038 tablet_replica.cc:333] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:01:25.238175  3038 raft_consensus.cc:2243] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:25.238881  3038 raft_consensus.cc:2272] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:25.242533  3038 ts_tablet_manager.cc:1916] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:25.247153  2778 raft_consensus.cc:993] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:25.247704  2778 raft_consensus.cc:1081] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:25.249887  1566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:41274
I20260709 13:01:25.250486  1566 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:25.250938  1566 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.256613  1566 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:25.259037  1566 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:25.259259  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:25.259670  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:25.260195  3038 ts_tablet_manager.cc:1929] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:01:25.260680  1709 raft_consensus.cc:3055] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:25.260735  3038 log.cc:1199] T 70a0a5f8de344870be97d83cec17c047 P 5184c101be204393adf3ac8b31f25ad6: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/wals/70a0a5f8de344870be97d83cec17c047
I20260709 13:01:25.259919  1640 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.261112  1709 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:25.262003  1709 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:25.263096  1709 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.263872  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet 70a0a5f8de344870be97d83cec17c047 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:25.268103  1640 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:25.269791  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:25.270025  1709 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:25.270656  2966 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:25.271215  2966 raft_consensus.cc:697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:25.272007  2966 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:25.283094  1465 catalog_manager.cc:5697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:25.284231  3031 raft_consensus.cc:993] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:25.284807  2778 raft_consensus.cc:1081] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:25.286391  1640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60390
I20260709 13:01:25.287015  1640 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:25.289559  1640 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.296473  1640 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:25.298295  1640 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:25.300164  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:25.300789  1709 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:25.300189  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:25.301096  1709 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:25.301921  1709 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:25.303283  1709 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.303400  1566 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:25.310391  1566 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:25.310391  1709 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:25.311677  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:25.312453  3027 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:25.312955  3027 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:25.313902  3027 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:25.325089  1465 catalog_manager.cc:5697] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:25.392198  1566 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:25.393402  2778 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:25.402935  1643 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:25.410281  3031 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:25.700850  1640 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:25.722151  2886 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:01:25.751643  1566 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:25.753837  1709 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:25.765962  3027 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:25.774778  1563 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:25.787943  1718 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:25.788487  3051 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:25.793164  3047 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:25.849275   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:25.883085  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 59115.1us
I20260709 13:01:25.892328  1641 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:01:25.929116  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 43572.4us
I20260709 13:01:25.929539  3048 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:01:25.942826  3058 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:01:26.130842  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 49913.8us
I20260709 13:01:26.250769   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:26.312677  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:26.314453  1716 raft_consensus.cc:606] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:01:26.317627  1568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:26.319140  1568 raft_consensus.cc:606] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:26.322819  1640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:01:26.323381  1640 raft_consensus.cc:606] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:26.324649   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:01:26.325999   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:26.331182  1568 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:26.343614  3067 raft_consensus.cc:993] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:26.344136  3067 raft_consensus.cc:1081] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:26.346278  1640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60378
I20260709 13:01:26.347523  1640 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:26.348224  1640 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.350142  1716 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:26.351293  1534 consensus_peers.cc:597] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 82743dc3081f4edba18591ca4b0d9c09. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:26.352430  3068 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:26.352739  1641 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:26.354861  3068 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:26.357959  1640 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:26.362454  1640 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:26.363430  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:26.364244  2302 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.365269  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:26.365978  1568 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:26.366281  1568 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:26.365334  3066 raft_consensus.cc:2955] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:26.367118  1568 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:26.366776  1641 raft_consensus.cc:2955] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:26.368448  1568 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:26.376237  1709 raft_consensus.cc:2955] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:26.381331  1568 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:26.383038  1607 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:26.383458  2302 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:26.384265  3025 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:26.384997  3025 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:26.386260  3025 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:26.385393  1462 catalog_manager.cc:5697] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:26.400830  1641 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:26.411773  1712 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:26.413501  3027 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:26.423004  3051 raft_consensus.cc:2955] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
W20260709 13:01:26.434129  1609 consensus_peers.cc:597] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5448ccc31ba14113b3d56168031c6714. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:26.451476  1712 raft_consensus.cc:2955] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:26.453220  1561 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:26.457607  3025 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:26.471024  1712 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:26.472254  1568 raft_consensus.cc:2955] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:26.486769  1641 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:01:26.488596  1683 consensus_peers.cc:597] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d883c107e7bd4b6daa6d09672acc2b23. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:26.489329  3031 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:26.495396  1568 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:26.497416  3056 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:26.499358  2922 raft_consensus.cc:2955] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:26.516755  1568 raft_consensus.cc:2955] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:26.525638  1641 raft_consensus.cc:2955] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:26.522079  1712 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:26.523392  1462 catalog_manager.cc:5697] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:26.544588  1641 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:26.544998  1683 consensus_peers.cc:597] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2a8ba0cc212d41e5b0ad586b885d07a7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:26.545857  1569 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:26.546878  2922 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:26.547827  2778 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:26.551667  1467 catalog_manager.cc:5697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 2 to 3, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:26.559123  2778 raft_consensus.cc:2955] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:26.568668  1640 raft_consensus.cc:2955] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:26.588155  1462 catalog_manager.cc:5697] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 4, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:26.595983  1569 raft_consensus.cc:2955] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:26.612071  1467 catalog_manager.cc:5697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:26.740209  3031 raft_consensus.cc:993] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:26.740909  2922 raft_consensus.cc:1081] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Signalling peer 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:26.742617  1568 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:41274
I20260709 13:01:26.743175  1568 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:26.743484  1568 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.748178  1568 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:26.750263  1568 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:26.750574  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:26.750814  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:26.751282  1712 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:26.751403  2302 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.751631  1712 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:26.752488  1712 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:26.753870  1712 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:26.762207  2302 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:01:26.769127  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:26.770138  3068 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:26.770629  3068 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:26.770597  1712 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:01:26.771698  3068 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:26.785099  1465 catalog_manager.cc:5697] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 3 to 4, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 4 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:26.821674  3025 raft_consensus.cc:993] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: : Instructing follower 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:26.822304  3025 raft_consensus.cc:1081] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Signalling peer 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:26.824152  1712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:36078
I20260709 13:01:26.824802  1712 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:26.825238  1712 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:26.825762  3085 ts_tablet_manager.cc:933] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
I20260709 13:01:26.828500  3085 tablet_copy_client.cc:323] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:36547
I20260709 13:01:26.830477  1579 tablet_copy_service.cc:140] P 3f64b76180a54492bb015e405caeacec: Received BeginTabletCopySession request for tablet 82743dc3081f4edba18591ca4b0d9c09 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:35222)
I20260709 13:01:26.831072  1579 tablet_copy_service.cc:161] P 3f64b76180a54492bb015e405caeacec: Beginning new tablet copy session on tablet 82743dc3081f4edba18591ca4b0d9c09 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:35222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-82743dc3081f4edba18591ca4b0d9c09
I20260709 13:01:26.833679  1712 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:26.837198  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:26.837383  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:26.837889  1640 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:26.838001  1569 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:26.838047  1579 tablet_copy_source_session.cc:215] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:26.838500  1640 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:26.839497  1640 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:26.840695  1640 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:26.847581  1569 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:26.848985  1681 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:26.838429  1712 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:26.849843  3031 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:26.850447  3031 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:26.850824  3085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82743dc3081f4edba18591ca4b0d9c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:26.851575  3031 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:26.863677  1640 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:26.875257  3085 tablet_copy_client.cc:806] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:26.876173  3085 tablet_copy_client.cc:670] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:26.893639  3085 tablet_copy_client.cc:538] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:26.899151  2302 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:26.901808  3027 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:26.916376  1462 catalog_manager.cc:5697] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 2, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:26.925834  1569 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:26.926651  3085 tablet_bootstrap.cc:492] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:26.928987  3074 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:27.004570  3085 tablet_bootstrap.cc:492] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:27.006016  3085 tablet_bootstrap.cc:492] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:27.006712  3085 ts_tablet_manager.cc:1403] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.080s	user 0.022s	sys 0.003s
I20260709 13:01:27.010453  3085 raft_consensus.cc:359] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:27.011523  3085 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:27.012444  3085 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:27.015182  3085 ts_tablet_manager.cc:1434] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:01:27.018462  1579 tablet_copy_service.cc:342] P 3f64b76180a54492bb015e405caeacec: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-82743dc3081f4edba18591ca4b0d9c09 received from {username='slave'} at 127.0.0.1:35222
I20260709 13:01:27.019043  1579 tablet_copy_service.cc:434] P 3f64b76180a54492bb015e405caeacec: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-82743dc3081f4edba18591ca4b0d9c09 on tablet 82743dc3081f4edba18591ca4b0d9c09 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:27.043224  3085 ts_tablet_manager.cc:933] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:27.059880  3093 ts_tablet_manager.cc:933] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:27.064364  3093 tablet_copy_client.cc:323] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:40023
I20260709 13:01:27.067408  3085 tablet_copy_client.cc:323] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:27.067829  1654 tablet_copy_service.cc:140] P 5184c101be204393adf3ac8b31f25ad6: Received BeginTabletCopySession request for tablet 5448ccc31ba14113b3d56168031c6714 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:53222)
I20260709 13:01:27.068564  1654 tablet_copy_service.cc:161] P 5184c101be204393adf3ac8b31f25ad6: Beginning new tablet copy session on tablet 5448ccc31ba14113b3d56168031c6714 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:53222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-5448ccc31ba14113b3d56168031c6714
I20260709 13:01:27.069953  1727 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet d883c107e7bd4b6daa6d09672acc2b23 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:27.070613  1727 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet d883c107e7bd4b6daa6d09672acc2b23 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-d883c107e7bd4b6daa6d09672acc2b23
I20260709 13:01:27.080152  1654 tablet_copy_source_session.cc:215] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:27.080303  1727 tablet_copy_source_session.cc:215] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:27.086087  3085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d883c107e7bd4b6daa6d09672acc2b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:27.097442  3095 ts_tablet_manager.cc:933] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:27.112398  3093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5448ccc31ba14113b3d56168031c6714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:27.123801  3095 tablet_copy_client.cc:323] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:27.124593  3085 tablet_copy_client.cc:806] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:27.125471  1727 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet 2a8ba0cc212d41e5b0ad586b885d07a7 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:27.125635  3085 tablet_copy_client.cc:670] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:27.126081  1727 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet 2a8ba0cc212d41e5b0ad586b885d07a7 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-2a8ba0cc212d41e5b0ad586b885d07a7
I20260709 13:01:27.133172  1727 tablet_copy_source_session.cc:215] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:27.136931  3085 tablet_copy_client.cc:538] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:27.137784  3095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a8ba0cc212d41e5b0ad586b885d07a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:27.150656  3085 tablet_bootstrap.cc:492] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:27.161514  3095 tablet_copy_client.cc:806] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:27.162281  3095 tablet_copy_client.cc:670] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:27.170352  3095 tablet_copy_client.cc:538] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:27.173010  3093 tablet_copy_client.cc:806] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:27.173736  3093 tablet_copy_client.cc:670] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:27.193770  3095 tablet_bootstrap.cc:492] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:27.207441  3093 tablet_copy_client.cc:538] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:01:27.227303  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 43863.4us
I20260709 13:01:27.229291  3085 tablet_bootstrap.cc:492] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:27.230583  3085 tablet_bootstrap.cc:492] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:27.231529  3085 ts_tablet_manager.cc:1403] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.081s	user 0.030s	sys 0.010s
I20260709 13:01:27.235819  1641 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:27.237461  3056 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:27.240166  2302 raft_consensus.cc:1217] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:27.263396  3085 raft_consensus.cc:359] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:27.264559  3085 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:27.265399  3085 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:27.268756  3085 ts_tablet_manager.cc:1434] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.004s
I20260709 13:01:27.275740  1727 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-d883c107e7bd4b6daa6d09672acc2b23 received from {username='slave'} at 127.0.0.1:33280
I20260709 13:01:27.276292  1727 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-d883c107e7bd4b6daa6d09672acc2b23 on tablet d883c107e7bd4b6daa6d09672acc2b23 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:27.286422  1568 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:27.288738  3056 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:27.298395  3093 tablet_bootstrap.cc:492] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:27.339442  1716 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:01:27.341727  3099 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:27.354231   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:27.389282  2302 raft_consensus.cc:1217] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:01:27.392788  2303 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:01:27.877460  3095 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.120s	user 0.001s	sys 0.001s
W20260709 13:01:27.915743  3108 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:01:27.925717  3095 tablet_bootstrap.cc:492] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:27.928194  3095 tablet_bootstrap.cc:492] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:27.968195  3093 tablet_bootstrap.cc:492] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:27.969563  3093 tablet_bootstrap.cc:492] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:27.970403  3093 ts_tablet_manager.cc:1403] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.672s	user 0.152s	sys 0.214s
I20260709 13:01:27.971454  3095 ts_tablet_manager.cc:1403] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.773s	user 0.076s	sys 0.094s
I20260709 13:01:28.004918  3095 raft_consensus.cc:359] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:28.005846  3095 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:27.974364  3137 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:01:28.010087  3093 raft_consensus.cc:359] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:28.006522  3095 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:28.126436  3095 ts_tablet_manager.cc:1434] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.131s	user 0.014s	sys 0.005s
I20260709 13:01:28.145593  3099 raft_consensus.cc:1064] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:28.149274  3099 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:28.074373  3093 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:28.166975  3093 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:28.184134  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:28.203099  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 215208us
I20260709 13:01:28.209470  3093 ts_tablet_manager.cc:1434] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.239s	user 0.010s	sys 0.007s
I20260709 13:01:28.231843  1727 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-2a8ba0cc212d41e5b0ad586b885d07a7 received from {username='slave'} at 127.0.0.1:33280
I20260709 13:01:28.232587  1727 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-2a8ba0cc212d41e5b0ad586b885d07a7 on tablet 2a8ba0cc212d41e5b0ad586b885d07a7 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:28.251720  1654 tablet_copy_service.cc:342] P 5184c101be204393adf3ac8b31f25ad6: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-5448ccc31ba14113b3d56168031c6714 received from {username='slave'} at 127.0.0.1:53222
I20260709 13:01:28.261726  1654 tablet_copy_service.cc:434] P 5184c101be204393adf3ac8b31f25ad6: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-5448ccc31ba14113b3d56168031c6714 on tablet 5448ccc31ba14113b3d56168031c6714 with peer 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:28.272974  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 798952us
I20260709 13:01:28.283643   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:28.305322  2301 raft_consensus.cc:1217] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:01:28.371637  2302 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:28.379940  3124 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:28.385185  1641 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:28.386757  1710 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:28.400965  3107 raft_consensus.cc:1064] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:28.402812  3128 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:28.403805  3107 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:28.416923  3124 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:01:28.448956  2301 raft_consensus.cc:1217] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:28.469135  2302 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:28.471457  3140 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:28.484310  1641 raft_consensus.cc:2955] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.487920  3124 raft_consensus.cc:2955] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.493721  1644 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:28.497465  1566 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:28.496157  1718 raft_consensus.cc:2955] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.514212  2922 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:28.519237  3031 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:28.524623  2302 raft_consensus.cc:2955] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.557605  1462 catalog_manager.cc:5697] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.574241  3140 raft_consensus.cc:2955] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.581317  1644 raft_consensus.cc:2955] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.604408  2305 raft_consensus.cc:2955] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.600198  1569 raft_consensus.cc:2955] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.658057  1569 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:28.663100  1464 catalog_manager.cc:5697] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 4 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.665135  2305 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:28.666411  1712 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:28.668278  3124 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:28.669153  3099 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:28.686000  3027 raft_consensus.cc:1064] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:28.688629  3027 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:28.691871  3137 raft_consensus.cc:2955] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.704963  1712 raft_consensus.cc:2955] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.709645  1718 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:28.711874  1644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "520f39ecc282462782487a3651e28b44"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:01:28.712533  1644 raft_consensus.cc:606] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:01:28.714504  2305 raft_consensus.cc:2955] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.720520  2302 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:28.721292  1567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:28.721967  1567 raft_consensus.cc:606] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:28.722898  3140 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:28.735607  1718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:28.736428  1718 raft_consensus.cc:606] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:01:28.735967  3031 raft_consensus.cc:2955] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.738153   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:01:28.739787   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:28.738884  2301 raft_consensus.cc:2955] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.752310  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 82743dc3081f4edba18591ca4b0d9c09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:28.751786  1644 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:28.772884  3118 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:28.796599  1644 raft_consensus.cc:2955] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.800696  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d883c107e7bd4b6daa6d09672acc2b23 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:28.803304  1464 catalog_manager.cc:5697] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.824643  1718 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:28.825119  1567 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:28.827988  3129 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:28.835774  3129 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:28.836439  2301 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:28.838768  3114 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:28.849799  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 82743dc3081f4edba18591ca4b0d9c09 with delete_type TABLET_DATA_TOMBSTONED (TS 5184c101be204393adf3ac8b31f25ad6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60370
I20260709 13:01:28.862547  1462 catalog_manager.cc:5697] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 5 to 6, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:28.883041  1554 tablet_service.cc:1558] Processing DeleteTablet for tablet d883c107e7bd4b6daa6d09672acc2b23 with delete_type TABLET_DATA_TOMBSTONED (TS 3f64b76180a54492bb015e405caeacec not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41262
I20260709 13:01:28.888566  3163 tablet_replica.cc:333] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:01:28.901656  3129 raft_consensus.cc:2955] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.922549  2302 raft_consensus.cc:2955] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.964896  1467 catalog_manager.cc:5697] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:28.998225  3164 tablet_replica.cc:333] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:01:28.999619  3164 raft_consensus.cc:2243] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:29.007591  3165 raft_consensus.cc:993] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: : Instructing follower 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:29.008126  3165 raft_consensus.cc:1081] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Signalling peer 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:29.010576  1569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "520f39ecc282462782487a3651e28b44"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:41282
I20260709 13:01:29.011201  1569 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:29.011590  1569 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.023202  1569 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:29.024853  3124 raft_consensus.cc:993] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:29.025357  3124 raft_consensus.cc:1081] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Signalling peer 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:29.025534  1569 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:29.027611  1718 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:36066
I20260709 13:01:29.027873  3164 raft_consensus.cc:2272] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:29.028200  1718 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:29.029918  1718 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.034374  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:29.035063  1712 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.043637  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:29.044461  1644 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:29.045302  1644 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:29.044430  1709 raft_consensus.cc:2955] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:29.046202  1644 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:29.047508  1644 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.059242  1712 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:29.060667  1532 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:29.061563  3099 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:29.062762  3099 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:29.063827  3099 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:29.066174  1567 raft_consensus.cc:2955] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:29.071478  3118 raft_consensus.cc:1064] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:29.074301  3118 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:29.087500  1718 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:29.089465  1643 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:29.090175  1718 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:29.097530  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:29.098446  1566 raft_consensus.cc:3055] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:29.098876  1566 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:29.102519  1566 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:29.104746  3164 ts_tablet_manager.cc:1916] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:29.108517  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:29.111359  1641 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.104602  1566 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.124984  1641 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:29.126734  1682 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:29.128089  3031 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:29.128707  3031 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:29.129739  3031 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:29.138810  1643 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:29.138867  2302 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:29.142403  3107 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:29.142939  1569 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:29.143797  3140 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:29.145778  3118 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:29.163180  1718 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:29.164214  1467 catalog_manager.cc:5697] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 2 to 3, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 3 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:29.165565  3163 raft_consensus.cc:2243] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:29.167500  3163 raft_consensus.cc:2272] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:29.173513  3163 ts_tablet_manager.cc:1916] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:29.180696  2302 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:29.110775  1644 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:29.200974  3165 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:29.202436  3074 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:29.210256  1464 catalog_manager.cc:5697] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 3, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:29.212263  3164 ts_tablet_manager.cc:1929] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:01:29.213554  3164 log.cc:1199] T d883c107e7bd4b6daa6d09672acc2b23 P 3f64b76180a54492bb015e405caeacec: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/wals/d883c107e7bd4b6daa6d09672acc2b23
I20260709 13:01:29.216441  1451 catalog_manager.cc:5028] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): tablet d883c107e7bd4b6daa6d09672acc2b23 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:29.225468  3118 raft_consensus.cc:2955] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:29.233203  3169 raft_consensus.cc:993] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:29.234575  3031 raft_consensus.cc:1081] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Signalling peer 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:29.236595  1569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:41274
I20260709 13:01:29.237221  1569 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:29.240896  1569 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.249457  3163 ts_tablet_manager.cc:1929] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:29.250005  3163 log.cc:1199] T 82743dc3081f4edba18591ca4b0d9c09 P 5184c101be204393adf3ac8b31f25ad6: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/wals/82743dc3081f4edba18591ca4b0d9c09
I20260709 13:01:29.252053  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet 82743dc3081f4edba18591ca4b0d9c09 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:29.256727  1566 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:29.260838  2301 raft_consensus.cc:2955] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:29.263293  1643 raft_consensus.cc:2955] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:29.264571  1569 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:29.267190  3129 raft_consensus.cc:2955] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:29.276834  1718 raft_consensus.cc:2955] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:29.268934  1567 raft_consensus.cc:2955] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:29.293947  1467 catalog_manager.cc:5697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:29.273065  2302 raft_consensus.cc:2955] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:29.297775  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:29.310827  1643 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.312022  1569 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:29.317284  1452 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5448ccc31ba14113b3d56168031c6714 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:29.333674  1643 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:29.340310  1641 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:29.351240  1710 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:29.353472  1462 catalog_manager.cc:5697] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 4 to 5, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 2 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:29.358227  3031 consensus_queue.cc:1059] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:29.359570  3169 raft_consensus.cc:3055] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:29.359988  3169 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:29.361166  3169 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:29.362571  3169 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:29.368789  2304 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:29.370057  1567 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:29.370761  2922 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:29.382014  1533 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:29.383302  3124 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:29.383867  3124 raft_consensus.cc:697] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:29.384759  3124 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:29.388999  2922 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:29.396570  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:29.409543  2922 raft_consensus.cc:2955] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:29.416579  1567 raft_consensus.cc:2955] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:29.426321  1554 tablet_service.cc:1558] Processing DeleteTablet for tablet 5448ccc31ba14113b3d56168031c6714 with delete_type TABLET_DATA_TOMBSTONED (TS 3f64b76180a54492bb015e405caeacec not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:41262
I20260709 13:01:29.430841  2301 raft_consensus.cc:2955] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:29.486567  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2a8ba0cc212d41e5b0ad586b885d07a7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:29.498839  3164 tablet_replica.cc:333] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:01:29.503135  3164 raft_consensus.cc:2243] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:29.503993  3164 raft_consensus.cc:2272] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:29.505095  1718 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:29.505486  1467 catalog_manager.cc:5697] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 2 to 3, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 3 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:29.509845  1464 catalog_manager.cc:5697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:29.516665  3164 ts_tablet_manager.cc:1916] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:29.528590  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a8ba0cc212d41e5b0ad586b885d07a7 with delete_type TABLET_DATA_TOMBSTONED (TS 5184c101be204393adf3ac8b31f25ad6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60370
I20260709 13:01:29.529662  3163 tablet_replica.cc:333] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:01:29.530900  3163 raft_consensus.cc:2243] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:29.531858  3163 raft_consensus.cc:2272] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:29.537750  3163 ts_tablet_manager.cc:1916] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:29.548467  3164 ts_tablet_manager.cc:1929] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:01:29.548974  3164 log.cc:1199] T 5448ccc31ba14113b3d56168031c6714 P 3f64b76180a54492bb015e405caeacec: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/wals/5448ccc31ba14113b3d56168031c6714
I20260709 13:01:29.550485  1451 catalog_manager.cc:5028] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): tablet 5448ccc31ba14113b3d56168031c6714 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:29.563340  3163 ts_tablet_manager.cc:1929] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:29.563799  3163 log.cc:1199] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 5184c101be204393adf3ac8b31f25ad6: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/wals/2a8ba0cc212d41e5b0ad586b885d07a7
I20260709 13:01:29.565256  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet 2a8ba0cc212d41e5b0ad586b885d07a7 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:29.621702  1643 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:29.622987  2922 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:29.634511  1567 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:29.636201  3118 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:29.645049  1718 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:29.646608  3124 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:29.677420  1643 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:29.679596  3124 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:29.740790   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:29.974076  1709 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:29.975344  3099 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:29.978375  1643 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:29.980502  3099 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:30.041754   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:30.081859  1567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:30.083060  1567 raft_consensus.cc:606] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:30.086539  1718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:30.087143  1718 raft_consensus.cc:606] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:30.089850  1567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f712f9c188e44e588e739161cac39330"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:30.090309  1567 raft_consensus.cc:606] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:30.091765   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:01:30.093670   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:30.125978  2922 raft_consensus.cc:993] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:30.126535  3118 raft_consensus.cc:1081] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:30.128441  1641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60390
I20260709 13:01:30.128894  1641 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:30.129150  1641 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:30.133857  1641 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:30.135751  1641 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:30.136895  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:30.137434  1718 raft_consensus.cc:3055] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:30.137694  1718 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:30.138295  1718 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:30.139217  1718 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:30.140116  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:30.140703  1567 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:30.146160  1567 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:30.150377  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:30.151708  3129 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:30.152225  3129 raft_consensus.cc:697] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:30.153133  3129 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:30.159600  1718 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:30.165342  1462 catalog_manager.cc:5697] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:30.334676  3099 raft_consensus.cc:993] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:30.335211  3124 raft_consensus.cc:1081] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Signalling peer 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:30.336705  1709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:36066
I20260709 13:01:30.337360  1709 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:30.337745  1709 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:30.343493  1709 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:30.345466  1709 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:30.345788  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:30.345971  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:30.346449  1567 raft_consensus.cc:3055] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:30.346652  2301 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:30.346807  1567 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:30.347666  1567 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:30.348778  1567 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:30.352360  2301 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:30.353646  1683 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:30.354439  3118 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:30.354677  1567 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:30.355016  3118 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:30.356052  3118 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:30.368126  1462 catalog_manager.cc:5697] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 2, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:30.531352  3128 raft_consensus.cc:993] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:30.531908  3124 raft_consensus.cc:1081] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Signalling peer 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:30.535741  1718 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f712f9c188e44e588e739161cac39330"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:36066
I20260709 13:01:30.536432  1718 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:30.536746  1718 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:30.542142  1718 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:30.544391  1718 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:30.544683  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:30.544972  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:30.545390  1569 raft_consensus.cc:3055] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:30.545691  1641 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:30.545869  1569 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:30.546708  1569 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:30.547896  1569 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:30.554481  1569 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:30.553545  1641 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:30.555935  1682 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:30.556691  2922 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:30.557132  2922 raft_consensus.cc:697] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:30.557891  2922 consensus_queue.cc:237] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:30.601646  1462 catalog_manager.cc:5697] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 2, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:30.618019  1515 auto_rebalancer.cc:629] Failed to schedule move for tablet f712f9c188e44e588e739161cac39330: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:01:30.621989  1718 consensus_queue.cc:237] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:30.628327  1641 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:30.628924  1567 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:30.628954  1683 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 84f86a4ba55242bd8d2459c6b9df56d2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:30.629612  2922 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:30.630528  3118 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:30.639799  3118 raft_consensus.cc:2955] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:30.641920  1641 raft_consensus.cc:2955] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:30.646368  1567 raft_consensus.cc:2955] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:30.658062  1462 catalog_manager.cc:5697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:30.663007  1709 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:30.680811  1641 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:01:30.681481  1683 consensus_peers.cc:597] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 250e1f9f055e4fda9987138306f2e0b7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:30.681964  2922 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:30.683576  1567 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:30.684983  2922 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:30.693028  3118 raft_consensus.cc:2955] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:30.694779  1641 raft_consensus.cc:2955] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:30.699177  1567 raft_consensus.cc:2955] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:30.713315  1569 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:30.718364  1464 catalog_manager.cc:5697] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 4, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:30.745100  1561 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:01:30.752621  1534 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1e353d5ccbb3478ba62d1d60ecc3ec05. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:30.752939  1709 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:30.754347  3124 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:30.755091  3074 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:30.759536  1641 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:30.762804  3124 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:30.771083  3128 raft_consensus.cc:2955] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:30.778713  1709 raft_consensus.cc:2955] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:30.784644  1718 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:30.791746  3129 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:30.799608  1641 raft_consensus.cc:2955] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:30.813588  1467 catalog_manager.cc:5697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:30.843712  1569 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:30.845383  2922 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:30.969493  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 129404us
I20260709 13:01:30.985524  2302 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:30.997643  3118 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:31.066166  1641 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:31.072489  2922 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:31.095618   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:31.105195  1566 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:31.118242  3191 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:31.149457  3219 ts_tablet_manager.cc:933] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:31.186264  3219 tablet_copy_client.cc:323] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:31.468572  1727 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet 84f86a4ba55242bd8d2459c6b9df56d2 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:31.469254  1727 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet 84f86a4ba55242bd8d2459c6b9df56d2 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-84f86a4ba55242bd8d2459c6b9df56d2
I20260709 13:01:31.547427  1727 tablet_copy_source_session.cc:215] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:01:31.590629  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 85812.6us
W20260709 13:01:31.639911  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 65705.1us
I20260709 13:01:31.683467  3237 ts_tablet_manager.cc:933] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
W20260709 13:01:31.716363  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 102106us
I20260709 13:01:31.723598  3246 ts_tablet_manager.cc:933] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:31.732296  3237 tablet_copy_client.cc:323] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:36547
I20260709 13:01:31.744969  3246 tablet_copy_client.cc:323] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:31.759305  1579 tablet_copy_service.cc:140] P 3f64b76180a54492bb015e405caeacec: Received BeginTabletCopySession request for tablet 1e353d5ccbb3478ba62d1d60ecc3ec05 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:35222)
I20260709 13:01:31.759491  1727 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet 250e1f9f055e4fda9987138306f2e0b7 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:31.759896  1579 tablet_copy_service.cc:161] P 3f64b76180a54492bb015e405caeacec: Beginning new tablet copy session on tablet 1e353d5ccbb3478ba62d1d60ecc3ec05 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:35222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-1e353d5ccbb3478ba62d1d60ecc3ec05
I20260709 13:01:31.760061  1727 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet 250e1f9f055e4fda9987138306f2e0b7 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-250e1f9f055e4fda9987138306f2e0b7
I20260709 13:01:31.766724  1727 tablet_copy_source_session.cc:215] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:31.767551  1579 tablet_copy_source_session.cc:215] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:31.781900  3237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e353d5ccbb3478ba62d1d60ecc3ec05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.793952  3219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84f86a4ba55242bd8d2459c6b9df56d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.805145  3237 tablet_copy_client.cc:806] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:31.805946  3237 tablet_copy_client.cc:670] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:31.812429  3246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 250e1f9f055e4fda9987138306f2e0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:31.825456  3237 tablet_copy_client.cc:538] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:31.839108  3237 tablet_bootstrap.cc:492] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:31.843171  3219 tablet_copy_client.cc:806] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:31.843918  3219 tablet_copy_client.cc:670] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:31.848687  3246 tablet_copy_client.cc:806] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:31.849399  3246 tablet_copy_client.cc:670] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:31.851584  3219 tablet_copy_client.cc:538] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:31.853821  3246 tablet_copy_client.cc:538] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:31.916492  3246 tablet_bootstrap.cc:492] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:31.917232  3219 tablet_bootstrap.cc:492] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:31.918450  3237 tablet_bootstrap.cc:492] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:31.920087  3237 tablet_bootstrap.cc:492] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:31.927789  3237 ts_tablet_manager.cc:1403] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.089s	user 0.048s	sys 0.018s
I20260709 13:01:31.931205  3237 raft_consensus.cc:359] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:31.932178  3237 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:31.933256  3237 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:31.952991  3237 ts_tablet_manager.cc:1434] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:01:31.958776  1579 tablet_copy_service.cc:342] P 3f64b76180a54492bb015e405caeacec: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-1e353d5ccbb3478ba62d1d60ecc3ec05 received from {username='slave'} at 127.0.0.1:35222
I20260709 13:01:31.959781  1579 tablet_copy_service.cc:434] P 3f64b76180a54492bb015e405caeacec: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-1e353d5ccbb3478ba62d1d60ecc3ec05 on tablet 1e353d5ccbb3478ba62d1d60ecc3ec05 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:32.010161  3219 tablet_bootstrap.cc:492] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:32.012060  3219 tablet_bootstrap.cc:492] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:32.013381  3219 ts_tablet_manager.cc:1403] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.096s	user 0.027s	sys 0.034s
I20260709 13:01:32.020274  3219 raft_consensus.cc:359] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:32.021553  3219 raft_consensus.cc:740] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:32.022334  3219 consensus_queue.cc:260] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:32.026484  3219 ts_tablet_manager.cc:1434] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:01:32.028650  1727 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-84f86a4ba55242bd8d2459c6b9df56d2 received from {username='slave'} at 127.0.0.1:33280
I20260709 13:01:32.029255  1727 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-84f86a4ba55242bd8d2459c6b9df56d2 on tablet 84f86a4ba55242bd8d2459c6b9df56d2 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:32.072795  3246 tablet_bootstrap.cc:492] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:32.074277  3246 tablet_bootstrap.cc:492] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:32.075559  3246 ts_tablet_manager.cc:1403] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.159s	user 0.045s	sys 0.042s
I20260709 13:01:32.079329  3246 raft_consensus.cc:359] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:32.080339  3246 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:32.081087  3246 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:32.084650  3246 ts_tablet_manager.cc:1434] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:01:32.087296  1727 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-250e1f9f055e4fda9987138306f2e0b7 received from {username='slave'} at 127.0.0.1:33280
I20260709 13:01:32.088065  1727 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-250e1f9f055e4fda9987138306f2e0b7 on tablet 250e1f9f055e4fda9987138306f2e0b7 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:32.145733  2304 raft_consensus.cc:1217] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:01:32.247427  2304 raft_consensus.cc:1217] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:01:32.349660  2301 raft_consensus.cc:1217] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
W20260709 13:01:32.390519  2326 debug-util.cc:398] Leaking SignalData structure 0x7b08002ec020 after lost signal to thread 434
W20260709 13:01:32.391700  2326 debug-util.cc:398] Leaking SignalData structure 0x7b08002ec700 after lost signal to thread 1501
W20260709 13:01:32.392930  2326 debug-util.cc:398] Leaking SignalData structure 0x7b080036b2a0 after lost signal to thread 1593
W20260709 13:01:32.394062  2326 debug-util.cc:398] Leaking SignalData structure 0x7b080023a200 after lost signal to thread 1668
W20260709 13:01:32.395274  2326 debug-util.cc:398] Leaking SignalData structure 0x7b080031bb80 after lost signal to thread 1742
W20260709 13:01:32.396391  2326 debug-util.cc:398] Leaking SignalData structure 0x7b080014d340 after lost signal to thread 2329
I20260709 13:01:32.475777   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:32.531432  3240 raft_consensus.cc:1064] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:32.534264  3240 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:32.604838  1641 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:32.607167  2304 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:32.611670  3118 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:32.615105  1641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:60670
I20260709 13:01:32.615866  1641 raft_consensus.cc:606] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:32.617799  3241 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:32.628351  1718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:32.629006  1718 raft_consensus.cc:606] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:32.655573  1561 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:32.657442  3221 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:32.658664  1718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f712f9c188e44e588e739161cac39330"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:32.659430  1718 raft_consensus.cc:606] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:01:32.667322   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:01:32.669099   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:32.726847  3118 raft_consensus.cc:2955] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:32.731073  1561 raft_consensus.cc:2955] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:32.737581  2303 raft_consensus.cc:2955] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:32.745977  1642 raft_consensus.cc:2955] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:32.758664  1467 catalog_manager.cc:5697] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 4 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:32.764977  3241 raft_consensus.cc:1064] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:32.768306  3241 consensus_queue.cc:237] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:32.786329  1641 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:32.788367  3221 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:32.799363  3129 raft_consensus.cc:993] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: : Instructing follower 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:32.799955  3129 raft_consensus.cc:1081] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Signalling peer 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:01:32.804563  1718 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:36078
I20260709 13:01:32.805439  1718 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:32.805970  1718 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:32.808636  3240 raft_consensus.cc:993] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:32.809208  3240 raft_consensus.cc:1081] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:32.813760  2303 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53448
I20260709 13:01:32.814390  2303 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:32.814772  2303 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:32.815137  1718 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:01:32.831611  2303 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:32.834964  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:32.836182  1568 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:32.845849  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:32.847493  1641 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:32.848268  1714 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:32.835450  1718 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:32.855949  1568 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:32.857813  1641 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:32.863373  1681 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:32.864270  3240 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:32.864737  3240 raft_consensus.cc:697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:32.865629  3240 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:01:32.867727  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:32.871220  3187 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:32.871817  3187 raft_consensus.cc:697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:32.872782  3187 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:32.876183  2304 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:32.881649  3118 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:32.861734  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:32.891710  1718 raft_consensus.cc:3055] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:32.893565  1718 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:32.862264  2303 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:32.915668  1466 catalog_manager.cc:5697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:32.910081  1718 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:32.928797  1718 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:32.929292  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:32.932335  1643 raft_consensus.cc:3055] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:32.932682  1643 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:32.930954  3221 raft_consensus.cc:2955] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:32.933537  1643 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:01:32.934835  1643 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:32.942895  1643 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:32.949261  2303 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:32.959192  3265 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:32.963963  1718 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:32.975855  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 250e1f9f055e4fda9987138306f2e0b7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:32.980719  1464 catalog_manager.cc:5697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 2, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:32.996963  2304 raft_consensus.cc:2955] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.027410  1644 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:33.027230  1466 catalog_manager.cc:5697] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 5 to 6, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:01:33.051079  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 64826.5us
W20260709 13:01:33.053555  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 85971.8us
I20260709 13:01:33.059021  1554 tablet_service.cc:1558] Processing DeleteTablet for tablet 250e1f9f055e4fda9987138306f2e0b7 with delete_type TABLET_DATA_TOMBSTONED (TS 3f64b76180a54492bb015e405caeacec not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41262
I20260709 13:01:33.064313  3268 raft_consensus.cc:2955] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.075752  2303 raft_consensus.cc:2955] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.091164  3221 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:33.096076  3276 tablet_replica.cc:333] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:01:33.097172  3276 raft_consensus.cc:2243] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:33.097896  3276 raft_consensus.cc:2272] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:01:33.098722  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 152945us
I20260709 13:01:33.109043  1561 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:01:33.113566  3266 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:33.114145  3265 raft_consensus.cc:993] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:33.114987  3265 raft_consensus.cc:1081] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Signalling peer 3f64b76180a54492bb015e405caeacec to start an election
I20260709 13:01:33.118024  1561 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f712f9c188e44e588e739161cac39330"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:41274
I20260709 13:01:33.118595  1561 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:33.119001  1561 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:33.111951  3276 ts_tablet_manager.cc:1916] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:33.131175  1561 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:33.135962  1643 raft_consensus.cc:2955] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.145746  1644 raft_consensus.cc:2955] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.151963  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:33.152626  1642 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:33.184088  3276 ts_tablet_manager.cc:1929] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:01:33.184680  3276 log.cc:1199] T 250e1f9f055e4fda9987138306f2e0b7 P 3f64b76180a54492bb015e405caeacec: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-0-root/wals/250e1f9f055e4fda9987138306f2e0b7
I20260709 13:01:33.186457  1451 catalog_manager.cc:5028] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): tablet 250e1f9f055e4fda9987138306f2e0b7 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:33.191918  1642 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:33.198974  3206 raft_consensus.cc:1064] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:33.217676  3206 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:33.220295  1568 raft_consensus.cc:2955] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.228981  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:33.229609  1718 raft_consensus.cc:3055] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:33.229887  1718 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:33.210218  1561 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:33.232890  1533 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:33.230600  1718 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:33.233651  3272 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:33.234279  1718 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:33.234289  3272 raft_consensus.cc:697] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:33.235813  3272 consensus_queue.cc:237] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:33.242432  2304 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:33.243181  1643 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:33.243341  1464 catalog_manager.cc:5697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 2 to 3, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.246513  1718 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:33.246438  3274 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:33.250578  1712 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:01:33.250974  3128 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:33.255105  3206 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:33.287703  1638 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:33.290230  3267 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:33.301905  1464 catalog_manager.cc:5697] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 2 to 3, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 3 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:33.303611  3272 raft_consensus.cc:2955] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.309306  1643 raft_consensus.cc:2955] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.313822  1712 raft_consensus.cc:2955] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.331156  2304 raft_consensus.cc:2955] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.334486  1561 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:33.337111  3266 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:33.339602  1638 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:33.343578  3187 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:33.361550  1464 catalog_manager.cc:5697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 3 to 4, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.402268  1712 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:33.404878  3233 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:33.420976  1563 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:33.428406  2304 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:33.428406  1644 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:33.430029  3274 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:33.432013  3274 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:33.442468  3128 raft_consensus.cc:2955] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.445569  2304 raft_consensus.cc:2955] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.457993  1643 raft_consensus.cc:2955] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.494783  1466 catalog_manager.cc:5697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 4 to 5, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.518189  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1e353d5ccbb3478ba62d1d60ecc3ec05 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:33.539000  1699 tablet_service.cc:1558] Processing DeleteTablet for tablet 1e353d5ccbb3478ba62d1d60ecc3ec05 with delete_type TABLET_DATA_TOMBSTONED (TS 28bc59d37f664ce0a6f5784f2619f0b5 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:36054
I20260709 13:01:33.543601  1712 consensus_queue.cc:237] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:33.554473  2304 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:33.556686  3265 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:33.568181  3294 tablet_replica.cc:333] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:01:33.569387  3294 raft_consensus.cc:2243] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:33.571055  3266 raft_consensus.cc:2955] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.573961  3294 raft_consensus.cc:2272] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:33.574009  1561 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:01:33.576445  3265 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:33.581779  3294 ts_tablet_manager.cc:1916] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:33.580885  1561 raft_consensus.cc:2955] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.577744  2304 raft_consensus.cc:2955] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.602438  3294 ts_tablet_manager.cc:1929] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:01:33.603113  3294 log.cc:1199] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/wals/1e353d5ccbb3478ba62d1d60ecc3ec05
I20260709 13:01:33.605108  1451 catalog_manager.cc:5028] TS 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): tablet 1e353d5ccbb3478ba62d1d60ecc3ec05 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:33.609163  1462 catalog_manager.cc:5697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 3 to 4, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:01:33.611358  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 84f86a4ba55242bd8d2459c6b9df56d2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:01:33.620966  1712 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:33.622810  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet 84f86a4ba55242bd8d2459c6b9df56d2 with delete_type TABLET_DATA_TOMBSTONED (TS 5184c101be204393adf3ac8b31f25ad6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60370
I20260709 13:01:33.623438  3128 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:01:33.639070  3295 tablet_replica.cc:333] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:01:33.640451  3295 raft_consensus.cc:2243] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:33.641613  3295 raft_consensus.cc:2272] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:33.644276  1643 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:33.652390  3295 ts_tablet_manager.cc:1916] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:01:33.652695  3206 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:33.672055   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:33.695079  3295 ts_tablet_manager.cc:1929] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:01:33.700577  3295 log.cc:1199] T 84f86a4ba55242bd8d2459c6b9df56d2 P 5184c101be204393adf3ac8b31f25ad6: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/wals/84f86a4ba55242bd8d2459c6b9df56d2
I20260709 13:01:33.712273  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet 84f86a4ba55242bd8d2459c6b9df56d2 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:01:34.716761  1566 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:34.732095  1718 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:01:34.733763  1534 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ca83cd5e36eb4e22b75011b87bfcd49d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:34.737573  3206 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:34.752221  3206 raft_consensus.cc:2955] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:34.754381  1642 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:34.754786  1718 raft_consensus.cc:2955] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:34.765291  3275 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:34.776829  1644 consensus_queue.cc:237] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:34.786190  1466 catalog_manager.cc:5697] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 4, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 3 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:34.800843  1642 raft_consensus.cc:2955] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:34.805670  1712 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:01:34.810521  1566 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:01:34.811487  1609 consensus_peers.cc:597] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3f22f7f5cede49bea2083e8905932fb7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:34.820811  3225 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:34.822853  3225 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:34.843140  3129 raft_consensus.cc:2955] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:34.852448  1566 raft_consensus.cc:2955] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:34.912415  1718 raft_consensus.cc:2955] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:34.930971   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:34.932586  1712 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:34.952132  1642 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:01:34.953545  1683 consensus_peers.cc:597] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fb294ac919b44e56960745f79e83dfc4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:34.953699  3300 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:34.964720  1568 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:34.980957  3300 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:34.992552  1466 catalog_manager.cc:5697] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 2, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:35.004284  3267 raft_consensus.cc:2955] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:35.006039  1638 raft_consensus.cc:2955] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:35.013128  1568 raft_consensus.cc:2955] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:35.030264  1642 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:35.037043  1466 catalog_manager.cc:5697] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
W20260709 13:01:35.043344  1609 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d59c4d21b6cb46edbfc2b3d50b51812a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:35.044551  1568 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:35.046271  3129 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:35.047847  1718 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:35.049451  3311 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:35.073031  3225 raft_consensus.cc:2955] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:35.095763  1568 raft_consensus.cc:2955] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:35.105269  1561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:01:35.106017  1561 raft_consensus.cc:606] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:35.111129  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:35.111732  1709 raft_consensus.cc:606] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:35.117662   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:01:35.119645   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:35.120752  1466 catalog_manager.cc:5697] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:35.129056  3320 ts_tablet_manager.cc:933] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547)
I20260709 13:01:35.145574  3320 tablet_copy_client.cc:323] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:36547
I20260709 13:01:35.147639  1579 tablet_copy_service.cc:140] P 3f64b76180a54492bb015e405caeacec: Received BeginTabletCopySession request for tablet ca83cd5e36eb4e22b75011b87bfcd49d from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:35222)
I20260709 13:01:35.148214  1579 tablet_copy_service.cc:161] P 3f64b76180a54492bb015e405caeacec: Beginning new tablet copy session on tablet ca83cd5e36eb4e22b75011b87bfcd49d from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:35222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-ca83cd5e36eb4e22b75011b87bfcd49d
I20260709 13:01:35.156176  1579 tablet_copy_source_session.cc:215] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:35.172933  3320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca83cd5e36eb4e22b75011b87bfcd49d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.176018  1712 raft_consensus.cc:2955] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:35.208629  3206 raft_consensus.cc:993] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:35.209120  3206 raft_consensus.cc:1081] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:01:35.212275  2301 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53464
I20260709 13:01:35.212847  2301 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:35.213196  2301 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:35.224896  2301 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:35.228516  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:35.229226  1568 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:35.229528  1568 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:35.229961  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:35.230844  1642 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:35.230296  1568 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:35.236966  1568 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:35.248425  2301 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:35.251852  1642 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:35.253614  2269 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:35.254483  3302 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:35.254968  3302 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:35.255880  3302 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:35.258726  1568 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
W20260709 13:01:35.267310  1739 debug-util.cc:398] Leaking SignalData structure 0x7b0800096f00 after lost signal to thread 434
W20260709 13:01:35.268477  1739 debug-util.cc:398] Leaking SignalData structure 0x7b08001e89c0 after lost signal to thread 1501
W20260709 13:01:35.270033  1739 debug-util.cc:398] Leaking SignalData structure 0x7b080009d520 after lost signal to thread 1593
W20260709 13:01:35.271314  1739 debug-util.cc:398] Leaking SignalData structure 0x7b0800162a20 after lost signal to thread 1668
W20260709 13:01:35.272526  1739 debug-util.cc:398] Leaking SignalData structure 0x7b08002cd860 after lost signal to thread 1742
W20260709 13:01:35.273846  1739 debug-util.cc:398] Leaking SignalData structure 0x7b080036cae0 after lost signal to thread 2329
I20260709 13:01:35.278590  3320 tablet_copy_client.cc:806] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:35.279458  3320 tablet_copy_client.cc:670] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:35.283676  3320 tablet_copy_client.cc:538] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:35.305225  3320 tablet_bootstrap.cc:492] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:35.319502  1462 catalog_manager.cc:5697] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 2, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:35.378970  3320 tablet_bootstrap.cc:492] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:35.380084  3320 tablet_bootstrap.cc:492] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:35.380861  3320 ts_tablet_manager.cc:1403] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.076s	user 0.033s	sys 0.010s
I20260709 13:01:35.384331  3320 raft_consensus.cc:359] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:35.385306  3320 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:35.386054  3320 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:35.389081  3320 ts_tablet_manager.cc:1434] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:01:35.389935  3323 ts_tablet_manager.cc:933] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:35.391677  3325 ts_tablet_manager.cc:933] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:35.394316  1579 tablet_copy_service.cc:342] P 3f64b76180a54492bb015e405caeacec: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-ca83cd5e36eb4e22b75011b87bfcd49d received from {username='slave'} at 127.0.0.1:35222
I20260709 13:01:35.394845  1579 tablet_copy_service.cc:434] P 3f64b76180a54492bb015e405caeacec: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-ca83cd5e36eb4e22b75011b87bfcd49d on tablet ca83cd5e36eb4e22b75011b87bfcd49d with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:35.397356  3323 tablet_copy_client.cc:323] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:01:35.409754  3325 tablet_copy_client.cc:323] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:40023
I20260709 13:01:35.411558  1654 tablet_copy_service.cc:140] P 5184c101be204393adf3ac8b31f25ad6: Received BeginTabletCopySession request for tablet 3f22f7f5cede49bea2083e8905932fb7 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:53222)
I20260709 13:01:35.412094  1654 tablet_copy_service.cc:161] P 5184c101be204393adf3ac8b31f25ad6: Beginning new tablet copy session on tablet 3f22f7f5cede49bea2083e8905932fb7 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:53222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-3f22f7f5cede49bea2083e8905932fb7
I20260709 13:01:35.418525  1654 tablet_copy_source_session.cc:215] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:35.423121  3325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f22f7f5cede49bea2083e8905932fb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:35.424355  3300 raft_consensus.cc:993] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:35.424914  3300 raft_consensus.cc:1081] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:35.436434  1642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60390
I20260709 13:01:35.437047  1642 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:35.437433  1642 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:35.441339  3325 tablet_copy_client.cc:806] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:35.442097  3325 tablet_copy_client.cc:670] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:35.446514  3325 tablet_copy_client.cc:538] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:35.469251  3320 ts_tablet_manager.cc:933] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:35.477357  3320 tablet_copy_client.cc:323] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:40023
I20260709 13:01:35.479292  1654 tablet_copy_service.cc:140] P 5184c101be204393adf3ac8b31f25ad6: Received BeginTabletCopySession request for tablet d59c4d21b6cb46edbfc2b3d50b51812a from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:53222)
I20260709 13:01:35.479877  1654 tablet_copy_service.cc:161] P 5184c101be204393adf3ac8b31f25ad6: Beginning new tablet copy session on tablet d59c4d21b6cb46edbfc2b3d50b51812a from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:53222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-d59c4d21b6cb46edbfc2b3d50b51812a
I20260709 13:01:35.482259  3325 tablet_bootstrap.cc:492] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:35.539443  1654 tablet_copy_source_session.cc:215] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:35.542403  1727 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet fb294ac919b44e56960745f79e83dfc4 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
I20260709 13:01:35.543669  1642 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:35.691753  1642 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:35.692832  1727 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet fb294ac919b44e56960745f79e83dfc4 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-fb294ac919b44e56960745f79e83dfc4
I20260709 13:01:36.774463   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:36.933880  3344 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:36.935261  3341 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:36.935487  3344 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:36.936264  3341 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:36.938961  3344 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:36.944137  3341 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:36.956696  3339 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:36.957535  3339 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:36.954111  3129 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:36.961833  3339 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:36.962060  3129 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:36.981329  3129 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:36.991976  3345 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:36.993547  3339 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:36.994375  3339 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:36.995477  3345 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:36.995788  3344 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.003592  3349 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.004761  3349 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.000033  3344 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.002821  3345 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.003052  3341 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.015086  3347 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.048872  3348 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.057353  3350 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.057574  3348 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.058226  3350 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.062193  3350 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.062815  3346 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.042483  3345 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.063696  3346 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.044409  3343 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.048977  3349 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.066062  3347 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:37.055554  3341 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.070403  3343 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:37.080489  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.388s	user 0.010s	sys 0.004s
I20260709 13:01:37.088079  3344 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.088762  3267 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:01:37.081526  1498 debug-util.cc:398] Leaking SignalData structure 0x7b0800034420 after lost signal to thread 434
I20260709 13:01:37.089666  3267 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:37.091293  1498 debug-util.cc:398] Leaking SignalData structure 0x7b080018fcc0 after lost signal to thread 1501
I20260709 13:01:37.092417  3267 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.093533  3354 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.095351  3349 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.096926  3349 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:37.097602  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.405s	user 0.011s	sys 0.005s
I20260709 13:01:37.097898  3347 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:37.096005  3341 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.104068  3346 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.104192  3343 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.110999  3341 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.111198  3339 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.111853  3341 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.112779  3347 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.113540  3347 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.114317  3341 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.115103  3345 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:37.115839  3347 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.124337  3346 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.125073  3346 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.127058  3352 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.136197  3348 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.117640  3345 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.130529  3343 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.139691  3348 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.139951  3343 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.140560  3348 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.117679  3267 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.130236  3349 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.143131  3267 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.120332  3350 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.136930  3352 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.147931  1727 tablet_copy_source_session.cc:215] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:01:37.117962  3347 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.158278  3342 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.145968  3341 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.162550  3350 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.174001  3351 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.174548  1727 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:33280 (request call id 330) took 1633 ms. Trace:
I20260709 13:01:37.178143  3359 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.179090  3359 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.175043  3351 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.187029  3359 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.196426  3361 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.197284  3361 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.189105  3351 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.200011  3362 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.200867  3357 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.201244  3343 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.201579  3348 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.201769  3357 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.202461  3267 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.203156  3267 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.157155  3354 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.203881  3267 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.200315  3361 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.205991  3354 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.205999  3267 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.183629  3349 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.209645  3355 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.210196  3349 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.210479  3355 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.177079  3346 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.215175  3349 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.216418  3349 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.217053  3349 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.219313  3347 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.209671  3339 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.221019  3349 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.222043  3347 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.221728  3339 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:37.177804  3345 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.226532  3352 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:37.227133  3352 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.227815  3352 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.229985  3352 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.159196  3342 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.223513  3345 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.239543  3342 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.222755  3347 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.242057  3358 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:01:37.249686  1610 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.558s	user 0.018s	sys 0.000s
W20260709 13:01:37.250106  1610 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.559s	user 0.018s	sys 0.000s
I20260709 13:01:37.213622  3341 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.249475  3358 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.248891  3349 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.262703  3349 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:37.265776  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.427s	user 0.001s	sys 0.013s
W20260709 13:01:37.277192  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.439s	user 0.002s	sys 0.013s
I20260709 13:01:37.277336  3255 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.243902  3346 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.250789  3343 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.175405  1727 rpcz_store.cc:276] 0709 13:01:35.541211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.542266 (+  1055us) service_pool.cc:224] Handling call
0709 13:01:37.174335 (+1632069us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:37.278239  3255 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.264861  3358 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.301851  3360 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.307763  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.252437  3342 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:01:37.311110  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.472s	user 0.009s	sys 0.007s
I20260709 13:01:37.263675  3347 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.317315  3302 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.322129  3266 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.394878  3364 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.200881  3362 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.407686  1714 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:01:37.408187  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.409107  1710 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:01:37.417078  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:37.417789  1710 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:37.418139  1710 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:37.418568  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.419301  1710 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:37.421015  1710 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:37.422156  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.345341  3349 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.431707  3343 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.346125  3341 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.319758  3352 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.266683  3361 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:01:37.437052  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.598s	user 0.009s	sys 0.007s
I20260709 13:01:37.345506  3355 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.433287  1710 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:37.403666  3267 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.454588  3255 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:37.455304  3361 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.455143  3302 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.456933  3345 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.459146  3347 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:37.461530  2270 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.611s	user 0.010s	sys 0.007s
I20260709 13:01:37.461803  3352 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.462241  3266 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.468763  3351 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.435237  3360 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.403450  3342 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.470777  1644 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:01:37.474969  2270 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.626s	user 0.012s	sys 0.007s
I20260709 13:01:37.471666  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.461551  3339 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.448683  3369 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.399119  3346 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.468539  3267 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:37.487021  1717 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:37.488478  3351 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.490025  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.461596  3357 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:37.491598  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.479360  3347 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.476586  3255 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.496125  3361 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.497674  3364 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.540724  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:37.542959  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:37.543643  1642 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 1.
I20260709 13:01:37.511452  3357 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.547387  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.557547  3368 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.557772  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.558454  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.561954  3356 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.562557  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.566212  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.567945  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.569382  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.569839  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.570549  1565 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:01:37.572474  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.572630  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.573194  1563 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:37.542209  3255 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.574256  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.578526  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.547564  3347 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.558382  3368 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.590319  3373 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.587956  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.565073  3357 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.595335  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.592598  1566 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:01:37.522349  3369 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.591215  3373 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.595204  3302 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.606106  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.606503  3366 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.606804  3342 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.609560  1716 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:37.598304  3343 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.610857  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.612675  3370 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.600586  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.585964  3356 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.613427  3370 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.616511  3361 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:37.619303  3365 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.642271  3267 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:37.612030  3371 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.643069  3380 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.647610  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.624212  3356 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:37.662462  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.629959  3368 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.672961  3364 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.607970  3366 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.674252  1718 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:37.675566  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.677812  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:37.678038  3370 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.678603  3378 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.678696  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:37.669286  3379 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.665129  3381 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.677323  1569 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:01:37.679245  3378 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.683832  3376 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.671528  3361 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:37.682519  3340 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:37.681670  3342 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:37.692951  3361 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:37.693295  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.697065  1569 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:01:37.699061  3369 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:37.679857  3365 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.684499  3376 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.705276  3376 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:37.700906  1532 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:37.708683  3373 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.691886  3255 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.696074  3342 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.698038  3385 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:01:37.707000  1498 debug-util.cc:398] Leaking SignalData structure 0x7b0800352d00 after lost signal to thread 1593
I20260709 13:01:37.700605  3340 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.706576  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.679620  3371 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.720978  3302 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.689961  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.706161  3368 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.709460  3356 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.715353  3342 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:37.717406  3344 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:37.719318  3340 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:37.735877  3330 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.736238  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.736922  1714 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:01:37.722729  3357 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.718150  3365 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.727397  3376 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.694037  3380 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.734869  3344 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.725114  3385 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:37.750361  1532 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:37.737179  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.753247  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.737681  3373 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.754688  1714 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:37.712828  3384 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.745214  1715 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:37.762850  1532 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:37.711724  3381 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.753455  3344 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:37.737543  3302 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.709594  3379 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.742070  3356 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.736678  3330 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.767634  3255 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.752880  3365 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.760263  3339 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:37.774355  3382 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.775787  1567 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:37.774897  3381 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.777324  3381 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.753611  3368 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.778155  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.768726  3343 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:37.781037  1714 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:01:37.782688  1532 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:37.759009  3376 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.796954  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.799875  1532 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:37.766280  3366 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.775452  3339 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.764623  3373 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.770792  3378 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.775012  3382 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.791414  3343 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.802166  3345 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:37.777992  3381 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.805575  3255 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.806406  3339 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:37.823415  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.802488  3365 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.822012  3343 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:37.825258  3378 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.826370  3392 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:37.824291  3345 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.832801  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:37.826439  3302 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.814858  3382 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.819406  3379 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.809571  3385 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.776388  3384 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:37.783710  3390 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.811615  3383 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.865895  3345 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:37.879595  3386 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:01:37.880149  1683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.192s	user 0.010s	sys 0.004s
I20260709 13:01:37.880036  3380 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:37.881194  3381 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.865388  3366 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.885421  3371 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.886616  3385 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.868577  3392 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.897126  3385 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:37.902151  1683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.214s	user 0.011s	sys 0.004s
I20260709 13:01:37.892817  1715 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:37.879606  3387 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.907477  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.908953  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:37.912602  1567 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:37.911602  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.918381  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.919130   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:37.919633  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.920703  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.921445  3373 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.921648  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.922420  3368 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:37.923198  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:37.924072  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.918423  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.922624  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.875468  3378 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:01:37.887480  3389 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.895048  3382 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.913705  3301 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:37.948259  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.929582  3356 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.947490  3255 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.953794  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.956454  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:37.959182  3325 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 2.461s	user 0.041s	sys 0.025s
I20260709 13:01:37.960536  1565 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:37.959592  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:37.951931  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:37.976048  1567 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:37.972700  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:37.936566  3388 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:37.952311  3301 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:37.913079  3390 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.943524  3366 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.936734  3386 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.977705  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:37.971308  3325 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 2.473s	user 0.042s	sys 0.025s
I20260709 13:01:37.969044  3376 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.952847  3382 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.988049  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:37.988662  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:37.972950  3374 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.962739  3387 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.982363  3301 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:37.943241  3383 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:37.985440  3365 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:37.991262  3368 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:37.989598  3330 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:38.009760  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:37.970569  3389 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:38.036356  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.040364  1642 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:38.030529  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.025139  2299 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:38.024310  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:38.032835  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 48890.2us
I20260709 13:01:38.071084  3366 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:38.033740  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 59948.7us
I20260709 13:01:38.073493  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:38.033665  3330 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:38.085398  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.085913  2305 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:01:38.025913  3344 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:38.086344  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:38.088627  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.092829  1643 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:38.097942  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.098827  1643 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:38.005475  3388 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:38.111761  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.110420  2298 raft_consensus.cc:1217] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:01:38.097199  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.090721  3378 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:38.035779  3374 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:38.130045  3394 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:38.118198  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.119328  2301 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:01:38.089596  3368 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:38.117432  3366 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:38.164006  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:38.185987  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 99734us
I20260709 13:01:38.140353  2269 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:38.209913  2303 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:01:38.162150  3267 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:38.216076  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.216719  3383 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:38.216447  3267 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:38.218983  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 87839.5us
I20260709 13:01:38.139605  3357 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:38.131481  1560 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:38.231556  3367 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:38.233222  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.201709  3386 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:38.177460  3389 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:38.211490  3390 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:38.181226  3395 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:38.203018  3382 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:38.122820  3376 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:38.213760  1681 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:38.248360  3395 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:38.262001  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.260460  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.165270  3388 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:38.239249  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.247372  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1369) took 1005 ms. Trace:
I20260709 13:01:38.208463  3394 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.181538  3387 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:38.267879  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.230161  3383 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:38.269328  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.227588  3357 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:38.232403  3367 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:38.271325  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.460233  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.460427  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.461436  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.461582  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.421984  3344 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:38.462898  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.463559  2303 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:01:38.463678  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:38.268580  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 54895us
I20260709 13:01:38.284313  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.348590  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.465226  2301 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:01:38.467862  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.468291  1638 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 1.
I20260709 13:01:38.491626  1565 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 1.
I20260709 13:01:38.492561  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.464671  3383 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:01:38.469327  3339 consensus_meta.cc:287] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.609s	user 0.005s	sys 0.006s
W20260709 13:01:38.421708  3325 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:01:38.469578  3267 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:38.500506  3339 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:38.495142  3383 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:38.361493  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.470153  3394 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:38.465749  1709 rpcz_store.cc:276] 0709 13:01:37.245298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.248713 (+  3415us) service_pool.cc:224] Handling call
0709 13:01:37.344607 (+ 95894us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.416353 (+ 71746us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.416387 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:37.416453 (+    66us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:37.416493 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.246986 (+830493us) spinlock_profiling.cc:243] Waited 64.7 ms on lock 0x7b5400239954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:38.247034 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.247170 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148873088}
I20260709 13:01:38.471477  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.472144  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:38.472339  3340 consensus_meta.cc:287] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.721s	user 0.013s	sys 0.004s
I20260709 13:01:38.510648  1563 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:38.473423  3301 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:38.476197  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.250905  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.286423  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.349040  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.298939  2297 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:01:38.339000  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.283136  3402 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:38.321995  3376 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:38.466485  3357 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:38.386006  3343 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.515s	user 0.004s	sys 0.007s
I20260709 13:01:38.320537  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.381234  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:38.492666  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.495816  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.496941  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.497750  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.495254  2269 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:38.524508  3360 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:38.524674  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.518879  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.525952  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.526610  2299 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:38.527135  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.527596  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.528409  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.496862  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.514693  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.528908  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.535756  1533 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:38.536653  3399 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:38.537266  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.537783  3301 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:38.538347  3367 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:38.539119  1717 raft_consensus.cc:606] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:38.538662  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.511729  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.542351  2303 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
W20260709 13:01:38.494450  3342 consensus_meta.cc:287] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.746s	user 0.015s	sys 0.004s
I20260709 13:01:38.480448  3352 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:38.514832  3340 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:38.515290  3325 tablet_bootstrap.cc:492] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:38.513223  3388 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:38.520277  3402 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:38.522357  3368 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:38.522440  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.521024  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:38.381170  3374 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:38.528970  1641 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:38.540102  3339 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:38.583467  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.585081  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:38.593607  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 85863.1us
I20260709 13:01:38.614986  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.550560  3366 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:38.614143  2301 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:38.619868  2268 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:38.647572  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1376) took 1148 ms. Trace:
I20260709 13:01:38.523569  3343 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:38.615167  3387 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:01:38.683539  3266 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:38.732415  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.733316  3390 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:38.650315  3394 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:38.654735  1566 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:38.686695  3325 tablet_bootstrap.cc:492] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:38.730667  3366 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:01:38.731433  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 210491us
I20260709 13:01:38.731539  3352 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:01:38.732292  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 112518us
I20260709 13:01:38.734009  1710 rpcz_store.cc:276] 0709 13:01:37.473310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.482873 (+  9563us) service_pool.cc:224] Handling call
0709 13:01:37.688450 (+205577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.695601 (+  7151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.695650 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:37.702391 (+  6741us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:37.702430 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.702487 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.617625 (+915138us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54003db8d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:38.619579 (+  1954us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442215680}
I20260709 13:01:38.651289  3402 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:38.742372  1569 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:38.683022  1609 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:38.740871  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:38.735152  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.740546  3352 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:38.745194  1638 raft_consensus.cc:2393] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:38.742301  1681 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:38.720211  3376 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:38.686084  3311 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:38.743548  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.739558  3366 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:38.698169  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.745177  3352 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:38.685837  3357 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:38.716058  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.624353  3342 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:38.744369  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 232682us
W20260709 13:01:38.808058  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.808223  1609 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:38.770861  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.768633  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.769704  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.749740  3389 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:01:38.698645  3345 consensus_meta.cc:287] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.798s	user 0.022s	sys 0.035s
I20260709 13:01:38.779889  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.748514  1681 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:38.749571  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.817925  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.686894  3388 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:38.813868   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:01:38.818410  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.749255  3266 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:38.814153  2269 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:38.825402  1561 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 2: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:38.769948  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.773780  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:01:38.818336  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.824584  1683 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:38.747499  3359 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:01:38.822162  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 73732.1us
I20260709 13:01:38.811935  3389 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:38.821355  1641 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:38.831449  1683 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:38.822149   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:38.810325  3368 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:38.833298  3300 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:38.833740  3300 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:38.834115  3300 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:38.811724  3340 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:38.754629  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.786145  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 337) took 1105 ms. Trace:
I20260709 13:01:38.811025  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.831975  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.832296  2269 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:01:38.823058  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.824692  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.806584  3311 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:38.834254  3362 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:38.783648  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.838769  1683 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:38.839638  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.851630  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.851732  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.828238  3346 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:38.852136  2269 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:38.853051  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.820484  3345 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:38.840989  3370 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:38.829051  3387 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:38.841500  3394 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:38.842530  3383 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:38.825644  3267 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:38.848292  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.840449  1715 rpcz_store.cc:276] 0709 13:01:37.680562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.185173 (+504611us) service_pool.cc:224] Handling call
0709 13:01:38.239790 (+ 54617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.514008 (+274218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.514045 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:38.514230 (+   185us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:38.514377 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.785838 (+271461us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54001473d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:38.785896 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.786066 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603499264}
I20260709 13:01:38.859630  3267 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:38.859930  3351 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:38.860131  3267 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:38.860499  3351 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:38.860859  3351 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:38.861227  3351 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:38.826262  3266 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:38.849794  3357 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:38.853130  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.854254  2269 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:38.811542  1636 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:01:38.835736  3389 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:38.868916  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 56434.2us
I20260709 13:01:38.839890  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:38.856950  1498 debug-util.cc:398] Leaking SignalData structure 0x7b08001b95c0 after lost signal to thread 1668
I20260709 13:01:38.855628  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.854131  3374 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:38.870486  3376 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:38.862161  3362 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:38.853052  3382 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:38.857229  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.857348  3392 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:38.909094  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.910471  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.910856  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.911700  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.911782  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.912010  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.882046  3350 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:38.913059  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.914469  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.915145  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.915513  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.916203  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.886855  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1371) took 1486 ms. Trace:
I20260709 13:01:38.857795  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:38.913020  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.831135  3359 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:38.837349  3325 ts_tablet_manager.cc:1403] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 3.355s	user 0.070s	sys 0.043s
I20260709 13:01:38.916860  1712 rpcz_store.cc:276] 0709 13:01:37.403999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.409772 (+  5773us) service_pool.cc:224] Handling call
0709 13:01:37.443037 (+ 33265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.509269 (+ 66232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.509310 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:37.518257 (+  8947us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:37.518312 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.886491 (+1368179us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x7b54003395d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:38.886534 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.886666 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113425792}
I20260709 13:01:38.919982  1563 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 2: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:38.871701  1609 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:38.874466  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:38.890038  1533 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.556s (SENT)
I20260709 13:01:38.894796  3352 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:38.895658  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.886309  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 1939) took 1640 ms. Trace:
I20260709 13:01:38.884886  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:38.940019  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.940927  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.922638  3359 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:38.891685  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1657) took 1001 ms. Trace:
I20260709 13:01:38.945950  1566 rpcz_store.cc:276] 0709 13:01:37.890581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.890692 (+   111us) service_pool.cc:224] Handling call
0709 13:01:37.899755 (+  9063us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.737407 (+837652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.891484 (+154077us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b54001ce654. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:38.891629 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1736013184}
I20260709 13:01:38.894450  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.948206  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.949143  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 1684) took 1533 ms. Trace:
I20260709 13:01:38.949434  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:38.949564  1713 rpcz_store.cc:276] 0709 13:01:37.415953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.419382 (+  3429us) service_pool.cc:224] Handling call
0709 13:01:37.419644 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.564378 (+144734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.564416 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:37.565865 (+  1449us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:37.565931 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.923730 (+1357799us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b5400148cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:38.948860 (+ 25130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.948963 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288374528}
I20260709 13:01:38.872350  3311 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:38.950783  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.951071  2297 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:01:38.879895  3347 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:38.882314  3388 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:38.952013  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.881224  3385 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:38.952764  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.953075  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:38.954468  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.955382  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 590) took 1144 ms. Trace:
I20260709 13:01:38.955835  2302 rpcz_store.cc:276] 0709 13:01:37.811161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.830611 (+ 19450us) service_pool.cc:224] Handling call
0709 13:01:37.860959 (+ 30348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.044096 (+183137us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.044136 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:38.044175 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:38.044202 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.924329 (+880127us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54005d2ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:38.954975 (+ 30646us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.955063 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":400006400}
I20260709 13:01:38.956969  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.958186  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:38.958287  3325 raft_consensus.cc:359] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:38.959172  3325 raft_consensus.cc:740] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:38.959883  3325 consensus_queue.cc:260] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:38.961894  3325 ts_tablet_manager.cc:1434] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 13:01:38.963253  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:38.963495  3345 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:38.965226  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.864125  3266 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:38.923866  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.971392  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.922437  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.974097  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:38.912982  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.974184  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:38.904953  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 131285us
I20260709 13:01:38.884357  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:38.873490  1498 debug-util.cc:398] Leaking SignalData structure 0x7b0800317ac0 after lost signal to thread 1742
W20260709 13:01:38.999703  1498 debug-util.cc:398] Leaking SignalData structure 0x7b08002f4b40 after lost signal to thread 2329
I20260709 13:01:39.000795  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 1962) took 1037 ms. Trace:
I20260709 13:01:39.001246  1561 rpcz_store.cc:276] 0709 13:01:37.963735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.963854 (+   119us) service_pool.cc:224] Handling call
0709 13:01:39.000469 (+1036615us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b5400216e54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:39.000645 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1933475072}
I20260709 13:01:39.002422  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.875865  3384 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:38.873456  3383 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:39.005426  3267 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:38.929967  3343 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:38.926601  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.945199  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.965483  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:38.965996  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.938906  1568 rpcz_store.cc:276] 0709 13:01:37.246149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.254624 (+  8475us) service_pool.cc:224] Handling call
0709 13:01:37.255686 (+  1062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.508543 (+252857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:37.508574 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:37.508615 (+    41us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:37.508643 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.885966 (+1377323us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54000a78d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:38.886011 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.886115 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":572366208}
I20260709 13:01:38.966840  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:38.884871  3374 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:38.974900  3390 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:38.926141  3392 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.029918  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:39.031018  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:39.031239  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:39.032315  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:39.035028  1498 debug-util.cc:398] Leaking SignalData structure 0x7b0800236120 after lost signal to thread 3325
I20260709 13:01:39.035759  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:39.037142  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:39.038308  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:39.038616  3351 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:38.979420  2268 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.040123  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:39.040354  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 61019.1us
I20260709 13:01:39.041290  3351 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:39.041646  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.039538  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.921595  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:38.980937  3352 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:38.975071  1709 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:38.986119  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:38.986459  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 282) took 1113 ms. Trace:
I20260709 13:01:39.045861  1644 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:01:39.045778  1569 rpcz_store.cc:276] 0709 13:01:37.873326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.875466 (+  2140us) service_pool.cc:224] Handling call
0709 13:01:37.895532 (+ 20066us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.743280 (+847748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.986224 (+242944us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b54002a9c54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:38.986400 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1947294080}
I20260709 13:01:39.048257  1569 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:39.006727  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:38.929292  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:39.006408  3366 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.011571  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.052598  1683 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.050800  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:39.053321  3352 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.054136  3366 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:39.013624  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:38.988982  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:39.015204  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:39.016856  3402 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.017963  3320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d59c4d21b6cb46edbfc2b3d50b51812a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:39.059531  3382 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.067040  3323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb294ac919b44e56960745f79e83dfc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:01:39.074632  1712 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:39.074540  1683 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:39.018345  3266 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:39.018370  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:39.019944  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:01:39.020457  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 148862us
I20260709 13:01:38.993400  1563 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 1.
I20260709 13:01:39.028769  3374 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:39.039368  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 280) took 1221 ms. Trace:
I20260709 13:01:39.043774  2268 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.082747  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:01:39.087812  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 44168.8us
I20260709 13:01:39.088650  2268 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.080870  2269 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.089942  2268 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.042412  3383 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.062021  3350 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:38.980813  3395 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:39.080392  1654 tablet_copy_service.cc:342] P 5184c101be204393adf3ac8b31f25ad6: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-3f22f7f5cede49bea2083e8905932fb7 received from {username='slave'} at 127.0.0.1:53222
I20260709 13:01:39.083731  3266 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:39.096685  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:39.104743  1712 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:39.096918  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:39.105386  1716 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:39.097257  3382 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:39.093011  3376 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.086311  1609 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.107197  1609 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:39.108465  1609 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.109937  1609 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.097745  3366 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.100358  1654 tablet_copy_service.cc:434] P 5184c101be204393adf3ac8b31f25ad6: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-3f22f7f5cede49bea2083e8905932fb7 on tablet 3f22f7f5cede49bea2083e8905932fb7 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.093186  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:39.102242  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:39.079715  3402 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:39.086573  3352 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:39.132659  1709 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:39.134505  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 185536us
I20260709 13:01:39.126765  1609 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.128667  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:39.100693  3397 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.092348  3342 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:39.114197  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:39.084280  3267 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:39.153077  1532 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:39.114118  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 224116us
I20260709 13:01:39.133983  1565 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:39.135993  1683 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.138170  1563 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 4: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:39.138290  3225 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:39.139360  3384 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.140147  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:39.135895  3332 consensus_queue.cc:1059] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:01:39.141909  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:39.172176  3332 raft_consensus.cc:3055] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:39.172637  3332 raft_consensus.cc:740] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:39.173533  3332 consensus_queue.cc:260] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:39.143949  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:39.144537  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:39.144660  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:39.138720  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:39.114316  3372 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:39.156955  3378 consensus_queue.cc:1059] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
W20260709 13:01:39.179627  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 122363us
I20260709 13:01:39.180353  1534 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:39.143165  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:39.143412  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:39.158540  1716 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:39.113497  3351 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.169754  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:39.174757  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:39.177368  3372 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:39.169937  3339 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:39.169209  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:01:39.168782  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 70152us
I20260709 13:01:39.130246  3392 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.175282  1561 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:39.181875  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:39.135572  3352 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:39.188293  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:01:39.168416  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 74358.7us
I20260709 13:01:39.168141  1709 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:39.179380  3357 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.184636  1683 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.199092  3311 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:39.088943  1560 rpcz_store.cc:276] 0709 13:01:37.818403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.820732 (+  2329us) service_pool.cc:224] Handling call
0709 13:01:37.853064 (+ 32332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.233745 (+380681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.039140 (+805395us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b5400270454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:39.039312 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558861184}
I20260709 13:01:39.200212  1713 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:39.112043  3390 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:01:39.180845  3384 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:01:39.185523  3372 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:39.202251  3384 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:39.187567  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:39.186141  3342 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:39.186908  3301 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:39.212801  3390 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:39.203115  3385 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:39.226411  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:39.210136  3342 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:39.203600  3352 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:39.219211  3389 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.187165  3339 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:39.231595  3339 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:39.228662  1562 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 2: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:39.222093  3301 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:39.235894  3301 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:39.235963  3389 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:39.236974  2296 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 2: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:01:39.238101  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.220989  3300 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.221715  1635 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:01:39.243901  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:39.246845  3300 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:39.247161  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:39.250146  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 114092us
I20260709 13:01:39.251962  3389 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:39.252420  3359 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.225204  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:39.223127  3390 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:39.257939  3390 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.259303  3390 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:39.226558  3385 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:39.202622  3384 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:39.224725  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:39.250837  1609 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:39.206028  3367 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:39.229831  3342 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:39.267836  3225 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:39.268244  3225 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:39.269042  3225 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:39.270383  3225 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:39.228670  3397 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.272023  3384 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:39.274242  3384 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:39.240088  3374 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.261467  1609 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.259629  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:39.261626  3351 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:39.263298  3385 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:39.265415  2303 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:39.284768  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 104515us
I20260709 13:01:39.276281  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.265020  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 605) took 1125 ms. Trace:
I20260709 13:01:39.288275  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:39.283645  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:39.266224  3382 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.288236  3311 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:39.261898  3332 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:39.289413  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:39.297770  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:39.298036  3389 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.292479  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:39.299299  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.293164  3378 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:39.294070  3357 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:39.299886  3389 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:39.300558  3357 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:39.300937  3357 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:39.301739  3357 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.304720  1533 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.305816  3389 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:39.305892  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:39.306463  3389 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.307026  3344 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.307721  3344 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:39.308045  3344 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:39.308408  3344 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:39.309158  3389 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:39.309643  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:39.310251  3388 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.300309  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:39.311854  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.313758  3378 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:39.315320  3388 raft_consensus.cc:3055] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:39.315739  3388 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:39.316658  3388 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:39.319478  3388 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:39.319948  3357 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:39.321587  1567 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:39.322256  1560 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:39.319492  1533 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.325466  2268 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.326624  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.319978  1534 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.328816  3343 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.325946  3359 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:39.284839  3392 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.285259  3351 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:39.331996  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:39.335784  1709 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:39.337276  1682 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.342447  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:39.343602  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:39.332188  3346 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.290153  3397 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.353809  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:39.296358  3374 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:39.371769  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:39.330123  1534 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.341706  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:39.353953  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:39.335258  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:39.342109  1533 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.328114  3382 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.369643  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.287902  2296 rpcz_store.cc:276] 0709 13:01:38.162312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.182008 (+ 19696us) service_pool.cc:224] Handling call
0709 13:01:38.783404 (+601396us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:39.264763 (+481359us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b54001112d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:39.264933 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":892397568}
I20260709 13:01:39.334847  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:39.345484  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:39.378006  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:39.341409  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:39.378590  3346 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:39.337695  3367 raft_consensus.cc:993] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:39.411373  3367 raft_consensus.cc:1081] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:01:39.379292  3409 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:01:39.412827  3409 raft_consensus.cc:3055] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:39.413180  3409 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:39.378109  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:39.382097  3343 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.380118  3347 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:39.414069  3409 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.296890  3339 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.379849  3320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:53446 (request call id 433) took 3878 ms. Trace:
I20260709 13:01:39.349614  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:39.333606  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:39.393215  3300 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:39.359953  3351 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:39.345846  3383 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.403942  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:39.404592  3402 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.405781  3266 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.410955  3346 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.442749  3266 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:39.444677  3346 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:39.414848  3347 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:39.445330  3347 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:39.382858  3345 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:39.417748  3323 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:53448 (request call id 607) took 4071 ms. Trace:
I20260709 13:01:39.334630  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:39.330057  3389 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.377579  2303 raft_consensus.cc:2410] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:39.397037  1533 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.399590  1682 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:39.448081  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 51125.5us
I20260709 13:01:39.449057  1533 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:01:39.420325  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 91593.6us
I20260709 13:01:39.422462  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:39.372829  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:01:39.422966  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 238653us
W20260709 13:01:39.428543  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 54085.8us
I20260709 13:01:39.330420  3411 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.391523  3352 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.431346  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:39.425520  3342 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.433861  3344 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:39.418570  3385 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:39.436936  3300 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:39.439415  3404 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:39.450426  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:39.478353  3340 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:39.478814  3340 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:39.479189  3340 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:39.477270  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:39.456856  1609 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:39.459385  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 59935us
I20260709 13:01:39.481699  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.463376  3360 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:39.451663  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:39.427263  3320 rpcz_store.cc:276] 0709 13:01:35.468510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.468653 (+   143us) service_pool.cc:224] Handling call
0709 13:01:35.469639 (+   986us) ts_tablet_manager.cc:934] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
0709 13:01:39.016347 (+3546708us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:01:39.170340 (+153993us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:39.170479 (+   139us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:01:39.270427 (+ 99948us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:01:39.270732 (+   305us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:39.288396 (+ 17664us) ts_tablet_manager.cc:989] Replica registered.
0709 13:01:39.346549 (+ 58153us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":220,"tablet-copy.queue_time_us":131,"thread_start_us":6895,"threads_started":1}
I20260709 13:01:39.465498  3409 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:39.445330  3266 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:39.467816  1639 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:39.472297  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:39.469575  3343 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:39.473600  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.483515  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.483601  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:39.503926  3413 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:39.504150  1532 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:39.504398  3413 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:39.504858  3413 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:39.471783  1636 raft_consensus.cc:2393] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:39.506506  3382 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.502180  1561 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
W20260709 13:01:39.509267  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 52472.5us
I20260709 13:01:39.507879  2268 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.438375  3348 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:01:39.511109  1681 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:39.512475  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.512686  1609 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.486265  1533 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.452374  3345 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:39.496275  1567 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:39.457988  3399 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.502882  1568 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:39.520701  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.522014  1682 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.522358  3415 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:39.523276  3414 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:01:39.525426  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 48216.8us
I20260709 13:01:39.526404  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:39.519767  3320 tablet_copy_client.cc:806] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:39.488191  3360 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:39.532783  3350 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.533178  3360 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:39.533591  3350 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:39.488556  3408 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:39.487953  3411 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.537439  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:39.546165  1682 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.550848  1682 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.485553  3384 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:01:39.508420  3343 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:39.558600  1563 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:39.534008  3394 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.536316  3345 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:39.536160  3362 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.519977  3382 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.520299  3392 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.488431  3301 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.484863  3376 consensus_queue.cc:1059] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:01:39.532866  3320 tablet_copy_client.cc:670] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:39.536528  3402 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.542258  3350 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.543149  3346 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.541801  3408 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:39.547049  3266 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:39.554998  3267 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:39.587567  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 61279.6us
I20260709 13:01:39.565272  3343 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:39.604640  1534 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.566596  3415 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:39.453625  3323 rpcz_store.cc:276] 0709 13:01:35.346337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.346477 (+   140us) service_pool.cc:224] Handling call
0709 13:01:35.390404 (+ 43927us) ts_tablet_manager.cc:934] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
0709 13:01:39.032441 (+3642037us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:01:39.225633 (+193192us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:39.229570 (+  3937us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:01:39.417033 (+187463us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:01:39.417314 (+   281us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:39.417595 (+   281us) ts_tablet_manager.cc:989] Replica registered.
0709 13:01:39.417692 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":279,"tablet-copy.queue_time_us":40815,"thread_start_us":46396,"threads_started":2}
I20260709 13:01:39.676741  3323 tablet_copy_client.cc:806] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:01:39.588649  3350 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:39.569751  3414 raft_consensus.cc:697] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:39.600095  3408 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:39.572577  3372 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:39.776091  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 227332us
I20260709 13:01:39.814365  3402 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.566952  3391 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.130s
I20260709 13:01:39.588060  3399 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:01:39.816150  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 266562us
I20260709 13:01:39.843465  1682 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:01:39.843770  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 238348us
W20260709 13:01:39.822878  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 218306us
W20260709 13:01:39.825802  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.847055  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:39.847759  1567 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:39.831153  1566 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:39.832827   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:39.835819  3357 consensus_meta.cc:287] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.515s	user 0.014s	sys 0.020s
I20260709 13:01:39.840610  3391 raft_consensus.cc:3055] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:01:39.838528  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.856235  1644 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:39.824230  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 65378.9us
I20260709 13:01:39.860826  1712 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:39.864464  1682 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.862501  3391 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:39.883484  3415 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:39.843354  3323 tablet_copy_client.cc:670] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:01:39.845239  3399 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:39.948042  3399 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:39.890403  3352 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:39.901324  1682 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:39.903121  1567 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:01:39.814863  3410 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:39.836337  3362 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:01:39.862928  3266 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:39.945673  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 97304us
I20260709 13:01:39.918133  3423 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:39.897231  3389 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:39.885298  3414 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:39.955044  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:39.939378  3344 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:39.954003  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 52816.3us
W20260709 13:01:39.959782  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 473611us
I20260709 13:01:39.968675  3372 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:39.974681  3362 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:39.956441  3301 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:39.974730  3352 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:01:40.016508  3351 consensus_meta.cc:287] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.565s	user 0.012s	sys 0.000s
I20260709 13:01:40.016239  3352 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:40.016916  3351 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:40.017803  3388 consensus_meta.cc:287] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.698s	user 0.007s	sys 0.004s
I20260709 13:01:40.016288  3372 consensus_queue.cc:1059] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
W20260709 13:01:39.976605  3332 consensus_meta.cc:287] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.680s	user 0.008s	sys 0.001s
I20260709 13:01:39.973323  3350 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:40.020649  3340 consensus_meta.cc:287] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.541s	user 0.007s	sys 0.003s
I20260709 13:01:40.021032  3340 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:40.018597  3352 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:40.022639  3350 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:40.023115  3332 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:40.023552  3332 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:01:39.440371  3387 consensus_queue.cc:1175] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3f64b76180a54492bb015e405caeacec" responder_term: 4 status { last_received { term: 3 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:39.953896  3342 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:40.024950  3332 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:40.026965  3360 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:39.956817  3300 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:39.987505  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:39.993261  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.973014  3339 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:40.039456  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 46274.1us
I20260709 13:01:40.040477  1682 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:40.041901  1682 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:39.951403  3385 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:39.987442  3395 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:40.014590  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 117165us
I20260709 13:01:40.015327  3430 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:01:39.980039  1635 consensus_meta.cc:287] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.000s
I20260709 13:01:40.016182  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:40.032534  1642 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
W20260709 13:01:40.036491  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 49131.7us
W20260709 13:01:40.047152  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 54503.3us
I20260709 13:01:40.051687  1682 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:40.010663  3362 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:40.055675  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:40.056375  3362 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:40.058001  3362 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:40.047499  3350 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:40.048060  3430 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:40.053849  3351 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:40.054890  3343 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:40.043815  3340 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:40.063205  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.011041  3389 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:40.069061  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 202214us
I20260709 13:01:40.069075  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:40.069381  1533 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:40.047821  3266 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:40.050295  3300 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:40.070662  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:40.063592  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 52257.1us
I20260709 13:01:40.046588  1635 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:40.104331  3320 tablet_copy_client.cc:538] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:01:40.105178  1568 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:40.078560  1641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60390
I20260709 13:01:40.078904  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:40.081498  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:40.065141  3382 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:40.107431  1637 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:40.108474  3430 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:40.109416  3413 consensus_meta.cc:287] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.604s	user 0.013s	sys 0.000s
I20260709 13:01:40.109817  3413 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:40.112331  3413 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:40.043192  3391 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:40.074235  1638 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:40.081290  1534 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:40.071357  3423 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:01:40.081700  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 213410us
I20260709 13:01:40.100843  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:40.102473  3421 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:40.127511  3360 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:40.129793  3389 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:40.130352  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:40.137156  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:40.154114  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:40.091416  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:01:40.162351  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 71057.8us
I20260709 13:01:40.163393  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:40.164690  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:40.166378  1568 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:40.166651  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 74899.9us
I20260709 13:01:40.171978  1533 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:40.173341  1533 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:40.173629  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:40.177374  1568 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:40.138868  3225 raft_consensus.cc:3055] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:40.124189  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:40.184949  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:40.182981  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:40.183652  3412 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:40.179143  3344 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:40.163372  3359 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:40.206938  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:40.175155  3411 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:40.208386  1715 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:40.206065  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:01:40.198180  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 151921us
I20260709 13:01:40.181666  3421 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:40.201649  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:40.199554  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:40.210248  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:01:40.200726  3347 consensus_meta.cc:287] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.755s	user 0.003s	sys 0.010s
I20260709 13:01:40.225816  2303 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:40.222548  1710 raft_consensus.cc:1240] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:40.219395  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:40.228588  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:40.219939  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:40.194505  3311 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:40.226949  1714 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:40.221424  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:40.201347  3225 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:40.184600  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:40.230980  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:40.232021  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:40.218312  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:40.206571  3367 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:40.208616  3385 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:40.235184  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:40.230010  2303 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:40.236195  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:01:40.323141  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 87101.8us
W20260709 13:01:40.321292  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 83165.1us
I20260709 13:01:40.220387  1682 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:01:40.221190  1711 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:40.220520  3394 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:40.180759  3404 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:40.234654  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:40.319520  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:40.321777  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:01:40.282375  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.221310  1717 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:40.238190  3347 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:40.224386  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:01:40.330860  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 110568us
I20260709 13:01:40.226246  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:40.406548  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:40.337316  3311 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:40.336704  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:40.448725  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 112109us
I20260709 13:01:40.448925  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:40.449487  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:01:40.402642  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 73654.3us
I20260709 13:01:40.306581  3344 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:40.236703  3359 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:40.450111  1682 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:40.329789  3421 raft_consensus.cc:697] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:40.328135  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:40.334080  3404 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:01:40.452307  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 124286us
I20260709 13:01:40.456099  1709 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:40.343375  3350 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:40.432358  3323 tablet_copy_client.cc:538] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:01:40.439658  3432 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
I20260709 13:01:40.083652  3425 consensus_queue.cc:1059] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.148s
I20260709 13:01:40.325158  3388 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.108s
I20260709 13:01:40.325726  3367 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:40.333240  3383 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:40.469353  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:40.445756  3436 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:01:40.451977  3394 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:40.451649  3395 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:40.482357  3388 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:40.486207  3311 consensus_queue.cc:1059] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:40.489195  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:40.460672  3374 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:40.459827  3389 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:40.455652  3360 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:01:40.471789  3385 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:40.491693  3345 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.907s	user 0.008s	sys 0.004s
I20260709 13:01:40.460467  3350 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:40.507586  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:40.467011  3346 consensus_queue.cc:1059] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:01:40.480991  3343 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:40.506620  3414 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:40.493146  3413 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:40.496145  3383 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:40.524076  3435 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:40.523612  3383 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:40.512603  3430 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:40.516592  3350 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:40.518294  3344 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:40.535732  3434 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:01:40.525561  3267 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:40.508589  3389 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:40.520327  3394 consensus_queue.cc:1059] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
W20260709 13:01:40.576550  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 107270us
I20260709 13:01:40.516394  3345 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:40.581454  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:40.575286  3408 consensus_meta.cc:287] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.732s	user 0.000s	sys 0.011s
W20260709 13:01:40.579674  3409 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.079s	user 0.008s	sys 0.005s
W20260709 13:01:40.594260  3437 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:01:40.595487  3320 tablet_bootstrap.cc:492] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:40.529356  3413 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:40.604149  1564 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:40.586712  3438 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:01:40.527160  3421 consensus_queue.cc:237] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:40.591195  3434 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:40.598012  3267 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:40.626659  3267 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:40.630489  2299 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:40.613008  3416 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:40.611124  3347 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:40.634724  3267 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:40.592418  3408 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:40.634371  3416 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:40.654465  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 141617us
W20260709 13:01:40.643363  3362 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.585s	user 0.013s	sys 0.002s
I20260709 13:01:40.689798  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:40.646307  3367 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:40.655730  3332 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:40.725484  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:40.660264  3343 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:01:40.690706  3399 consensus_meta.cc:287] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.742s	user 0.009s	sys 0.003s
I20260709 13:01:40.664966  3435 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 13:01:40.663398  3394 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 13:01:40.750231  2298 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:01:40.751084  3389 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:40.750677  3392 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:01:40.752087  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:40.757129  1641 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:40.765471  1641 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:40.757920  3382 raft_consensus.cc:3055] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:40.759923  1712 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:40.768181  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 465) took 1251 ms. Trace:
I20260709 13:01:40.768741  2300 rpcz_store.cc:276] 0709 13:01:39.516369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.520107 (+  3738us) service_pool.cc:224] Handling call
0709 13:01:40.758875 (+1238768us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:01:40.768096 (+  9221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42764032}
I20260709 13:01:40.788375  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1507) took 1454 ms. Trace:
I20260709 13:01:40.788865  1644 rpcz_store.cc:276] 0709 13:01:39.333867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.333999 (+   132us) service_pool.cc:224] Handling call
0709 13:01:39.334259 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.863637 (+529378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.788086 (+924449us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b54001e8f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:40.788271 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1200062336}
I20260709 13:01:40.790769  3384 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:40.791234  3384 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:40.791460  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:40.791906  3339 raft_consensus.cc:1064] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:40.792032  1644 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:40.759311  3385 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:40.741344  3399 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:40.802955  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:40.803951  1639 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 2: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:40.774574  1636 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:40.743907  3343 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:01:40.812640  1644 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:40.804762  3399 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:40.808300  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:40.769842  3382 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:40.818936  3442 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:40.822301  1644 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:40.823246  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:40.794075  3345 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:40.792079  3384 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:40.901633  3384 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:40.907516  1641 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:40.796562  3392 consensus_queue.cc:1243] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:01:40.827596  3416 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:40.955592  3343 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:40.898787  3412 consensus_meta.cc:287] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.701s	user 0.014s	sys 0.000s
I20260709 13:01:40.946717  3323 tablet_bootstrap.cc:492] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
I20260709 13:01:40.992101  3382 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.011200  3420 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:41.046826  1641 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:40.888073  3413 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:40.885275  3374 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:40.810212  3385 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:41.057331  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:40.939149  3399 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:41.049163  1639 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:01:41.074808  1683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.048s	sys 0.120s
W20260709 13:01:41.075237  1683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.048s	sys 0.120s
W20260709 13:01:41.076872  1681 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.047s	sys 0.076s
W20260709 13:01:41.077242  1681 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.047s	sys 0.076s
I20260709 13:01:41.050618  3412 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:41.059701  3408 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:41.063277  3382 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:41.070995  3389 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:41.095124  3357 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:41.099052  3357 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:41.103900  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.075551  3436 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:01:41.104914  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.066112  3374 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.071391  3399 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:40.969375  3442 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.114117  3367 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.125883  3443 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:41.132212  3267 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:41.133252  3267 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.139544  3352 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:41.140244  3352 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.126009  3367 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:41.102139  3357 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:41.108963  3389 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.148540  3436 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.087617  3339 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.162757  3332 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:41.082542  3420 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.181887  3424 raft_consensus.cc:3055] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:41.169132  3388 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.182380  3424 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:41.183763  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:41.162133  3357 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
W20260709 13:01:41.191171  3411 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.973s	user 0.007s	sys 0.008s
I20260709 13:01:41.192010  3411 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:41.175688  3352 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:41.178627  3404 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:41.126740  3443 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:41.192986  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1506) took 1854 ms (client timeout 1986 ms). Trace:
I20260709 13:01:41.220592  3385 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:41.185794  3424 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.223893  3424 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:41.184453  3332 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.187783  3388 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:41.236580  3436 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:41.218510  3374 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:41.216686  3389 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:41.214459  3267 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:41.212895  3420 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:41.217789  3399 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:41.225407  3357 raft_consensus.cc:3055] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:41.239436  1643 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:01:41.219100  3442 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:41.268287  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:41.268765  3443 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:41.273555  3414 raft_consensus.cc:3055] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:41.274070  3414 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:41.274976  3414 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.220857  3352 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:01:41.245448  3430 raft_consensus.cc:3055] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:01:41.221649  1635 rpcz_store.cc:269] 0709 13:01:39.333210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.333332 (+   122us) service_pool.cc:224] Handling call
0709 13:01:41.182616 (+1849284us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b54002d8054. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:41.187697 (+  5081us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86552448}
I20260709 13:01:41.266856  3357 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:41.303838  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 614) took 1079 ms. Trace:
I20260709 13:01:41.303575  3357 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.252696  3397 raft_consensus.cc:3055] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:41.304652  2298 rpcz_store.cc:276] 0709 13:01:40.223953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.224092 (+   139us) service_pool.cc:224] Handling call
0709 13:01:41.303614 (+1079522us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400544cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:41.303756 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375191936}
I20260709 13:01:41.306757  3357 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:41.299106  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 1974) took 1223 ms. Trace:
I20260709 13:01:41.289853  3430 raft_consensus.cc:740] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:41.322609  1637 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:01:41.323129  1714 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:01:41.319160  3391 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:41.323566  1638 rpcz_store.cc:276] 0709 13:01:40.076157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.116335 (+ 40178us) service_pool.cc:224] Handling call
0709 13:01:41.268108 (+1151773us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:41.299014 (+ 30906us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24821120}
I20260709 13:01:41.326114  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:41.326478  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1414) took 1260 ms. Trace:
I20260709 13:01:41.348282  1712 rpcz_store.cc:276] 0709 13:01:40.066426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.066578 (+   152us) service_pool.cc:224] Handling call
0709 13:01:41.326290 (+1259712us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b5400268ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:41.326386 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1335131392}
I20260709 13:01:41.326867  3332 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:41.327319  3367 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:41.331894  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:41.332360  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:41.335915  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:41.337224  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:41.328152  3350 consensus_meta.cc:287] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.761s	user 0.005s	sys 0.009s
I20260709 13:01:41.358048  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:01:41.359227  1534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.028s	sys 0.032s
W20260709 13:01:41.359683  1534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.028s	sys 0.032s
W20260709 13:01:41.406136  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 48209.8us
I20260709 13:01:41.419811  3433 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:41.325160  3397 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:41.424013  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 367) took 1885 ms. Trace:
I20260709 13:01:41.431579  1709 rpcz_store.cc:276] 0709 13:01:39.539104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.543985 (+  4881us) service_pool.cc:224] Handling call
0709 13:01:39.563242 (+ 19257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.456604 (+893362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.423731 (+967127us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b5400147dd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:41.423922 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2053100928}
W20260709 13:01:41.427903  1607 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.027s	sys 0.019s
I20260709 13:01:41.487234  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1694) took 2012 ms. Trace:
I20260709 13:01:41.420522  3433 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:01:41.480885  1607 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.027s	sys 0.019s
I20260709 13:01:41.421097  3423 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.487756  1564 rpcz_store.cc:276] 0709 13:01:39.474645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.474775 (+   130us) service_pool.cc:224] Handling call
0709 13:01:39.475057 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.605243 (+1130186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.486631 (+881388us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54002986d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:41.486903 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2596458880}
W20260709 13:01:41.420812  3435 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.010s	sys 0.029s
I20260709 13:01:41.423513  3412 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:41.352483  3411 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.138s
I20260709 13:01:41.468115  3359 raft_consensus.cc:3055] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:41.491252  3372 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:01:41.500700  3435 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.010s	sys 0.030s
I20260709 13:01:41.351776  3430 consensus_queue.cc:260] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:41.519673  3387 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:41.421509  3350 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:41.525280  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1521) took 1342 ms (client timeout 1783 ms). Trace:
I20260709 13:01:41.509290  3359 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:41.516297  3449 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:41.547246  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.512236  3412 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:41.544241  3388 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:41.554041  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.971s	user 0.011s	sys 0.016s
W20260709 13:01:41.555681  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.973s	user 0.011s	sys 0.016s
W20260709 13:01:41.543144  1643 rpcz_store.cc:269] 0709 13:01:40.182785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.221114 (+ 38329us) service_pool.cc:224] Handling call
0709 13:01:41.525013 (+1303899us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b54002d8a54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:41.525185 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2119215872}
I20260709 13:01:41.565018  1564 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:41.566789  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:41.567780  1567 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:41.572875  1712 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:41.573321  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:41.573418  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:41.535934  3387 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:41.582641  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:41.546773  3433 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.534551  3372 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:41.591326  1610 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.020s	sys 0.018s
W20260709 13:01:41.593313  1610 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.021s	sys 0.018s
W20260709 13:01:41.597604  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.603132  1567 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:41.607641  1682 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:41.585846  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:41.609093  3360 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:01:41.615542  2271 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.012s	sys 0.020s
W20260709 13:01:41.615971  2271 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.012s	sys 0.020s
I20260709 13:01:41.616169   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:41.500542  3423 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:41.617480  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.586704  3388 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:41.635746  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.015s	sys 0.031s
I20260709 13:01:41.637184  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:41.639678  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:41.643923  1561 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:41.645632  1642 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:41.643636  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.015s	sys 0.031s
I20260709 13:01:41.649415  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:41.650408  1561 raft_consensus.cc:2393] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 3: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:41.548640  3397 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.651997  3397 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:41.601706  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:41.653900  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:41.657604  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.660063  3419 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:41.660670  3419 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:41.661696  1609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.039s	sys 0.037s
W20260709 13:01:41.662067  1609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.039s	sys 0.037s
I20260709 13:01:41.640033  3266 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:41.641166  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:41.664783  1561 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:41.668186  1533 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.009s	sys 0.013s
I20260709 13:01:41.671001  3350 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:41.640816  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.034s	sys 0.032s
I20260709 13:01:41.681283  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:41.689132  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.034s	sys 0.032s
W20260709 13:01:41.684135  1533 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.009s	sys 0.013s
W20260709 13:01:41.685552  2270 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.033s	sys 0.017s
I20260709 13:01:41.653246  3427 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.648838  3383 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:41.603847  3449 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:41.699615  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:41.700515  2300 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:41.703660  2269 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:01:41.692804  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 535626us
I20260709 13:01:41.606485  3225 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:41.705201  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.705547  3225 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:41.706219  1715 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:41.706369  1714 raft_consensus.cc:3055] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:41.706691  1714 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:41.707320  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:41.708066  1712 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:01:41.708369  1713 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:41.709156  1714 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:41.709749  1533 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:41.710105  1714 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:41.693097  2270 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.033s	sys 0.017s
W20260709 13:01:41.694093  3344 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.930s	user 0.030s	sys 0.091s
I20260709 13:01:41.656513  3442 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:41.711688  3442 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.669528  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:41.656442  3453 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:41.654197  3388 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:41.736584  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:41.737799  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:41.737993  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:41.744145  1643 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
W20260709 13:01:41.740870  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.657878  3412 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:41.689268  3348 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:41.712268  3434 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:41.763258  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.758482  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:41.703030  3436 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:41.759184  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:41.691349  3435 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:41.759829  1715 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:41.761353  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:41.698050  3419 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:41.640841  3359 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.757427  1644 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
W20260709 13:01:41.720142  3344 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.030s	sys 0.092s
I20260709 13:01:41.765545  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:41.766294  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:41.762151  3345 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:41.737645  3449 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:41.770447  2303 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:01:41.782094  1561 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:41.783728  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:41.681926  3387 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.765167  1639 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:41.713451  3383 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.712916  3427 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.860313  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:41.780886  3266 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:41.866835  3414 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:41.768832  3350 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:01:41.917943  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.918813  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 147567us
I20260709 13:01:41.919720  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:41.766628  3458 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:41.915266  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:41.915592  3419 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:41.925792  1568 raft_consensus.cc:2393] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 3: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:41.760813  3385 consensus_queue.cc:1059] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.110s
I20260709 13:01:41.928263  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:41.819110  3360 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:41.785768  1637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 1978) took 1693 ms (client timeout 1650 ms). Trace:
I20260709 13:01:41.817252  3332 consensus_queue.cc:1059] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.132s
I20260709 13:01:41.742460  3421 consensus_queue.cc:1059] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.114s
I20260709 13:01:41.861543  3449 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:41.942909  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:41.857265  3459 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:41.931078  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 167026us
I20260709 13:01:41.775559  3436 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.773211  3348 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.870792  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:41.947546  3376 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.947546  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:41.969131  1568 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 2: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:41.949085  3385 raft_consensus.cc:3055] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:41.852897  3345 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:41.955320  3225 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:41.770054  3453 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.974368  3225 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:41.975119  3225 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:41.788125  3435 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.979210  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:41.946686  3452 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:41.929541  3439 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:01:41.999078  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.950841  3372 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:41.998518  3452 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.008061  3452 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:41.928356  3266 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.965191  1562 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:41.979355  1640 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:41.998621  3427 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:42.002610  3388 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:42.002427  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:42.026906  3433 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:42.003193  3442 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:42.004765  3345 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:42.005383  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:41.994257  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:42.015817  1636 consensus_meta.cc:287] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.192s	user 0.014s	sys 0.001s
I20260709 13:01:41.927862  3350 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.965739  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:41.970242  3347 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.936545  3419 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:42.020896  3457 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:42.021600  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:42.021708  1560 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:41.973322  3340 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.938113  3360 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:41.977337  3385 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:01:42.026479  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 1988) took 1863 ms (client timeout 1532 ms). Trace:
W20260709 13:01:42.016625  3424 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.792s	user 0.006s	sys 0.006s
I20260709 13:01:42.009770  1714 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:01:41.932710  1637 rpcz_store.cc:269] 0709 13:01:40.079945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.108177 (+ 28232us) service_pool.cc:224] Handling call
0709 13:01:41.182245 (+1074068us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:41.691168 (+508923us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400123c54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:41.770112 (+ 78944us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":356894336}
I20260709 13:01:41.998504  3389 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.987946  1717 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:41.959695  3449 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:42.056304  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.057520  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:42.028034  3392 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.981732  3376 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.965344  3459 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:42.000566  3431 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.387s
I20260709 13:01:41.935020  3458 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:41.999331  3439 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:42.062474  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:42.562075  1643 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:42.089763  3225 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:42.564793  1712 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
W20260709 13:01:42.406018  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.072s	sys 0.109s
W20260709 13:01:42.204607  1681 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.011s	sys 0.038s
W20260709 13:01:42.567389  1681 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.023s	sys 0.044s
I20260709 13:01:42.070344  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:42.567919  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 884336us
W20260709 13:01:42.568842  1566 rpcz_store.cc:269] 0709 13:01:40.163106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.163277 (+   171us) service_pool.cc:224] Handling call
0709 13:01:42.026240 (+1862963us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54001ce654. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:42.026366 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3829022080}
W20260709 13:01:42.456172  2270 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.009s	sys 0.049s
W20260709 13:01:42.572980  2270 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.014s	sys 0.049s
I20260709 13:01:42.574766  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:01:42.094130  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:42.580178  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.584321  3480 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:42.585048  3480 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:42.460325  3460 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.587549  3460 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:42.588794  3480 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:42.590104  3460 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:42.395413  1533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.013s	sys 0.019s
W20260709 13:01:42.591988  1533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.013s	sys 0.026s
W20260709 13:01:42.592376  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 839101us
W20260709 13:01:42.434728  1532 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.005s	sys 0.031s
W20260709 13:01:42.566720  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.074s	sys 0.115s
W20260709 13:01:42.593407  1532 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.005s	sys 0.038s
W20260709 13:01:42.593981  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 890445us
I20260709 13:01:42.592669  3225 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:42.095484  1568 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:01:42.626046  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:42.593897  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 820861us
I20260709 13:01:42.601755  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:42.671190  1710 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:42.597259  2269 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:42.627848  1642 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 5. Ops: []
I20260709 13:01:42.670478  2305 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 2: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:01:42.755401  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 158229us
I20260709 13:01:42.405325  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:42.652350  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:42.756695  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 351474us
W20260709 13:01:42.756916  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 130997us
I20260709 13:01:42.757939  1533 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:01:42.758539  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:42.595534  3323 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.001s	sys 0.010s
I20260709 13:01:42.559285  3350 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:42.626345  1683 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:42.607025  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:42.053084  1636 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:42.765250  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 1999) took 1146 ms. Trace:
W20260709 13:01:42.704823  2303 consensus_meta.cc:287] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.841s	user 0.009s	sys 0.000s
I20260709 13:01:42.405978  3388 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:42.752718  2302 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
W20260709 13:01:42.764967  3323 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.005s	sys 0.017s
W20260709 13:01:42.766328  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:42.656611  3414 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.731s	user 0.008s	sys 0.000s
I20260709 13:01:42.787672  3414 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:42.788259  3414 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:01:42.601639  3397 consensus_meta.cc:287] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.949s	user 0.009s	sys 0.004s
I20260709 13:01:42.760215  3409 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.765866  1564 rpcz_store.cc:276] 0709 13:01:41.619336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.619491 (+   155us) service_pool.cc:224] Handling call
0709 13:01:42.606805 (+987314us) spinlock_profiling.cc:243] Waited 4.92 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:42.765179 (+158374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11308288}
I20260709 13:01:42.455750  3357 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:42.777468  2303 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:01:42.791971  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 478) took 1031 ms. Trace:
W20260709 13:01:42.643603  3467 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.544s	user 0.000s	sys 0.001s
I20260709 13:01:42.097965  3266 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:42.797850  3458 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:42.798420  3266 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.792977  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:42.798974  3266 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:42.799832  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 319) took 1016 ms. Trace:
I20260709 13:01:42.800300  1569 rpcz_store.cc:276] 0709 13:01:41.783575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.783699 (+   124us) service_pool.cc:224] Handling call
0709 13:01:42.792778 (+1009079us) spinlock_profiling.cc:243] Waited 42.6 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:42.799742 (+  6964us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97951488}
W20260709 13:01:42.801645  3452 consensus_meta.cc:287] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.793s	user 0.003s	sys 0.004s
I20260709 13:01:42.801828  3266 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:42.601639  3402 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:01:42.140492  3423 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:42.639011  3382 consensus_meta.cc:287] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.546s	user 0.009s	sys 0.007s
I20260709 13:01:42.779198  3350 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.804044  3350 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:42.806191  3350 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:42.801713  3342 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:42.807574  3342 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:42.808416  3342 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:42.813388  3383 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:42.761181  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 1987) took 1114 ms. Trace:
I20260709 13:01:42.782796  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:42.070982  1638 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:42.584383  3470 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:42.641101  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:42.636883  3385 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:42.601521  3340 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:42.766628  3374 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:42.647614  3435 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:42.637903  3453 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:42.669533  3376 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:42.669834  3345 consensus_queue.cc:1059] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.117s
I20260709 13:01:42.792410  2303 rpcz_store.cc:276] 0709 13:01:41.760933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.761127 (+   194us) service_pool.cc:224] Handling call
0709 13:01:42.791901 (+1030774us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:42.802141  3452 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:42.070633  3347 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:42.658816  3389 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:42.782105  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:42.893036  2302 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:42.894274  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:42.779443  1636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1526) took 2349 ms (client timeout 1590 ms). Trace:
I20260709 13:01:42.097301  3457 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:42.626725  3392 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:42.821669  1642 rpcz_store.cc:276] 0709 13:01:41.647060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.659649 (+ 12589us) service_pool.cc:224] Handling call
0709 13:01:42.627303 (+967654us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:01:42.627602 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.755622 (+128020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.761113 (+  5491us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15204224}
I20260709 13:01:42.912772  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:42.913575  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:42.819240  3362 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:01:42.837585  3412 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.887s	user 0.004s	sys 0.032s
I20260709 13:01:42.834200  2297 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:42.881899  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 323) took 1226 ms. Trace:
I20260709 13:01:42.820920  3383 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:42.806784  3357 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:42.920265  1713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.959s	user 0.000s	sys 0.038s
I20260709 13:01:42.791414  3388 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:42.821358  3409 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:42.914410  1642 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 1.
I20260709 13:01:42.930677  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:42.911624  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:42.915709  3412 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.965s	user 0.004s	sys 0.032s
I20260709 13:01:42.883965  3474 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:42.916252  1609 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.951s	user 0.018s	sys 0.019s
I20260709 13:01:42.890266  3350 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:42.886772  3374 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:01:42.920629  1713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.000s	sys 0.038s
I20260709 13:01:42.822589  3414 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:42.892352  3266 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.926860  3225 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:42.919482  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:42.949867  3348 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:43.007987  3323 tablet_bootstrap.cc:492] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:01:42.917665  2300 consensus_meta.cc:287] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.214s	user 0.014s	sys 0.000s
I20260709 13:01:43.008651  2300 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:42.950950  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 322) took 1303 ms. Trace:
I20260709 13:01:42.890431  3423 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.881963  3470 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:42.927785  3435 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.007908  3348 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:42.956663  3359 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:43.009490  3420 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:42.943868  1462 catalog_manager.cc:5697] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 2, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:42.920151  1638 rpcz_store.cc:276] 0709 13:01:41.655364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.928300 (+272936us) service_pool.cc:224] Handling call
0709 13:01:42.879879 (+951579us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:43.011099  3323 tablet_bootstrap.cc:492] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:42.951503  3301 raft_consensus.cc:3055] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:42.955108  3411 raft_consensus.cc:3055] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:01:42.960454  1609 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.995s	user 0.021s	sys 0.020s
W20260709 13:01:43.032404  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 1.07894e+06us
I20260709 13:01:42.932572  2269 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:42.919895  3385 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:42.930526  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 165036us
I20260709 13:01:42.941046  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 1995) took 1348 ms. Trace:
W20260709 13:01:43.025355  3433 consensus_meta.cc:287] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.998s	user 0.010s	sys 0.002s
I20260709 13:01:42.982362  3488 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:42.959568  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:42.989292  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:43.014492  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.050554  2303 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:42.951081  3388 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:42.947279  3340 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:42.945257  3439 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:43.020079  3452 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:42.986109  3350 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.021515  3374 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:42.932883  3362 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.089s
W20260709 13:01:43.022394  3339 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.162s	user 0.028s	sys 0.046s
I20260709 13:01:43.015367  1641 rpcz_store.cc:276] 0709 13:01:41.647949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.648094 (+   145us) service_pool.cc:224] Handling call
0709 13:01:41.648382 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.029195 (+380813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:42.029226 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:42.029267 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:42.029305 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.950678 (+921373us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b54001e9454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:42.950728 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.950865 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":847384960}
I20260709 13:01:43.123440  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:43.124243  1641 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 1.
W20260709 13:01:42.927851  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 301641us
W20260709 13:01:43.127509  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.131728  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:01:42.934617  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.016s	user 0.027s	sys 0.032s
I20260709 13:01:43.009342  3449 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:43.003966  1635 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:42.931335  3460 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:01:43.034164  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 101676us
W20260709 13:01:43.031845  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.136452  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:43.136745  3448 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.034535  1609 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:43.136929  2269 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:01:42.912511  1636 rpcz_store.cc:269] 0709 13:01:40.429649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.746541 (+316892us) service_pool.cc:224] Handling call
0709 13:01:42.778407 (+2031866us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:43.137286  3448 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:43.139336  2269 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:42.944720  3357 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:42.953974  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:43.009371  2300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 471) took 1310 ms (client timeout 1618 ms). Trace:
I20260709 13:01:43.027835  3482 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:43.017637  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 98268.6us
I20260709 13:01:42.950150  3459 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:43.018785  3225 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.024629  3266 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:43.051987  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 124146us
I20260709 13:01:42.956161  3389 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:42.957280  3347 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:43.049387  1563 rpcz_store.cc:276] 0709 13:01:41.592739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.592941 (+   202us) service_pool.cc:224] Handling call
0709 13:01:42.940216 (+1347275us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:01:42.940442 (+   226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.940649 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:42.940682 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:42.940734 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:42.940780 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.940846 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.940970 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28774656}
I20260709 13:01:43.158619  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:43.159266  1563 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:43.160250  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:43.160847  1563 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:42.981221  3474 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:43.030133  3348 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:42.958777  3457 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:43.027817  3423 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.031951  3301 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:43.012145  3383 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.032230  3411 raft_consensus.cc:740] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:43.053484  3323 ts_tablet_manager.cc:1403] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 2.107s	user 0.024s	sys 0.041s
I20260709 13:01:43.052352  3488 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:43.169443  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 1984) took 1527 ms. Trace:
I20260709 13:01:43.030333  3435 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:43.079710  3362 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:43.080605  3394 raft_consensus.cc:3055] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:43.081068  3345 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.081597  3436 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:43.081765  3451 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:43.085144  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 1737) took 1076 ms. Trace:
I20260709 13:01:43.087075  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:43.087769  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:43.087875  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:43.088456  3350 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:43.088639  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:43.089486  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:43.090188  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1437) took 1340 ms. Trace:
I20260709 13:01:43.090380  3483 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.091135  3475 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.091409  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:43.093392  3484 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.094097  3392 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:43.100965  3473 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:43.102438  2297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 622) took 2868 ms (client timeout 1965 ms). Trace:
I20260709 13:01:43.103022  3486 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.090850  3340 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.090215  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:43.090847  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:43.105800  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:43.105989  1714 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:43.053715  1711 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:43.051519  3453 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:43.109527  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:43.110680  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:43.112365  1717 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:43.058228  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:43.118685  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.046219  3471 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:43.133196  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.214s	user 0.027s	sys 0.034s
I20260709 13:01:43.139190  3446 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:42.931766  2302 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
W20260709 13:01:43.140127  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 105685us
W20260709 13:01:43.122737  3339 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.306s	user 0.029s	sys 0.048s
I20260709 13:01:43.141644  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 325) took 1485 ms. Trace:
I20260709 13:01:43.127094  1637 raft_consensus.cc:2368] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5.
I20260709 13:01:43.135130  3431 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:43.148329  3225 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:43.149303  3266 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:43.110745  3470 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:43.153086  3357 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:43.153206  3487 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:43.154168  3419 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:43.155340  3366 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:43.151026  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:43.140493  1639 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 2: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:43.160442  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:43.140282  3452 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:43.139813  3448 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:43.140604  3482 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:43.141697  3433 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:43.164502  3397 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:43.144863  3460 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:43.456692  3394 raft_consensus.cc:740] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:43.336115  3267 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.216287  3409 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:43.457054  3484 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.222906  1642 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:43.457707  3340 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.265168  3383 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:43.288383  3301 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.282362  3402 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:43.429556  3404 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:43.274088  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:43.459631  3345 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:43.576359  3482 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:43.257229  3352 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:43.460423  3439 consensus_queue.cc:1059] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:43.255277  3488 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:43.278478  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 477) took 1529 ms. Trace:
I20260709 13:01:43.600095  3490 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:01:43.593480  3392 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:43.607494  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:43.608104  3392 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.607391  2301 rpcz_store.cc:276] 0709 13:01:41.748591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.751557 (+  2966us) service_pool.cc:224] Handling call
0709 13:01:43.100629 (+1349072us) spinlock_profiling.cc:243] Waited 8.77 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:01:43.277255 (+176626us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7ce9c 00007fd79eb7ca5b 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:01:43.277550 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.277820 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.277860 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:43.278070 (+   210us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:43.278130 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.278227 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.278392 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61567616}
I20260709 13:01:43.600344  3402 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:43.608749  3345 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:43.609583  2301 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:43.610036  2269 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:43.610761  2301 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 5. Ops: []
I20260709 13:01:43.461448  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:43.575438  1712 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
W20260709 13:01:43.578444  2297 rpcz_store.cc:269] 0709 13:01:40.233696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.237450 (+  3754us) service_pool.cc:224] Handling call
0709 13:01:41.057220 (+819770us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cf8c 00007fd79eb7cef6 0000000000375939 00007fd7a6bf01e0 00007fd7a6cb90c4 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:43.102189 (+2044969us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b54003a2354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:43.102324 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4020493824}
W20260709 13:01:43.577459  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 426590us
W20260709 13:01:43.381824  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.186777  1635 rpcz_store.cc:276] 0709 13:01:41.642269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.647865 (+  5596us) service_pool.cc:224] Handling call
0709 13:01:42.940112 (+1292247us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:01:42.986145 (+ 46033us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.141908 (+155763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.163477 (+ 21569us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6518656}
I20260709 13:01:43.460642  1566 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:43.254837  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:43.579268  1715 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 2: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:01:43.459043  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.192750  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:43.290959  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 480) took 1325 ms. Trace:
W20260709 13:01:43.534021  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.439467  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 307840us
I20260709 13:01:43.576397  1569 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
W20260709 13:01:43.255602  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 1989) took 1602 ms (client timeout 1685 ms). Trace:
W20260709 13:01:43.627317  1644 rpcz_store.cc:269] 0709 13:01:41.653522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.653642 (+   120us) service_pool.cc:224] Handling call
0709 13:01:43.255421 (+1601779us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54000d16d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:43.255541 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236466432}
I20260709 13:01:43.628587  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:43.629206  1644 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:43.627034  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 653) took 1011 ms. Trace:
I20260709 13:01:43.626248  2305 rpcz_store.cc:276] 0709 13:01:41.965520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.965714 (+   194us) service_pool.cc:224] Handling call
0709 13:01:43.290600 (+1324886us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b54007102d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:43.290835 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1423443840}
I20260709 13:01:43.629959  2302 rpcz_store.cc:276] 0709 13:01:42.615884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.592846 (+976962us) service_pool.cc:224] Handling call
0709 13:01:43.626511 (+ 33665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.626680 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.626702 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:43.626750 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:43.626793 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.626872 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.626982 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:43.631466  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:43.631990  1644 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:43.461066  1710 rpcz_store.cc:276] 0709 13:01:42.009217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.009615 (+   398us) service_pool.cc:224] Handling call
0709 13:01:43.084970 (+1075355us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b54001482d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:43.085082 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1399516416}
I20260709 13:01:43.582502  3449 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.458652  3471 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.583149  3374 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.583763  3323 raft_consensus.cc:359] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:43.584647  3460 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:43.584954  3431 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.640245  3323 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:43.643235  1711 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:01:43.642900  3323 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:43.296022  3320 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:01:43.294687  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:43.576850  1641 rpcz_store.cc:276] 0709 13:01:41.656076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.140900 (+1484824us) service_pool.cc:224] Handling call
0709 13:01:43.141194 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.141364 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.141386 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:43.141424 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:43.141456 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.141506 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.141594 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:43.592120  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.592290  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:43.168010  3474 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:43.653373  2304 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:43.596074  1609 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:43.459360  3420 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:43.496137  3387 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:43.576850  3366 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:43.140417  2300 rpcz_store.cc:269] 0709 13:01:41.699050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.699219 (+   169us) service_pool.cc:224] Handling call
0709 13:01:43.009287 (+1310068us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:43.644090  3332 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:43.423768  3430 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:43.184619  3348 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:43.600603  1718 raft_consensus.cc:2393] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:43.189983  3362 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:43.643678  3431 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:43.465874  3413 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:01:43.586692  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 426338us
I20260709 13:01:43.690171  1718 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:43.220703  3445 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:43.455998  3423 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:43.261551  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:43.298205  3360 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:43.603071  1567 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:43.457345  3483 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.458002  3473 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.591154  3486 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.459501  1713 rpcz_store.cc:276] 0709 13:01:41.749412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.762776 (+ 13364us) service_pool.cc:224] Handling call
0709 13:01:41.763096 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:41.851727 (+ 88631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.851774 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:41.874132 (+ 22358us) log.cc:844] Serialized 26 byte log entry
0709 13:01:43.089589 (+1215457us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b540016d3a0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0b090 00007fd7a1c186b8 00007fd7a1ca192b 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c
0709 13:01:43.089658 (+    69us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:43.089716 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.089820 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:43.089868 (+    48us) raft_consensus.cc:1685] finished
0709 13:01:43.089890 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.090123 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3431,"raft.queue_time_us":2033,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":80913,"spinlock_wait_cycles":13727616,"thread_start_us":1073519,"threads_started":1,"wal-append.queue_time_us":1050745}
I20260709 13:01:43.592252  3266 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:43.579955  3452 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:43.578153  3357 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:43.593590  3267 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:43.595525  3480 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:01:43.222080  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1520) took 3039 ms (client timeout 2071 ms). Trace:
I20260709 13:01:43.606216  3352 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:43.615897  3439 raft_consensus.cc:3055] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:43.618777  3488 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.619601  3397 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:43.621433  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:43.622085  3394 consensus_queue.cc:260] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.586340  3433 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:43.596720  3487 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.623231  3448 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:43.623641  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 162286us
I20260709 13:01:43.627381  1683 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:01:43.603904  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 326) took 1007 ms. Trace:
I20260709 13:01:43.634447  3383 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:43.635139  3484 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:43.634739  3449 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.634454  3489 consensus_queue.cc:1059] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:01:43.635876  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:43.634464  3409 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:43.630744  2269 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:43.640494  3225 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.625874  1638 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 4: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:43.640885  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1528) took 1953 ms. Trace:
I20260709 13:01:43.644505  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:43.608747  3392 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:43.663784  1714 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:43.592396  3453 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.614782  3301 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:43.665967  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:43.671401  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 75397.3us
I20260709 13:01:43.662276  3340 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:43.643760  3444 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.263159  3411 consensus_queue.cc:260] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:43.639855  3311 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:43.639756  3471 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:43.639915  3359 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.262336  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:43.591210  3475 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.681355  3497 consensus_queue.cc:1059] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:01:43.675483  3420 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.685684  3472 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:43.686228  3332 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.292878  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:43.668442  3482 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:44.056504  3472 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:44.057559  3332 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:44.058351  3482 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:43.457729  3451 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:43.995110  1639 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:44.059355  3497 raft_consensus.cc:3055] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:44.059742  3497 raft_consensus.cc:740] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:01:43.935626  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 304967us
W20260709 13:01:43.903168  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 210573us
W20260709 13:01:43.938301  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 311012us
I20260709 13:01:43.937376  1641 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:43.940418  1644 rpcz_store.cc:276] 0709 13:01:41.687167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.640052 (+1952885us) service_pool.cc:224] Handling call
0709 13:01:43.640317 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.640564 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.640590 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:43.640629 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:43.640661 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.640710 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.640830 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:43.941344  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:43.678309  3366 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:43.912873  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.941877  1715 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
W20260709 13:01:43.938858  1640 rpcz_store.cc:269] 0709 13:01:40.182497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.230731 (+ 48234us) service_pool.cc:224] Handling call
0709 13:01:43.221769 (+2991038us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54003efb54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:43.221930 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3287343360}
I20260709 13:01:43.769858  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1527) took 2083 ms. Trace:
W20260709 13:01:43.946619  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.947187  3348 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.948311  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:43.951498  3225 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:44.003153  1709 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:01:44.005604  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:44.041870  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.095932  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.995545  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.026383  1714 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
W20260709 13:01:43.851830  3320 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:01:44.101413  3267 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:44.101567  3350 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:44.105137  3320 tablet_bootstrap.cc:492] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:01:44.106225  3320 tablet_bootstrap.cc:492] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:01:44.106289  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:44.062050  2299 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:44.107314  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:44.000923  3433 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:44.108093  3320 ts_tablet_manager.cc:1403] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 3.513s	user 0.044s	sys 0.042s
I20260709 13:01:44.108685  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:44.109344  1566 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:01:44.089672  1713 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:44.099577  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:44.003455  1561 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:43.767881  1466 catalog_manager.cc:5697] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 2, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:01:43.768297  3435 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:43.938552  3439 raft_consensus.cc:740] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:44.087751  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:44.089172  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:44.061457  2269 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:43.944698  3452 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:43.998744  3323 ts_tablet_manager.cc:1434] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.808s	user 0.008s	sys 0.010s
W20260709 13:01:44.324935  3323 ts_tablet_manager.cc:1467] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Tablet startup took 1137ms
I20260709 13:01:43.951012  3397 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:44.099812  1565 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:44.060369  3359 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:44.051939  1712 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:01:43.938799  3383 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:43.694711  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:43.762370  3374 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:44.004097  3444 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.722067  3445 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:43.997987  3352 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:44.111317  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:43.936764  1637 rpcz_store.cc:276] 0709 13:01:42.596658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.926730 (+330072us) service_pool.cc:224] Handling call
0709 13:01:43.599808 (+673078us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:43.994513  2298 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:44.206450  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:44.248281  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1448) took 1106 ms. Trace:
I20260709 13:01:43.701385  3413 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:44.230947  3453 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:44.002547  3487 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:44.395955  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:44.397248  1637 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:44.397912  3413 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:44.109213  3420 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:44.222223  3402 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:44.213439  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:44.212424  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:44.176990  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:44.213636  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:44.111397  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 163143us
I20260709 13:01:44.219516  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:44.112372  1683 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:44.206076  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:44.213744  3449 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:44.225937  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:44.212760  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:44.193285  3414 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:44.087276  1636 rpcz_store.cc:276] 0709 13:01:41.686791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.138346 (+1451555us) service_pool.cc:224] Handling call
0709 13:01:43.138705 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.464468 (+325763us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.502478 (+ 38010us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:43.555647 (+ 53169us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:43.565186 (+  9539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.769567 (+204381us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x7b540032c3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:43.769617 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.769772 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113426688}
I20260709 13:01:44.213073  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:44.225471  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:43.993803  3488 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:44.316537  3451 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:44.297643  1641 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:44.091159  3311 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:44.050822  3475 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:43.995469  3387 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:44.355949  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:44.003736  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 334) took 1189 ms. Trace:
I20260709 13:01:44.226178  1682 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:44.145495  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.210575  3225 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:44.369686  1713 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:44.322177  3483 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:43.951361  3357 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:44.384711  1715 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:43.951035  3448 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:44.382644  1714 rpcz_store.cc:276] 0709 13:01:43.141455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.146059 (+1004604us) service_pool.cc:224] Handling call
0709 13:01:44.247459 (+101400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.247831 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:44.247863 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:44.247922 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:44.247969 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.248048 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.248207 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:44.096201  3385 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:44.173108  3470 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:44.213909  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:44.285871  1533 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.520s (SENT) [suppressed 19 similar messages]
I20260709 13:01:44.101429  3486 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:44.226061  3433 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.185s
W20260709 13:01:44.690810  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.316280  3497 consensus_queue.cc:260] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:44.551622  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.785141  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2000) took 1146 ms. Trace:
W20260709 13:01:44.785677  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.785705  1641 rpcz_store.cc:276] 0709 13:01:43.638252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.088899 (+450647us) service_pool.cc:224] Handling call
0709 13:01:44.785084 (+696185us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:44.782184  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 720834us
W20260709 13:01:44.684176  1607 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.000s	sys 0.009s
W20260709 13:01:44.787761  1607 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.002s	sys 0.010s
I20260709 13:01:44.788765  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:44.789779  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:44.791420  2303 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:01:44.369143  3383 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:44.490821  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.786345  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.788098  2269 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:44.419085  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:44.211998  3404 consensus_meta.cc:287] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.000s
I20260709 13:01:44.417272  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1446) took 1295 ms. Trace:
W20260709 13:01:44.416596  3474 consensus_meta.cc:287] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.746s	user 0.011s	sys 0.000s
I20260709 13:01:44.398555  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:44.818512  1643 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:44.802189  1718 rpcz_store.cc:276] 0709 13:01:43.121442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.691059 (+569617us) service_pool.cc:224] Handling call
0709 13:01:43.691358 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:43.999159 (+307801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:43.999191 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:43.999229 (+    38us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:43.999262 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.416935 (+417673us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b54001478d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:44.416997 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.417198 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":707456768}
I20260709 13:01:44.791494  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:44.796128  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1711) took 1710 ms (client timeout 1925 ms). Trace:
I20260709 13:01:44.820088  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:44.796339  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 1748) took 1035 ms. Trace:
I20260709 13:01:44.821100  1715 rpcz_store.cc:276] 0709 13:01:43.760658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.092419 (+331761us) service_pool.cc:224] Handling call
0709 13:01:44.796281 (+703862us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:44.793831  2268 connection.cc:256] RPC call timeout handler was delayed by -1.7022601850000569s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15500000000000003s for a response
W20260709 13:01:44.626573  3301 consensus_meta.cc:287] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.682s	user 0.010s	sys 0.001s
I20260709 13:01:44.779284  3225 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.315829  3473 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:44.785960  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:44.375093  3435 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:44.313995  3409 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:44.624194  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:44.547598  3332 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.823585  1710 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 2: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:44.823702  1714 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:44.415727  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:44.842370  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:44.410153  3487 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.834453  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:44.843036  3404 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:44.843573  3404 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:44.843750  3471 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.784s	user 0.008s	sys 0.001s
I20260709 13:01:44.844020  3404 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:44.844250  3471 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:44.844444  3332 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:44.845592  2300 raft_consensus.cc:2368] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3.
I20260709 13:01:44.848861  3397 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:44.844129  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:44.851440  1641 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:01:44.823361  3225 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:44.410686  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 665) took 1280 ms. Trace:
I20260709 13:01:44.829460  1532 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:44.829689  1718 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:44.417941  3472 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:44.783392  3350 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:44.351675  3392 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:44.822649  2268 connection.cc:256] RPC call timeout handler was delayed by -1.55736018500005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17110000000000003s for a response
I20260709 13:01:44.616324  3366 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:44.834997  3433 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:44.839109  1635 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:44.828497  3409 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.819366  3372 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.786712  1638 rpcz_store.cc:276] 0709 13:01:42.814174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.049790 (+235616us) service_pool.cc:224] Handling call
0709 13:01:43.994222 (+944432us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:44.809538  2297 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:44.840399  3320 raft_consensus.cc:359] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:44.840869  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:44.894479  1710 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
W20260709 13:01:44.820129  1567 rpcz_store.cc:269] 0709 13:01:43.085718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.090610 (+  4892us) service_pool.cc:224] Handling call
0709 13:01:44.795978 (+1705368us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54001cdc54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:44.796075 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":335795456}
I20260709 13:01:44.892722  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1450) took 1747 ms. Trace:
I20260709 13:01:44.843315  3487 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:44.852371  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:44.832890  3420 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:44.128082  3348 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:44.853732  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1449) took 1710 ms. Trace:
I20260709 13:01:44.853807  2269 leader_election.cc:400] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5.
I20260709 13:01:44.787357  3470 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:44.796813  3387 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:44.562513  3480 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:01:44.863265  2268 connection.cc:256] RPC call timeout handler was delayed by -1.4313601850000168s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
I20260709 13:01:44.416091  3413 consensus_queue.cc:1059] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.416s
I20260709 13:01:44.801314  3445 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:44.890713  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:44.802915  3449 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:44.889318  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 663215us
I20260709 13:01:44.866251  1532 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:44.801343  2299 raft_consensus.cc:2376] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3. Re-sending same reply.
I20260709 13:01:44.896097  1715 rpcz_store.cc:276] 0709 13:01:43.145818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.829970 (+1684152us) service_pool.cc:224] Handling call
0709 13:01:44.892377 (+ 62407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.892502 (+   125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:44.892516 (+    14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:44.892542 (+    26us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:44.892564 (+    22us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.892595 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:44.892676 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:44.601001  3359 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:44.903820  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 791502us
I20260709 13:01:44.796389  3452 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:44.798822  3414 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:44.804755  3483 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 13:01:44.825121  3474 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:01:44.415644  3439 consensus_queue.cc:260] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:44.872961  3471 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:45.156164  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:45.152907  1534 leader_election.cc:336] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.965s (SENT)
I20260709 13:01:45.159992  2269 leader_election.cc:403] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:44.831926  3435 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:45.037112  2304 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 479) took 3273 ms (client timeout 1507 ms). Trace:
I20260709 13:01:45.160934  1569 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:01:45.161022  3512 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:45.101287  3360 raft_consensus.cc:3055] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:45.038738  1713 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
W20260709 13:01:44.325356  3323 ts_tablet_manager.cc:1469] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Trace:
0709 13:01:35.346337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.346477 (+   140us) service_pool.cc:224] Handling call
0709 13:01:35.390404 (+ 43927us) ts_tablet_manager.cc:934] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
0709 13:01:39.032441 (+3642037us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:01:39.225633 (+193192us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:39.229570 (+  3937us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:01:39.417033 (+187463us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:01:39.417314 (+   281us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:39.417595 (+   281us) ts_tablet_manager.cc:989] Replica registered.
0709 13:01:39.417692 (+    97us) inbound_call.cc:177] Queueing success response
0709 13:01:40.946173 (+1528481us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:01:43.192295 (+2246122us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:01:43.299615 (+107320us) tablet_replica.cc:261] Starting instrumentation
0709 13:01:43.423692 (+124077us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":279,"mutex_wait_us":1414,"raft.queue_time_us":7190,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":4548,"tablet-copy.queue_time_us":40815,"tablet-download-pool.queue_time_us":65728,"tablet-download-pool.run_cpu_time_us":1225,"tablet-download-pool.run_wall_time_us":389701,"thread_start_us":46396,"threads_started":2}
I20260709 13:01:44.907155  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.161852  3480 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:44.890511  3383 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:44.831312  1637 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:45.162472  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:45.162930  2268 connection.cc:256] RPC call timeout handler was delayed by -1.2495601850000639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
I20260709 13:01:45.028338  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:45.009572  3507 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:44.963462  1642 raft_consensus.cc:2368] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3.
I20260709 13:01:44.969213  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:44.960911  3225 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:45.005321  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 215634us
I20260709 13:01:44.956598  3397 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:45.164808  1718 rpcz_store.cc:276] 0709 13:01:43.143028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.829153 (+1686125us) service_pool.cc:224] Handling call
0709 13:01:44.829465 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:44.853338 (+ 23873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.853658 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:45.163765  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 297636us
W20260709 13:01:44.956928  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 671110us
I20260709 13:01:45.029871  3506 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:44.858785  3503 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.350s
I20260709 13:01:44.890859  1639 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
W20260709 13:01:45.199750  1681 leader_election.cc:336] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.532s (SENT)
W20260709 13:01:45.198825  2304 rpcz_store.cc:269] 0709 13:01:41.763641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.763760 (+   119us) service_pool.cc:224] Handling call
0709 13:01:43.254640 (+1490880us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:45.036858 (+1782218us) spinlock_profiling.cc:243] Waited 26.4 ms on lock 0x7b54005d2ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:45.037023 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66070144}
I20260709 13:01:44.862738  3350 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:44.891692  3433 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.863001  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:45.197778  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:44.893852  3409 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:45.028076  3448 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:45.035547  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:45.034162  3487 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:45.165674  3420 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.902573  3372 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:45.292459  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.354840  1710 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 2: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:01:45.355020  2268 connection.cc:256] RPC call timeout handler was delayed by -1.1631601850000379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2149s for a response
I20260709 13:01:45.263429  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:44.903599  3320 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:01:45.032033  3488 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:45.296700  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.370244  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2016) took 1054 ms. Trace:
I20260709 13:01:45.037161  3366 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:45.008214  3473 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:45.164014  3449 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.166765  3470 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:45.033033  3446 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:45.208966  3512 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:01:45.264050  3480 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:45.292896  3509 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.166538  3387 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:45.371632  1638 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:45.395365  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2008) took 1152 ms. Trace:
I20260709 13:01:45.395745  1638 rpcz_store.cc:276] 0709 13:01:44.243021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.969668 (+726647us) service_pool.cc:224] Handling call
0709 13:01:45.395290 (+425622us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.396839  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:45.350486  3506 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.295388  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.398975  3506 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:45.399945  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.400875  1562 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 3: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:45.394793  3350 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:45.401355  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2010) took 1087 ms. Trace:
I20260709 13:01:45.294060  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.351771  1714 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:45.402879  1566 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:45.403232  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:45.403841  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2014) took 1088 ms. Trace:
I20260709 13:01:45.404286  1566 rpcz_store.cc:276] 0709 13:01:44.315031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.293660 (+978629us) service_pool.cc:224] Handling call
0709 13:01:45.403760 (+110100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.404844  1714 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:45.405479  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1451) took 1789 ms. Trace:
I20260709 13:01:45.286604  3452 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:44.907316  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2002) took 1268 ms. Trace:
I20260709 13:01:45.406291  1635 rpcz_store.cc:276] 0709 13:01:43.638676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.623724 (+985048us) service_pool.cc:224] Handling call
0709 13:01:44.903079 (+279355us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:45.352525  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2003) took 1713 ms (client timeout 1650 ms). Trace:
W20260709 13:01:45.297721  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 1740) took 2192 ms (client timeout 1806 ms). Trace:
W20260709 13:01:45.407698  1709 rpcz_store.cc:269] 0709 13:01:43.104927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.105481 (+   554us) service_pool.cc:224] Handling call
0709 13:01:45.297388 (+2191907us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54001351d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:45.297568 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250091008}
I20260709 13:01:45.357007  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:45.371340  1636 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:45.408900  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2007) took 1188 ms. Trace:
I20260709 13:01:45.409161  1709 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:45.409276  3350 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:45.409817  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1452) took 1793 ms. Trace:
I20260709 13:01:45.296759  3348 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:45.347141  3433 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:45.410192  3435 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:45.412354  3411 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:45.359143  3504 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.359694  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.415248  3504 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:44.970103  3385 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:45.356318  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:45.416324  3383 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:45.416764  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 60517.6us
I20260709 13:01:45.418092  3483 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.363659  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2015) took 1048 ms. Trace:
I20260709 13:01:45.225447  1644 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:45.418601  3483 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:45.418908  1567 rpcz_store.cc:276] 0709 13:01:44.315355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.906817 (+591462us) service_pool.cc:224] Handling call
0709 13:01:45.363600 (+456783us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:45.419624  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2004) took 1780 ms (client timeout 2005 ms). Trace:
I20260709 13:01:45.420109  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.420639  3483 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:45.421154  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2022) took 1010 ms. Trace:
I20260709 13:01:45.421664  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.171228  3404 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:45.422420  1564 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:45.224428  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:45.423163  1569 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:45.348011  3439 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:45.370654  1563 rpcz_store.cc:276] 0709 13:01:44.315639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.296371 (+980732us) service_pool.cc:224] Handling call
0709 13:01:45.370177 (+ 73806us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.360989  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:45.424438  3404 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:44.890879  3472 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:44.862638  2298 rpcz_store.cc:276] 0709 13:01:43.130597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.261219 (+130622us) service_pool.cc:224] Handling call
0709 13:01:44.399269 (+1138050us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:45.376226  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 210618us
I20260709 13:01:45.265321  3471 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5.
I20260709 13:01:45.202232  3360 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:45.360348  3332 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:45.381491  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:45.362401  3420 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:45.262748  3474 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:45.298243  3414 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:45.280889  3460 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:01:45.376505  3357 consensus_meta.cc:287] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.575s	user 0.007s	sys 0.003s
I20260709 13:01:45.351131  3507 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:45.392374  3512 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:45.405336  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.407246  3320 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:45.415493  1568 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:01:45.417018  3383 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:45.421573  1567 rpcz_store.cc:276] 0709 13:01:44.410726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.419784 (+1009058us) service_pool.cc:224] Handling call
0709 13:01:45.421089 (+  1305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.423322  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:45.358860  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2011) took 1060 ms. Trace:
I20260709 13:01:45.469296  1641 rpcz_store.cc:276] 0709 13:01:44.297979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.028021 (+730042us) service_pool.cc:224] Handling call
0709 13:01:45.358801 (+330780us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:45.359545  1637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 1999) took 1721 ms (client timeout 2070 ms). Trace:
I20260709 13:01:45.424901  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.478758  3352 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:45.479454  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2017) took 1163 ms. Trace:
I20260709 13:01:45.479861  1563 rpcz_store.cc:276] 0709 13:01:44.334295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.424650 (+1090355us) service_pool.cc:224] Handling call
0709 13:01:45.479375 (+ 54725us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.481024  1563 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:45.481848  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2026) took 1006 ms. Trace:
I20260709 13:01:45.482195  1563 rpcz_store.cc:276] 0709 13:01:44.475847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.480534 (+1004687us) service_pool.cc:224] Handling call
0709 13:01:45.480850 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.481553 (+   703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.481762 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:45.425288  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 68370.5us
I20260709 13:01:45.170364  3451 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:45.410209  1709 rpcz_store.cc:276] 0709 13:01:43.616000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.408549 (+1792549us) service_pool.cc:224] Handling call
0709 13:01:45.408839 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.409596 (+   757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.409767 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.426780  1714 rpcz_store.cc:276] 0709 13:01:43.615670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.404296 (+1788626us) service_pool.cc:224] Handling call
0709 13:01:45.404562 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.405249 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.405428 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:45.205154  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 384675us
W20260709 13:01:45.420015  1644 rpcz_store.cc:269] 0709 13:01:43.639171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.099316 (+460145us) service_pool.cc:224] Handling call
0709 13:01:45.419468 (+1320152us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002d8054. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:45.419568 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2561848064}
I20260709 13:01:45.378269  3449 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:45.428360  1683 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:01:45.407109  1639 rpcz_store.cc:269] 0709 13:01:43.638832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.096471 (+457639us) service_pool.cc:224] Handling call
0709 13:01:45.352451 (+1255980us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:45.429697  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.448776  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2019) took 1093 ms. Trace:
I20260709 13:01:45.448825  1710 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:01:45.511524  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1456) took 1381 ms. Trace:
W20260709 13:01:45.512387  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.401715  1562 rpcz_store.cc:276] 0709 13:01:44.313654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.399676 (+1086022us) service_pool.cc:224] Handling call
0709 13:01:45.401298 (+  1622us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:45.478331  1637 rpcz_store.cc:269] 0709 13:01:43.638014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.398265 (+760251us) service_pool.cc:224] Handling call
0709 13:01:45.359474 (+961209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.427870  3472 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.513865  3472 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:45.514151  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:45.457134  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2012) took 1142 ms. Trace:
I20260709 13:01:45.514777  1637 raft_consensus.cc:2376] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2. Re-sending same reply.
I20260709 13:01:45.515041  1568 rpcz_store.cc:276] 0709 13:01:44.314386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.359396 (+1045010us) service_pool.cc:224] Handling call
0709 13:01:45.457066 (+ 97670us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.457198  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2009) took 1260 ms. Trace:
I20260709 13:01:45.446002  3484 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:45.409253  1636 rpcz_store.cc:276] 0709 13:01:44.220951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.819273 (+598322us) service_pool.cc:224] Handling call
0709 13:01:45.408845 (+589572us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.459875  3360 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:45.511569  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1706) took 2417 ms. Trace:
I20260709 13:01:45.517191  3360 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:45.512154  1714 rpcz_store.cc:276] 0709 13:01:44.130320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.510498 (+1380178us) service_pool.cc:224] Handling call
0709 13:01:45.510843 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.511067 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.511108 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:45.511169 (+    61us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:45.511216 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.511293 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.511444 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.460366  3357 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:45.457289  3460 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:45.462747  1466 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45164 (request call id 361) took 1846 ms. Trace:
I20260709 13:01:45.511059  1564 rpcz_store.cc:276] 0709 13:01:44.355930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.426137 (+1070207us) service_pool.cc:224] Handling call
0709 13:01:45.448277 (+ 22140us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.448457 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.448485 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:45.448526 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:45.448560 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.448609 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.448716 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.465315  3497 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:45.480849  3397 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:45.383001  3473 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:45.385731  3446 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:45.386629  3509 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:45.485517  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:45.367169  1681 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:45.486259  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2027) took 1010 ms. Trace:
W20260709 13:01:45.575073  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 89640us
I20260709 13:01:45.225818  1640 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:45.484503  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2020) took 1113 ms. Trace:
W20260709 13:01:45.576480  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 376798us
I20260709 13:01:45.576885  1567 rpcz_store.cc:276] 0709 13:01:44.371038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.468470 (+1097432us) service_pool.cc:224] Handling call
0709 13:01:45.483840 (+ 15370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.484056 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.484093 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:45.484147 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:45.484203 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.484271 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.484435 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:45.507354  2269 connection.cc:256] RPC call timeout handler was delayed by -1.0033502480000038s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
I20260709 13:01:45.577661  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:45.578297  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.445058  3348 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:45.509531  3451 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.559669  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:45.538537  1533 leader_election.cc:336] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.590s (SENT)
I20260709 13:01:45.547180  1709 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:45.553138  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 129837us
I20260709 13:01:45.553603  3460 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:45.554131  3357 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:45.579885  3520 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:45.592451  3409 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:45.593344  3433 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:45.594170  3394 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:45.515830  1566 rpcz_store.cc:276] 0709 13:01:44.196504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.405078 (+1208574us) service_pool.cc:224] Handling call
0709 13:01:45.457128 (+ 52050us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.596120  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 1746) took 1902 ms. Trace:
I20260709 13:01:45.596530  1718 rpcz_store.cc:276] 0709 13:01:43.693947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.424983 (+1731036us) service_pool.cc:224] Handling call
0709 13:01:45.425269 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.485934 (+ 60665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.485983 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:45.486043 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:45.486100 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.595903 (+109803us) spinlock_profiling.cc:243] Waited 60.4 ms on lock 0x7b5400148cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:45.595944 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.596064 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138922752}
I20260709 13:01:45.597204  3387 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:45.555259  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 104640us
W20260709 13:01:45.580448  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 152233us
I20260709 13:01:45.560305  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:45.582103  1569 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:45.600522  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2024) took 1157 ms. Trace:
I20260709 13:01:45.600747  1569 rpcz_store.cc:276] 0709 13:01:44.443387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.199464 (+756077us) service_pool.cc:224] Handling call
0709 13:01:45.600440 (+400976us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.565783  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1447) took 2444 ms. Trace:
I20260709 13:01:45.588864  1533 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:45.567759  3362 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:45.569366  1711 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 1727) took 3997 ms (client timeout 1562 ms). Trace:
W20260709 13:01:45.571767  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 410227us
I20260709 13:01:45.416792  3385 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:45.510195  3471 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.575310  1563 rpcz_store.cc:276] 0709 13:01:44.476212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.485448 (+1009236us) service_pool.cc:224] Handling call
0709 13:01:45.485710 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.485854 (+   144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.485888 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:45.485936 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:45.485982 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.486053 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.486195 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.525166  3445 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:01:45.578747  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.578855  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:01:45.579290  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 426454us
I20260709 13:01:45.517429  1561 rpcz_store.cc:276] 0709 13:01:43.084320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.086767 (+  2447us) service_pool.cc:224] Handling call
0709 13:01:45.501116 (+2414349us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b5400297cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:45.501329 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":980533504}
I20260709 13:01:45.555531  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1458) took 1361 ms. Trace:
W20260709 13:01:45.601984  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:45.599943  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 335) took 2461 ms (client timeout 1512 ms). Trace:
I20260709 13:01:45.601449  1716 rpcz_store.cc:276] 0709 13:01:43.121732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.131264 (+1009532us) service_pool.cc:224] Handling call
0709 13:01:45.303047 (+1171783us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.303315 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.303356 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:45.303418 (+    62us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:45.303487 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.303571 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.565549 (+261978us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001351d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:45.565695 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2503895296}
W20260709 13:01:45.604632  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 66176.9us
I20260709 13:01:45.589783  3320 ts_tablet_manager.cc:1434] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 1.481s	user 0.011s	sys 0.049s
I20260709 13:01:45.629089  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:45.581117  3507 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:45.632524  3471 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:45.633608  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:45.639665  1710 rpcz_store.cc:276] 0709 13:01:44.193833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.512062 (+1318229us) service_pool.cc:224] Handling call
0709 13:01:45.516903 (+  4841us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.555271 (+ 38368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.555302 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:45.555331 (+    29us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:45.555350 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.555389 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.555487 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:45.633471  3366 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:45.650260  1718 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:45.588655  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:45.639214  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.589895  3504 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:45.584519  3473 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:45.639788  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:45.640730  1533 leader_election.cc:336] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.665s (SENT)
W20260709 13:01:45.640460  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 61708.3us
I20260709 13:01:45.663412  1607 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:45.586079  3483 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:01:45.640283  1642 rpcz_store.cc:269] 0709 13:01:43.138262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.607181 (+468919us) service_pool.cc:224] Handling call
0709 13:01:45.599761 (+1992580us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54003efb54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:45.599875 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3100781952}
I20260709 13:01:45.635934  3362 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:45.637715  3445 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:45.590607  3435 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:45.580106  3452 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:45.657285  1534 leader_election.cc:336] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.610s (SENT)
I20260709 13:01:45.521133  1466 rpcz_store.cc:276] 0709 13:01:43.616469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.616584 (+   115us) service_pool.cc:224] Handling call
0709 13:01:45.462637 (+1846053us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:44.303429 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:44.304162 (+   733us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:44.304217 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:01:44.304247 (+    30us) tablet.cc:662] Decoding operations
0709 13:01:44.304711 (+   464us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:44.304783 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:44.304833 (+    50us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:01:44.305083 (+   250us) tablet.cc:701] Row locks acquired
0709 13:01:44.305121 (+    38us) write_op.cc:260] PREPARE: finished
0709 13:01:44.305360 (+   239us) write_op.cc:270] Start()
0709 13:01:44.305509 (+   149us) write_op.cc:276] Timestamp: P: 1783602104305290 usec, L: 0
0709 13:01:44.305540 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:01:44.415923 (+110383us) log.cc:844] Serialized 541 byte log entry
0709 13:01:44.792976 (+377053us) op_driver.cc:464] REPLICATION: finished
0709 13:01:45.264203 (+471227us) write_op.cc:301] APPLY: starting
0709 13:01:45.264286 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:45.264406 (+   120us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:45.280838 (+ 16432us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:45.281216 (+   378us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:45.281501 (+   285us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:01:45.281554 (+    53us) write_op.cc:312] APPLY: finished
0709 13:01:45.281775 (+   221us) log.cc:844] Serialized 23 byte log entry
0709 13:01:45.281962 (+   187us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:45.282041 (+    79us) write_op.cc:454] Released schema lock
0709 13:01:45.282099 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:45.352232 (+ 70133us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b540013f954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c9ce28 00007fd7a56d7581 00007fd7a56d6d58 00007fd7a56db9f1 00007fd7a56db9a9 00007fd7a56db939 00007fd7a56db901 00007fd7a56dab2d 000000000038f354 000000000038f199
Metrics: {"mutex_wait_us":2071,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":464000,"apply.run_cpu_time_us":2316,"apply.run_wall_time_us":18342,"num_ops":1,"prepare.queue_time_us":134,"prepare.run_cpu_time_us":19404,"prepare.run_wall_time_us":1048980,"replication_time_us":183307,"spinlock_wait_cycles":252158208,"thread_start_us":263387,"threads_started":1}]]}
I20260709 13:01:45.586277  3451 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:45.689406  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:45.586055  3348 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:45.606086  3409 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:45.664291  1533 leader_election.cc:336] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.700s (SENT)
I20260709 13:01:45.699396  1533 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:45.648520  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:45.665330  1682 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:45.643325  3320 ts_tablet_manager.cc:1467] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Tablet startup took 1532ms
I20260709 13:01:45.666785  1718 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:45.700359  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51928.3us
I20260709 13:01:45.701525  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:45.701042  1683 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:45.669667  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:01:45.638533  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:45.636775  1711 rpcz_store.cc:269] 0709 13:01:41.571671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.575954 (+  4283us) service_pool.cc:224] Handling call
0709 13:01:45.569089 (+3993135us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54003dbdd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:45.569258 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3309972224}
I20260709 13:01:45.590770  3472 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:45.676654  3504 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:45.685513  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:45.686228  3435 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:45.643393  3347 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:45.688513  1534 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:45.607045  3484 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:01:45.697292  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 94220.5us
W20260709 13:01:45.698849  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.663517  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:45.677403  3507 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:45.706276  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:45.643664  3460 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:45.705298  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:45.707261  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:45.685588  3445 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:45.678490  3522 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:01:45.715292  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:45.714344  3504 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:45.691273  3387 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:45.683568  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.764739  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:45.722363  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:01:45.722569  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 93636.3us
I20260709 13:01:45.765362  2302 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:45.765483  1642 raft_consensus.cc:2376] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2. Re-sending same reply.
I20260709 13:01:45.692457  3452 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:45.707183  3266 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:45.755498  3414 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.735033  3347 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:45.767747  1641 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:45.729063  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:45.730223  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:45.771230  1563 raft_consensus.cc:2368] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4.
I20260709 13:01:45.702162  3520 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.703428  3357 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:45.745138  3366 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.739701  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:45.707741  3362 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:45.745543  1682 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:45.798386  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 52934.1us
I20260709 13:01:45.757180  3433 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:45.725342  3435 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:45.750156  3332 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:45.751077  3345 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:45.677590  3483 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:45.714076  3512 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:45.721700  3489 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:45.721215  3383 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:45.714187  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:45.702661  3320 ts_tablet_manager.cc:1469] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Trace:
0709 13:01:35.468510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.468653 (+   143us) service_pool.cc:224] Handling call
0709 13:01:35.469639 (+   986us) ts_tablet_manager.cc:934] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
0709 13:01:39.016347 (+3546708us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:01:39.170340 (+153993us) tablet_metadata.cc:665] Metadata flushed
0709 13:01:39.170479 (+   139us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:01:39.270427 (+ 99948us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:01:39.270732 (+   305us) tablet_replica.cc:206] Creating consensus instance
0709 13:01:39.288396 (+ 17664us) ts_tablet_manager.cc:989] Replica registered.
0709 13:01:39.346549 (+ 58153us) inbound_call.cc:177] Queueing success response
0709 13:01:40.594982 (+1248433us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:01:44.108668 (+3513686us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:01:44.487188 (+378520us) tablet_replica.cc:261] Starting instrumentation
0709 13:01:44.838189 (+351001us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":220,"mutex_wait_us":55137,"raft.queue_time_us":775,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":144,"spinlock_wait_cycles":1932160,"tablet-copy.queue_time_us":131,"tablet-download-pool.queue_time_us":226517,"tablet-download-pool.run_cpu_time_us":1451,"tablet-download-pool.run_wall_time_us":287110,"thread_start_us":6895,"threads_started":1}
I20260709 13:01:45.761245  1564 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:45.762322  1609 leader_election.cc:336] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.618s (SENT)
I20260709 13:01:45.763281  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.763741  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:45.828181  1609 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:45.828621  1568 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 2: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:45.828552  2305 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:01:45.728688  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.735363  3507 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.772037  1569 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:45.773608  3414 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:45.774552  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:45.704100  3473 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:01:45.736689  1607 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:45.901808  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 165214us
I20260709 13:01:45.902943  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:45.753990  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:45.799217  1682 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:45.746109  3402 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:45.755187  2297 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:01:45.764701  3504 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:45.764467  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:45.707901  3519 consensus_queue.cc:1059] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:01:45.758324  2298 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:45.766975  3452 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:45.971464  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:45.803642  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 96112.3us
I20260709 13:01:45.804412  3266 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:45.806998  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:45.946574  3480 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:45.795732  3484 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:45.816313  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:45.816324  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.726301  3518 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.089s
I20260709 13:01:45.829298  1561 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:46.001602  1567 raft_consensus.cc:2393] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 2: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:01:45.829281  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 165035us
I20260709 13:01:45.800599  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:45.815948  3397 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:45.881624  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 119351us
I20260709 13:01:45.832639  1681 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:01:46.009088  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 178817us
I20260709 13:01:45.811452  3512 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:45.818517  3489 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:45.801218  3366 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:45.818387  3490 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:45.780341  3348 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:45.890673  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:01:45.896275  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 244735us
I20260709 13:01:45.881572  3509 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:45.932875  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 133747us
I20260709 13:01:45.972308  1560 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:46.024358  2305 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 2: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:45.995020  3480 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:45.944825  3394 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.945503  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.034550  1564 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:01:45.804647  3520 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:45.903121  3507 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:45.975164  1642 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:46.039851  1654 tablet_copy_service.cc:342] P 5184c101be204393adf3ac8b31f25ad6: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-d59c4d21b6cb46edbfc2b3d50b51812a received from {username='slave'} at 127.0.0.1:53222
I20260709 13:01:46.040354  1654 tablet_copy_service.cc:434] P 5184c101be204393adf3ac8b31f25ad6: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-d59c4d21b6cb46edbfc2b3d50b51812a on tablet d59c4d21b6cb46edbfc2b3d50b51812a with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:46.045646  1642 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:45.779827  3357 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.946903  3404 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:45.951736  3350 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:45.925045  3511 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:45.994827  2302 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:01:45.946615  3402 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:46.011504  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.060907  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.061743  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:46.062238  1561 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:45.957199  1637 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:46.063599  3402 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.064406  3366 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:46.064226  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:46.054651  3350 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:46.065394  3350 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:46.067426  3449 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.009s
I20260709 13:01:46.067755  3311 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:46.035825  3480 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.067883  3449 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:46.069128  1715 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:45.956820  1717 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
W20260709 13:01:46.014298  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.018975  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 361766us
W20260709 13:01:46.071084  1534 leader_election.cc:336] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.665s (SENT)
I20260709 13:01:46.071506  1534 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:46.073722  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.020351  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:45.974841  3266 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:46.024962  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:45.944314  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:46.020203  1711 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:46.003851  3483 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:46.028322  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.036778  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.116971  1567 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:46.118163  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:46.015965  3512 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.015684  3352 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:46.159245  3352 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.159771  3352 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:46.019421  3489 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:46.055555  1713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1459) took 1820 ms (client timeout 2346 ms). Trace:
I20260709 13:01:46.161264  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.022223  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.019961  1562 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:46.047142  3357 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:46.021468  3360 consensus_queue.cc:1059] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:01:46.172340  3360 raft_consensus.cc:3055] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:46.172719  3360 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:46.173609  3360 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:01:46.174284  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:01:46.074842  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 54556.3us
W20260709 13:01:46.176451  1712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1453) took 2560 ms (client timeout 2400 ms). Trace:
W20260709 13:01:46.176924  1712 rpcz_store.cc:269] 0709 13:01:43.616258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.644240 (+ 27982us) service_pool.cc:224] Handling call
0709 13:01:46.176277 (+2532037us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54001365d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:46.176384 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261104512}
W20260709 13:01:46.082230  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 78073.4us
I20260709 13:01:46.162379  3433 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:46.179258  1533 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.986s (SENT)
I20260709 13:01:46.179678  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:46.180714  3518 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:46.180783  1718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:01:46.179294  3433 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.181945  3433 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:46.114566  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.050585  3404 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:46.183506  3404 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.117183  2305 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:46.046644  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:01:46.119567  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.050935  3394 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:46.161764  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.228789  2302 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:01:46.161948  1568 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:46.028167  3490 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:46.161643  1713 rpcz_store.cc:269] 0709 13:01:44.235626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.791256 (+555630us) service_pool.cc:224] Handling call
0709 13:01:46.055379 (+1264123us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b540020ba54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:46.055477 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502542080}
I20260709 13:01:46.167891  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.252074  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:46.252660  1564 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:46.170567  3392 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:46.148744  3420 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.661s	user 0.002s	sys 0.008s
I20260709 13:01:46.261673  3420 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:46.264570  3420 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:46.266636  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 387) took 1438 ms. Trace:
I20260709 13:01:46.267247  1717 rpcz_store.cc:276] 0709 13:01:44.832576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.842067 (+  9491us) service_pool.cc:224] Handling call
0709 13:01:46.266330 (+1424263us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003db3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:46.266552 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2533123840}
I20260709 13:01:46.129590  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.210605  1642 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:46.162493  3509 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:46.080173  3345 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:46.084125  3266 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:46.211599  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:46.212047  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:46.278668  3509 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:46.301456  3402 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:46.181464  1718 raft_consensus.cc:606] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:01:46.305361  1718 raft_consensus.cc:614] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:46.185832  1711 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:46.214124  3531 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
W20260709 13:01:46.217056  1715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 393) took 1313 ms (client timeout 1738 ms). Trace:
W20260709 13:01:46.313654  1715 rpcz_store.cc:269] 0709 13:01:44.903652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.201411 (+297759us) service_pool.cc:224] Handling call
0709 13:01:46.216851 (+1015440us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b5400337cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:46.216998 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1635334400}
I20260709 13:01:46.217695  1714 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:46.219486  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:46.226032  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.295217  3360 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:46.200337  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.273586  1534 leader_election.cc:336] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 2.027s (SENT)
I20260709 13:01:46.325982  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:46.302520  3345 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:46.286094  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:46.330091  2302 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 2: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:46.288039  3392 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.291553  1712 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:01:46.328737  3483 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:46.332286  3509 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:46.293126  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 113909us
I20260709 13:01:46.335435  3490 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:46.335855  1533 leader_election.cc:336] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 2.071s (SENT)
I20260709 13:01:46.336294  1533 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:46.294548  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 82570.6us
I20260709 13:01:46.337510  1532 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:46.212329  3471 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:46.185276  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:01:46.291565  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:46.345016  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.185631  3527 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:46.311527  1642 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:46.360370  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 343) took 2299 ms. Trace:
I20260709 13:01:46.360776  1642 rpcz_store.cc:276] 0709 13:01:44.061037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.046399 (+1985362us) service_pool.cc:224] Handling call
0709 13:01:46.360307 (+313908us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.226686  3347 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:46.323828  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 298939us
W20260709 13:01:46.326983  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 53457.6us
I20260709 13:01:46.279943  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.292551  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.381937  1567 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:01:46.382110  1682 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:46.339668  3449 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:46.326303  2297 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.320434  3472 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:46.387118  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:46.387897  1718 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 2: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:46.387810  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.362778  2269 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:46.390278  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:46.391007  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.329613  3345 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:46.402757  1607 connection.cc:256] RPC call timeout handler was delayed by -1.2510644739999748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1806s for a response
I20260709 13:01:46.406154  3402 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:46.387562  3508 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:46.407434  3508 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.407935  3508 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:46.408036  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:46.410431  3484 consensus_queue.cc:1059] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:46.386111  3372 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:46.411971  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1547) took 1024 ms. Trace:
I20260709 13:01:46.412572  1641 rpcz_store.cc:276] 0709 13:01:45.387142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.633299 (+246157us) service_pool.cc:224] Handling call
0709 13:01:46.411695 (+778396us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400155754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:46.411888 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279526272}
I20260709 13:01:46.414196  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:46.415252  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:46.416569  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:46.416924  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1709) took 3331 ms (client timeout 2888 ms). Trace:
I20260709 13:01:46.417423  1644 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
W20260709 13:01:46.417369  1560 rpcz_store.cc:269] 0709 13:01:43.085143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.087502 (+  2359us) service_pool.cc:224] Handling call
0709 13:01:46.416735 (+3329233us) spinlock_profiling.cc:243] Waited 2.37 s on lock 0x7b5400298bd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:46.416846 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5459323264}
I20260709 13:01:46.418238  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1557) took 1028 ms. Trace:
I20260709 13:01:46.419965  1462 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60118 (request call id 227) took 4487 ms. Trace:
I20260709 13:01:46.418696  1644 rpcz_store.cc:276] 0709 13:01:45.389892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.416256 (+1026364us) service_pool.cc:224] Handling call
0709 13:01:46.418165 (+  1909us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.403517  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:46.428377  2302 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 2: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:46.429929  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:46.487452  1644 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:46.488461  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1556) took 1098 ms. Trace:
I20260709 13:01:46.488981  1644 rpcz_store.cc:276] 0709 13:01:45.389645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.429572 (+1039927us) service_pool.cc:224] Handling call
0709 13:01:46.488382 (+ 58810us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.328953  1562 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:46.380774  3526 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:46.418777  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.383417  1682 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:01:46.436625  3446 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:46.380435  3394 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:46.440181  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.542963  2304 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
W20260709 13:01:46.545516  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.547354   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:01:46.440992  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:46.446592  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.549909  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:46.420495  1462 rpcz_store.cc:276] 0709 13:01:41.959576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.973327 (+ 13751us) service_pool.cc:224] Handling call
0709 13:01:46.419847 (+4446520us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:43.613271 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:43.695218 (+ 81947us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:43.759421 (+ 64203us) write_op.cc:435] Acquired schema lock
0709 13:01:43.770805 (+ 11384us) tablet.cc:662] Decoding operations
0709 13:01:43.786013 (+ 15208us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:43.786107 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:43.786146 (+    39us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:01:43.786454 (+   308us) tablet.cc:701] Row locks acquired
0709 13:01:43.786485 (+    31us) write_op.cc:260] PREPARE: finished
0709 13:01:43.786658 (+   173us) write_op.cc:270] Start()
0709 13:01:43.786811 (+   153us) write_op.cc:276] Timestamp: P: 1783602103786594 usec, L: 0
0709 13:01:43.786832 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:01:43.787679 (+   847us) log.cc:844] Serialized 536 byte log entry
0709 13:01:43.993151 (+205472us) op_driver.cc:464] REPLICATION: finished
0709 13:01:45.376192 (+1383041us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b18001109e0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:01:45.382108 (+  5916us) write_op.cc:301] APPLY: starting
0709 13:01:45.385660 (+  3552us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:45.417461 (+ 31801us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:45.417487 (+    26us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:45.417745 (+   258us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:45.417909 (+   164us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:01:45.417958 (+    49us) write_op.cc:312] APPLY: finished
0709 13:01:45.429979 (+ 12021us) log.cc:844] Serialized 23 byte log entry
0709 13:01:45.430224 (+   245us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:45.430322 (+    98us) write_op.cc:454] Released schema lock
0709 13:01:45.430359 (+    37us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":28497,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":425057,"apply.run_cpu_time_us":1836,"apply.run_wall_time_us":51598,"num_ops":1,"prepare.queue_time_us":494489,"prepare.run_cpu_time_us":19554,"prepare.run_wall_time_us":334098,"raft.queue_time_us":82674,"raft.run_cpu_time_us":116927,"raft.run_wall_time_us":1120784,"replication_time_us":206120,"spinlock_wait_cycles":44486016,"thread_start_us":706837,"threads_started":4,"wal-append.queue_time_us":159140,"wal-append.run_cpu_time_us":6623,"wal-append.run_wall_time_us":2490449}]]}
I20260709 13:01:46.455617  3444 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:46.432540  3348 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.441578  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:46.360178  3527 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:46.461030  1568 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
W20260709 13:01:46.578797  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 209120us
I20260709 13:01:46.438870  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.467805  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1454) took 2615 ms. Trace:
I20260709 13:01:46.581740  1709 rpcz_store.cc:276] 0709 13:01:43.852818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.498124 (+1645306us) service_pool.cc:224] Handling call
0709 13:01:45.498508 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.589048 (+ 90540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.467456 (+878408us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b540033aed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:46.467717 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56789248}
I20260709 13:01:46.469048  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.402643  3471 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:01:46.475255  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 1994) took 2838 ms (client timeout 1673 ms). Trace:
W20260709 13:01:46.433940  3451 consensus_meta.cc:287] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.730s	user 0.018s	sys 0.014s
I20260709 13:01:46.484843  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.426467  1641 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
W20260709 13:01:46.595376  1641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2013) took 2296 ms (client timeout 3024 ms). Trace:
W20260709 13:01:46.595933  1641 rpcz_store.cc:269] 0709 13:01:44.298573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.413831 (+2115258us) service_pool.cc:224] Handling call
0709 13:01:46.595291 (+181460us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.597163  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:46.598145  1641 raft_consensus.cc:2368] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3.
I20260709 13:01:46.424481  3345 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:46.512905  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.516233  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:46.639904  1681 leader_election.cc:400] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4.
I20260709 13:01:46.640290  1681 leader_election.cc:403] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:01:46.640954  1681 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:46.642218  1681 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:46.521201  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:46.521948  3526 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.522146  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:01:46.523083  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 1992) took 4777 ms (client timeout 2569 ms). Trace:
W20260709 13:01:46.648120  1643 rpcz_store.cc:269] 0709 13:01:41.745720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.611141 (+865421us) service_pool.cc:224] Handling call
0709 13:01:46.522836 (+3911695us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b5400195ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:46.522988 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4741241088}
I20260709 13:01:46.524794  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1462) took 2200 ms. Trace:
I20260709 13:01:46.649367  1714 rpcz_store.cc:276] 0709 13:01:44.324130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.518612 (+1194482us) service_pool.cc:224] Handling call
0709 13:01:46.217526 (+698914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.316529 (+ 99003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.524527 (+207998us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b5400337cd0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:46.524715 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1606712192}
I20260709 13:01:46.525218  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.526258  3301 raft_consensus.cc:3055] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:46.439997  1727 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-fb294ac919b44e56960745f79e83dfc4 received from {username='slave'} at 127.0.0.1:33280
W20260709 13:01:46.666893  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.551532  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.555550   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:01:46.556351  3445 consensus_meta.cc:287] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.792s	user 0.016s	sys 0.016s
I20260709 13:01:46.558485  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.559736  3520 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:46.670770  2301 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:46.594415  1635 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:46.674152  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:46.546689  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:01:46.562139  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 178865us
W20260709 13:01:46.675458  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 128882us
I20260709 13:01:46.335460  3392 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:46.568161  1564 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:46.573022  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.573683  1609 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:01:46.578282  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 193879us
I20260709 13:01:46.579893  1533 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:46.437882  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.711530  2299 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:46.711082  3527 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:46.712498  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 493) took 1073 ms. Trace:
I20260709 13:01:46.712913  3527 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:46.713130  2299 rpcz_store.cc:276] 0709 13:01:45.639206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.643297 (+  4091us) service_pool.cc:224] Handling call
0709 13:01:46.437648 (+794351us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:46.712395 (+274747us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9708288}
I20260709 13:01:46.636641  3451 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:46.717270  3451 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:46.658046  3526 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:46.719792  3483 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:46.722759  1565 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 3: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:01:46.585842  1640 rpcz_store.cc:269] 0709 13:01:43.636773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.087444 (+450671us) service_pool.cc:224] Handling call
0709 13:01:46.474656 (+2387212us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b54001955d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:46.474828 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1971515776}
I20260709 13:01:46.724776  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:46.725898  1640 raft_consensus.cc:2368] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3.
I20260709 13:01:46.593856  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:46.727407  2269 leader_election.cc:400] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3.
I20260709 13:01:46.727329  1716 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:46.598625  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1548) took 1211 ms. Trace:
I20260709 13:01:46.606786  1567 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:46.607888  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:46.610752  3350 consensus_meta.cc:287] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.545s	user 0.041s	sys 0.030s
I20260709 13:01:46.611202  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:46.612697  3332 consensus_meta.cc:287] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.797s	user 0.016s	sys 0.020s
W20260709 13:01:46.614135  3497 consensus_meta.cc:287] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.052s	user 0.022s	sys 0.016s
W20260709 13:01:46.615424  3435 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.804s	user 0.022s	sys 0.007s
I20260709 13:01:46.665380  3444 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:46.620484  3414 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:46.583823  3348 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:46.572443  3516 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:46.758373  3516 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:01:46.758776  3489 consensus_meta.cc:287] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.011s
I20260709 13:01:46.759258  3489 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:46.760146  3404 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.576s	user 0.017s	sys 0.014s
I20260709 13:01:46.760569  3404 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:46.645401  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:46.646627  1561 consensus_meta.cc:287] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.584s	user 0.021s	sys 0.024s
W20260709 13:01:46.762337  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 117001us
I20260709 13:01:46.762456  1561 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:46.599340  3372 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.763013  3392 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:46.763195  3372 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:46.763796  3352 consensus_meta.cc:287] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.604s	user 0.016s	sys 0.015s
I20260709 13:01:46.674762  2301 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:46.764249  3352 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:46.764465  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.765101  1644 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:46.765237  1567 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:46.729611  3446 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:46.765838  3446 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.766256  3446 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:46.766378  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:46.677000  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.777236  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.778257  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.778618  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:01:46.704660  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.705433  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.779480  1561 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 2: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:01:46.779683  1565 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2018) took 2429 ms (client timeout 1707 ms). Trace:
I20260709 13:01:46.780090  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.781030  1564 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 3: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:01:46.706269  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 126452us
W20260709 13:01:46.706437  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 132847us
I20260709 13:01:46.781790  1609 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:46.741951  3332 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:46.707870  1709 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:46.685168  3445 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:46.702587  2302 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:46.788661  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 672) took 1046 ms. Trace:
I20260709 13:01:46.789095  2302 rpcz_store.cc:276] 0709 13:01:45.742668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.475116 (+732448us) service_pool.cc:224] Handling call
0709 13:01:46.788599 (+313483us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.727929  2269 leader_election.cc:403] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:46.751714  3542 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:46.729141  1568 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:46.758857  3516 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:46.674232  2296 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:46.777436  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.738514  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:46.619052  1464 catalog_manager.cc:5697] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 3 to 4, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:46.659361  3301 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:46.745719  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:46.698165  3357 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:46.805462  1563 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:46.676395  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:46.747721  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.747673  2305 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:46.749958  1642 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:46.751372  3471 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:01:46.735950  3470 consensus_meta.cc:287] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.350s	user 0.009s	sys 0.027s
W20260709 13:01:46.753726  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.754325  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:46.734824  1727 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-fb294ac919b44e56960745f79e83dfc4 on tablet fb294ac919b44e56960745f79e83dfc4 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:46.699440  3402 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:46.699680  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 494) took 1055 ms. Trace:
I20260709 13:01:46.671475  2298 raft_consensus.cc:2376] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3. Re-sending same reply.
W20260709 13:01:46.757563  3452 consensus_meta.cc:287] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.786s	user 0.028s	sys 0.029s
I20260709 13:01:46.757931  3348 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:46.822131  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.763643  3392 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:46.764395  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:46.675607  1682 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:46.562829  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.841461  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.768039  3266 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:46.768648  1711 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:01:46.771167  3528 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:01:46.773547  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 99390.1us
I20260709 13:01:46.801826  3451 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
W20260709 13:01:46.735541  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2006) took 2515 ms (client timeout 2616 ms). Trace:
I20260709 13:01:46.736106  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 333) took 1003 ms. Trace:
I20260709 13:01:46.774466  3446 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:46.775506  3507 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.736s	user 0.015s	sys 0.005s
I20260709 13:01:46.747973  3444 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.685741  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:46.862043  1643 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:46.775386  1710 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:46.783083  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1554) took 1394 ms. Trace:
I20260709 13:01:46.863343  1644 rpcz_store.cc:276] 0709 13:01:45.389094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.515766 (+1126672us) service_pool.cc:224] Handling call
0709 13:01:46.782902 (+267136us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54000d1454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:46.783028 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295549824}
I20260709 13:01:46.864621  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:46.783160  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:46.865655  1561 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:46.783218  1609 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:46.706606  1714 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:46.781793  1533 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:46.784312  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.869102  1564 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:46.784869  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.870471  1567 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:01:46.871814  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:46.734134  3350 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:46.787884  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:46.780172  1565 rpcz_store.cc:269] 0709 13:01:44.350768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.132121 (+781353us) service_pool.cc:224] Handling call
0709 13:01:46.779475 (+1647354us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b5400270954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:46.779612 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3506635008}
W20260709 13:01:46.790083  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 62839.1us
I20260709 13:01:46.873298  1566 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:46.728456  1641 rpcz_store.cc:276] 0709 13:01:45.387489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.596848 (+1209359us) service_pool.cc:224] Handling call
0709 13:01:46.598559 (+  1711us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.755858  3541 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.777858  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.818514  3470 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:46.733170  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:46.766476  3352 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:46.882387  3332 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:46.801152  2301 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:46.890082  3332 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:46.890538  3332 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:46.761636  3489 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:46.658376  3431 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:46.891793  3420 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3.
W20260709 13:01:46.803336  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 64933.5us
I20260709 13:01:46.893646  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.894703  2299 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:46.895188  2268 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:46.806356  1569 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
W20260709 13:01:46.806581  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 130286us
I20260709 13:01:46.897755  1683 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:46.806893  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 674) took 1063 ms. Trace:
I20260709 13:01:46.898082  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.898496  2305 rpcz_store.cc:276] 0709 13:01:45.743271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.185009 (+441738us) service_pool.cc:224] Handling call
0709 13:01:46.806833 (+621824us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.807961  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.899415  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:46.808465  1713 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:46.785913  3536 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:46.752806  3435 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:46.808506  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.902034  1644 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:46.907217  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 350) took 1269 ms. Trace:
I20260709 13:01:46.907707  1644 rpcz_store.cc:276] 0709 13:01:45.638096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.864344 (+1226248us) service_pool.cc:224] Handling call
0709 13:01:46.907136 (+ 42792us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.908151  1532 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:01:46.800748  3506 consensus_meta.cc:287] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.401s	user 0.012s	sys 0.022s
I20260709 13:01:46.818005  2300 rpcz_store.cc:276] 0709 13:01:45.643828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.646746 (+  2918us) service_pool.cc:224] Handling call
0709 13:01:46.500985 (+854239us) spinlock_profiling.cc:243] Waited 9.86 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:46.699613 (+198628us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22688128}
W20260709 13:01:46.814996  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2015) took 1654 ms (client timeout 2153 ms). Trace:
W20260709 13:01:46.914621  1642 rpcz_store.cc:269] 0709 13:01:45.160790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.361637 (+1200847us) service_pool.cc:224] Handling call
0709 13:01:46.814781 (+453144us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b54001e8a54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:46.814938 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":767795584}
I20260709 13:01:46.761624  1639 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:46.823082  2302 raft_consensus.cc:2368] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3.
I20260709 13:01:46.825846  3372 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:46.826877  2297 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:46.804014  3483 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:01:46.840582  3392 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.846349  2304 raft_consensus.cc:2368] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4.
I20260709 13:01:46.808693  3360 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:46.842051  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:46.930022  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:46.930488  1715 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:46.860023  3301 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:46.847183  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 171660us
I20260709 13:01:46.853571  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:46.876621  3507 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:46.935460  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 82042.9us
I20260709 13:01:46.935557  1682 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:46.935868  3539 raft_consensus.cc:3055] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:46.936357  3539 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:01:46.848811  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 84503.3us
I20260709 13:01:46.936728  1681 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:46.937215  3507 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:46.937402  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:46.938030  3507 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:46.813832  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 496) took 1148 ms. Trace:
I20260709 13:01:46.938557  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:46.834700  3512 consensus_meta.cc:287] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.649s	user 0.021s	sys 0.008s
I20260709 13:01:46.939340  3512 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:46.851428  3535 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4.
I20260709 13:01:46.822597  3404 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:46.830740  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.874258  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:46.875391  1709 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:46.866719  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 83586.7us
I20260709 13:01:46.848961  3497 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:46.913019  3471 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:46.878908  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 97175.7us
W20260709 13:01:46.853535  3433 consensus_meta.cc:287] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.671s	user 0.019s	sys 0.011s
I20260709 13:01:46.836113  3445 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:46.883401  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:46.884429  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:46.791287  1712 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:46.855163  3348 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:46.888154  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 348) took 1335 ms. Trace:
I20260709 13:01:46.888857  1561 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:46.876667  3444 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:46.854668  1562 rpcz_store.cc:276] 0709 13:01:45.732735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.512475 (+779740us) service_pool.cc:224] Handling call
0709 13:01:46.721577 (+209102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.918893  3451 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:46.800282  3542 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:46.899633  1568 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
I20260709 13:01:46.841188  3452 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:46.904748  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.905489  1683 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:46.975394  1569 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
W20260709 13:01:46.975565  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 70161.6us
I20260709 13:01:46.976369  1683 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:46.977319  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:46.978143  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1553) took 1589 ms. Trace:
I20260709 13:01:46.978492  1642 rpcz_store.cc:276] 0709 13:01:45.388854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.953956 (+1565102us) service_pool.cc:224] Handling call
0709 13:01:46.978083 (+ 24127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.909945  2269 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:46.812312  1716 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:46.918007  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.918249  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:46.980659  2305 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:46.980798  1644 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:46.941869  3357 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:46.921474  3519 consensus_queue.cc:1059] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
W20260709 13:01:46.854787  1635 rpcz_store.cc:269] 0709 13:01:44.220602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.469379 (+1248777us) service_pool.cc:224] Handling call
0709 13:01:46.441280 (+971901us) spinlock_profiling.cc:243] Waited 9.25 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:46.722863 (+281583us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21265664}
I20260709 13:01:46.926327  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1468) took 1235 ms. Trace:
I20260709 13:01:46.998723  1710 rpcz_store.cc:276] 0709 13:01:45.690821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.936413 (+245592us) service_pool.cc:224] Handling call
0709 13:01:45.936730 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.877748 (+941018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.926034 (+ 48286us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b54003395d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:46.926246 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1928359680}
W20260709 13:01:46.922449  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1546) took 1543 ms (client timeout 1665 ms). Trace:
W20260709 13:01:47.000265  1639 rpcz_store.cc:269] 0709 13:01:45.379614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.516478 (+136864us) service_pool.cc:224] Handling call
0709 13:01:46.210905 (+694427us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:46.922371 (+711466us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53366144}
I20260709 13:01:47.001420  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:47.002096  1639 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:47.002297  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.937310  3539 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:46.929309  3392 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:47.027660  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.883431  3541 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:46.934737  3301 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:46.984553  1717 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:01:46.817800  3402 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:46.873926  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:46.941582  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.949682  1609 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:46.949520  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:46.936728  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:46.939673  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:46.951100  1533 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:46.903079  1560 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:46.952600  1640 rpcz_store.cc:276] 0709 13:01:45.552591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.732789 (+1180198us) service_pool.cc:224] Handling call
0709 13:01:46.888081 (+155292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.949920  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:46.955106  1715 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:46.959335  1643 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:01:46.918694  3449 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:46.998433  3497 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:46.851773  3544 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:46.938654  3435 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:46.971889  3470 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:46.971971  3472 raft_consensus.cc:3055] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:01:46.944635  3508 consensus_meta.cc:287] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.536s	user 0.000s	sys 0.016s
I20260709 13:01:46.965768  3490 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:46.939599  3489 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:46.974057  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 329) took 1241 ms. Trace:
I20260709 13:01:46.922554  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 675) took 1116 ms. Trace:
I20260709 13:01:46.918612  2303 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:46.938810  2296 rpcz_store.cc:276] 0709 13:01:45.665103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.665758 (+   655us) service_pool.cc:224] Handling call
0709 13:01:46.438713 (+772955us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:46.811239 (+372526us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23513472}
I20260709 13:01:46.951897  3348 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.918612  3506 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:46.962062  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:46.884918  3414 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:46.988776  1718 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:46.992049  1644 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:46.968245  3372 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:46.970330  3507 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:47.060212  3539 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:47.002645  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1552) took 1614 ms. Trace:
I20260709 13:01:47.117493  1639 rpcz_store.cc:276] 0709 13:01:45.388601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.001139 (+1612538us) service_pool.cc:224] Handling call
0709 13:01:47.002588 (+  1449us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.118498  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:47.119417  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:47.120205  2296 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:47.120674  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 638) took 1185 ms. Trace:
I20260709 13:01:47.120983  2296 rpcz_store.cc:276] 0709 13:01:45.935701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.119160 (+1183459us) service_pool.cc:224] Handling call
0709 13:01:47.120615 (+  1455us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.121937  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:47.122509  2296 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.111667  3372 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.123059  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:47.123401  3372 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.123844  1635 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:47.123914  3507 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:47.126389  3543 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 13:01:47.115949  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:47.127295  2300 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:47.128350  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:47.128999  2300 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:01:47.113881  3266 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:47.131264  1638 raft_consensus.cc:2368] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3.
I20260709 13:01:47.133181  3541 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:47.134603  1638 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1539) took 1965 ms (client timeout 1520 ms). Trace:
W20260709 13:01:47.135082  1638 rpcz_store.cc:269] 0709 13:01:45.169163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.396527 (+227364us) service_pool.cc:224] Handling call
0709 13:01:47.134375 (+1737848us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b54002d8f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:47.134529 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3987349888}
I20260709 13:01:47.136250  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:47.136942  1638 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:47.137840  3483 raft_consensus.cc:3055] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:47.137952  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:47.138278  3483 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:47.138552  1638 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:01:47.013410  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:47.028839  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:47.139225  3483 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:47.139899  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:47.140877  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1473) took 1069 ms. Trace:
I20260709 13:01:47.141008  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:47.141559  2301 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:47.141340  1717 rpcz_store.cc:276] 0709 13:01:46.071598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.268383 (+196785us) service_pool.cc:224] Handling call
0709 13:01:46.268856 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:47.054859 (+786003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.140647 (+ 85788us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b5400268ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:47.140822 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1554486144}
W20260709 13:01:47.141963  3480 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.073s	user 0.019s	sys 0.011s
I20260709 13:01:46.966958  3433 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:47.030934  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 135826us
I20260709 13:01:47.128346  3506 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:47.143960  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:47.123142  3350 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:47.144510  3433 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:47.145166  3350 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.041301  3547 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:47.145380  3470 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:46.938270  1681 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:47.145684  3547 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:47.145853  3470 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:46.984714  3360 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:47.146131  3526 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:47.047907  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 345) took 1539 ms (client timeout 1954 ms). Trace:
I20260709 13:01:47.147120  3445 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:47.147114  1643 rpcz_store.cc:269] 0709 13:01:45.508665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.685475 (+1176810us) service_pool.cc:224] Handling call
0709 13:01:47.047851 (+362376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.149294  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1530) took 5461 ms. Trace:
W20260709 13:01:47.048889  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 112240us
I20260709 13:01:47.049839  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:46.964031  2298 raft_consensus.cc:2393] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:46.984517  3431 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:47.150291  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 644) took 1208 ms. Trace:
I20260709 13:01:47.150691  2298 rpcz_store.cc:276] 0709 13:01:45.941782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.830404 (+888622us) service_pool.cc:224] Handling call
0709 13:01:47.150230 (+319826us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.151382  3431 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:47.151727  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:47.152009  3431 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.152437  2298 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:47.153439  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:46.985220  3451 raft_consensus.cc:697] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:46.996527  3352 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.154687  3451 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.155253  3352 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.053264  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:47.070603  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 160779us
I20260709 13:01:47.042793  1636 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 2: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:47.157785  3452 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:47.158078  2304 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
W20260709 13:01:47.158509  3345 consensus_meta.cc:287] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.510s	user 0.003s	sys 0.008s
I20260709 13:01:47.003976  3446 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:47.054731  3402 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.068717  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:47.159004  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:47.159618  2305 raft_consensus.cc:2393] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:47.159675  2304 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:47.160100  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 645) took 1217 ms. Trace:
I20260709 13:01:47.160564  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:47.161160  2304 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:47.161705  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 670) took 1419 ms. Trace:
I20260709 13:01:47.162093  2300 rpcz_store.cc:276] 0709 13:01:45.742070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.128074 (+1386004us) service_pool.cc:224] Handling call
0709 13:01:47.161644 (+ 33570us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.055243  3509 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:47.057417  3394 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:47.163094  3509 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.163391  3394 raft_consensus.cc:697] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:47.163561  3509 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.163841  1682 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:01:47.055044  3497 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.164651  3497 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.051191  3449 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:01:46.979871  3504 consensus_meta.cc:287] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.007s	user 0.021s	sys 0.013s
W20260709 13:01:47.052527  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.065142  3420 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:47.058137  3544 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:47.074599  3404 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:47.146282  3547 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.146427  3470 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:47.149662  3516 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:47.061808  2302 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 2: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:47.158906  3345 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:47.086326  3490 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:47.160424  2305 rpcz_store.cc:276] 0709 13:01:45.942624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.044628 (+1102004us) service_pool.cc:224] Handling call
0709 13:01:47.160047 (+115419us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:47.089483  2303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 673) took 1346 ms (client timeout 1776 ms). Trace:
I20260709 13:01:47.058343  3538 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:47.079028  3472 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:47.146828  3360 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:47.205444  2301 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:01:47.115370  3348 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.142575  3480 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:47.274699  2303 rpcz_store.cc:269] 0709 13:01:45.742970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.971167 (+228197us) service_pool.cc:224] Handling call
0709 13:01:47.089243 (+1118076us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b54005dc0d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:47.089400 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2176522496}
I20260709 13:01:47.275862  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:47.216792  3527 consensus_meta.cc:287] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.503s	user 0.006s	sys 0.004s
I20260709 13:01:47.183972  3544 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.174240  3414 raft_consensus.cc:697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:47.145675  3350 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.167815  3512 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:47.168241  3535 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:47.157922  3352 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:47.197969  3449 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.168691  3483 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.173198  3542 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:47.086658  3508 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:47.275491  3490 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.279265  3550 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:01:47.306283  3420 raft_consensus.cc:697] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:47.325021  3445 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:47.278306  3538 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:47.216325  3504 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.309890  3472 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.327651  3472 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.328948  3504 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:47.306628  3535 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:47.312057  3526 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:47.313723  3508 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:47.334275  3516 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:47.332267  3535 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.331897  3544 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.336408  3544 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:47.331333  3538 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:46.981987  1532 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:01:47.369494  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 387392us
W20260709 13:01:47.051455  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 100548us
W20260709 13:01:47.008540  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.372895  1533 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:01:47.373450  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:47.376078  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:47.376785  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:47.377337  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.380627  1566 raft_consensus.cc:2410] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:01:47.381027  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.081542  1568 rpcz_store.cc:276] 0709 13:01:45.732164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.807653 (+1075489us) service_pool.cc:224] Handling call
0709 13:01:46.973989 (+166336us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.032667  1569 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:01:47.382354  1567 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:01:47.046657  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 107049us
I20260709 13:01:47.387530  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:47.392211  1533 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:01:47.393129  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:47.398173  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:47.399008  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:47.052594  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 103001us
W20260709 13:01:47.146344  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 208140us
I20260709 13:01:47.420212  1609 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:01:47.158840  1636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1533) took 1991 ms (client timeout 1837 ms). Trace:
W20260709 13:01:47.421670  1636 rpcz_store.cc:269] 0709 13:01:45.167717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.516331 (+348614us) service_pool.cc:224] Handling call
0709 13:01:46.407800 (+891469us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:47.158627 (+750827us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b54002d7b54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:47.158758 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1391993600}
W20260709 13:01:47.423435  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 278007us
I20260709 13:01:47.424743  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:47.426373  1636 raft_consensus.cc:2393] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:01:47.428210  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.428288  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:47.429702  1709 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:01:47.430353  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.430298  1636 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:47.431538  1713 raft_consensus.cc:2393] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:47.431696  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.431849  1711 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:47.432086  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:47.432878  1710 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:47.433075  1636 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.411459  1637 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:47.140897  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 347) took 1627 ms. Trace:
I20260709 13:01:47.429742  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.416106  1641 raft_consensus.cc:2393] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:47.179843  1642 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
W20260709 13:01:47.441211  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1562) took 1714 ms (client timeout 2206 ms). Trace:
W20260709 13:01:47.441817  1642 rpcz_store.cc:269] 0709 13:01:45.726756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.979190 (+1252434us) service_pool.cc:224] Handling call
0709 13:01:47.440914 (+461724us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.179872  1643 rpcz_store.cc:276] 0709 13:01:41.687940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.148254 (+5460314us) service_pool.cc:224] Handling call
0709 13:01:47.148644 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.148856 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:47.148894 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:47.148948 (+    54us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:47.148997 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.149071 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.149216 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.444449  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:47.445730  1643 raft_consensus.cc:2368] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4.
I20260709 13:01:47.112543  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 346) took 1599 ms. Trace:
I20260709 13:01:47.434523  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:47.420536  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.439977  1712 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:01:47.446899  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:47.446944  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:47.457368  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:47.459841  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:47.462684  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:47.460762  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.464233  1712 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:47.460218  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:47.466307  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:01:47.463063  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.086876  2297 rpcz_store.cc:276] 0709 13:01:45.806050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.219157 (+413107us) service_pool.cc:224] Handling call
0709 13:01:46.922478 (+703321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.492220  3301 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:47.488289  3414 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:01:47.283221  2305 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:47.174335  2299 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:01:47.322644  3420 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.171676  3332 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.277590  3360 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:47.501219  3558 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:01:47.164316  3394 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.217353  3404 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:47.184477  3444 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.312371  3402 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.547099  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:47.548110  1567 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:47.471846  2302 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 4. Ops: []
I20260709 13:01:47.469861  2269 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:47.487502  2303 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:47.163089  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:47.448069  1718 raft_consensus.cc:2376] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5. Re-sending same reply.
I20260709 13:01:47.488747  3471 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:47.449719  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:47.472606  1683 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:47.450631  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:47.469702  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:47.492775  3301 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:47.316725  3449 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:47.555387  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 86000.5us
I20260709 13:01:47.556368  2269 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:47.558077  3449 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:47.558784  3544 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:47.559692  3544 raft_consensus.cc:697] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:47.559893  2300 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:47.560783  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 677) took 1016 ms. Trace:
I20260709 13:01:47.561220  2300 rpcz_store.cc:276] 0709 13:01:46.544769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.162781 (+618012us) service_pool.cc:224] Handling call
0709 13:01:47.560661 (+397880us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.561285  3544 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.563285  1710 raft_consensus.cc:2393] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:47.563699  3516 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:47.564429  3516 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:47.564594  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.565348  1714 raft_consensus.cc:1240] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:47.566175  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.566769  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:47.567763  1568 raft_consensus.cc:2368] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3.
I20260709 13:01:47.551541  3471 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:47.568305  3471 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:47.551456  3301 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:47.572242  2301 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:47.572513  3332 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:47.572995  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 635) took 1763 ms. Trace:
I20260709 13:01:47.573433  2301 rpcz_store.cc:276] 0709 13:01:45.809298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.571343 (+1762045us) service_pool.cc:224] Handling call
0709 13:01:47.571759 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.572699 (+   940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.572914 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:47.563308  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 91105.6us
I20260709 13:01:47.576794  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:01:47.576581  3431 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:47.578048  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 626) took 1812 ms. Trace:
I20260709 13:01:47.578356  1683 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:47.578476  2302 rpcz_store.cc:276] 0709 13:01:45.765795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.470383 (+1704588us) service_pool.cc:224] Handling call
0709 13:01:47.471148 (+   765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.553571 (+ 82423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.577974 (+ 24403us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.579481  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:47.580058  2302 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:47.451915  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.580721  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 637) took 1763 ms. Trace:
I20260709 13:01:47.581041  2302 rpcz_store.cc:276] 0709 13:01:45.897563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.579345 (+1681782us) service_pool.cc:224] Handling call
0709 13:01:47.579763 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.580499 (+   736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.580675 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.581487  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:47.582216  1715 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.582388  2302 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 LEARNER]: Advancing to term 2
W20260709 13:01:47.451457  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.414783  1638 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:01:47.452497  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.494653  2297 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:47.439539  1639 rpcz_store.cc:276] 0709 13:01:45.552157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.118216 (+1566059us) service_pool.cc:224] Handling call
0709 13:01:47.140808 (+ 22592us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:47.504940  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 106159us
I20260709 13:01:47.447256  1644 rpcz_store.cc:276] 0709 13:01:45.512963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.917914 (+1404951us) service_pool.cc:224] Handling call
0709 13:01:47.112114 (+194200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.448040  1636 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
W20260709 13:01:47.510989  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 91264.4us
I20260709 13:01:47.511381  3446 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:47.513909  3536 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:47.508809  1635 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:01:47.473210  3451 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:47.502662  3480 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:47.518301  3528 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:47.519799  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:47.520900  3550 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:47.596743  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:47.599679  3550 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.599825  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:47.600157  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:47.600668  1642 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:47.527309  3350 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:47.480738  3345 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:47.527717  1641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1563) took 1516 ms (client timeout 1641 ms). Trace:
I20260709 13:01:47.527979  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 629) took 1739 ms. Trace:
I20260709 13:01:47.532125  1569 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
W20260709 13:01:47.518517  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 58738.5us
I20260709 13:01:47.537240  3404 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.540096  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.541229  1562 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:01:47.543299  3372 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:47.535249  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 138404us
I20260709 13:01:47.551895  1643 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:47.548367  3402 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:47.565909  3516 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.438341  1681 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:47.584215  1712 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:47.579514  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.589020  1710 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:47.548242  3444 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:47.592520  3446 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.598748  3352 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:47.597147  3528 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:47.599009  1560 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.599457  2269 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:47.600154  3550 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:47.601640  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 349) took 2005 ms. Trace:
I20260709 13:01:47.601686  1709 raft_consensus.cc:1240] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 1. Current term is 2. Ops: []
W20260709 13:01:47.525792  1637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1558) took 2135 ms (client timeout 1860 ms). Trace:
I20260709 13:01:47.523909  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.570434  3497 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.576606  3547 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:47.694094  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 57781.2us
I20260709 13:01:47.677459  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.675092  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:47.635488  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:47.663965  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:47.635978  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:47.751786  1638 rpcz_store.cc:276] 0709 13:01:45.596100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.137696 (+1541596us) service_pool.cc:224] Handling call
0709 13:01:47.593992 (+456296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.752238  2305 rpcz_store.cc:276] 0709 13:01:45.768959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.494155 (+1725196us) service_pool.cc:224] Handling call
0709 13:01:47.494640 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.497723 (+  3083us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:47.497763 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:47.499640 (+  1877us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:47.505165 (+  5525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.505230 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.505830 (+   600us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.600210  1644 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
W20260709 13:01:47.687793  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 102555us
I20260709 13:01:47.751705  1569 pending_rounds.cc:85] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 1
W20260709 13:01:47.675457  1563 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.865s	user 0.004s	sys 0.016s
I20260709 13:01:47.681073  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:01:47.750937  1641 rpcz_store.cc:269] 0709 13:01:46.011517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.881869 (+870352us) service_pool.cc:224] Handling call
0709 13:01:47.527377 (+645508us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b54001e9454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:47.527614 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1065572608}
I20260709 13:01:47.671288  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.763062  1715 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:47.755800  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:01:47.769750  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 170387us
I20260709 13:01:47.769953  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:47.771468  1715 raft_consensus.cc:2393] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:47.593020  3536 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.775137  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:01:47.776263  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 173684us
W20260709 13:01:47.673465  3392 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.628s	user 0.012s	sys 0.000s
I20260709 13:01:47.657371  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 628) took 1865 ms. Trace:
I20260709 13:01:47.594218  3451 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.789294  1713 raft_consensus.cc:2393] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 2: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:47.693073  3431 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:47.676607  1532 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:47.760212  3528 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.597375  2300 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Advancing to term 3
W20260709 13:01:47.785588  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.787168  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:47.762009  1569 pending_rounds.cc:107] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:01:47.800611  1569 raft_consensus.cc:2889] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:47.787782  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:47.660984  3449 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:47.779477  3536 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:47.796300  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 627) took 2030 ms. Trace:
I20260709 13:01:47.656250  3541 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:47.656982  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 355) took 1375 ms. Trace:
I20260709 13:01:47.784301  1718 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:47.784526  1563 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:01:47.798794  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:47.800993  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.757102  3446 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:47.801754  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.689556  3472 consensus_queue.cc:1059] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.100s
I20260709 13:01:47.801551  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 642) took 1855 ms. Trace:
I20260709 13:01:47.686635  3348 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:47.782358  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 344416us
I20260709 13:01:47.799505  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:47.662492  3470 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.807875  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.809756  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:47.805042  3449 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.802052  3352 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:47.936625  1683 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:47.811167  1638 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:47.812078  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1725) took 1834 ms. Trace:
I20260709 13:01:47.804832  3392 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:47.813308  3508 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:47.815080  2304 consensus_meta.cc:287] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.640s	user 0.010s	sys 0.006s
I20260709 13:01:47.815536  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.816561  3541 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.818529  3517 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:01:47.819275  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.792447  3451 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:47.971102  3431 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:47.971652  3431 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.972100  3431 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:47.974505  3538 consensus_meta.cc:287] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.635s	user 0.008s	sys 0.002s
I20260709 13:01:47.822680  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:47.975483  1716 raft_consensus.cc:2393] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:47.784369  2297 rpcz_store.cc:276] 0709 13:01:45.766332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.493582 (+1727250us) service_pool.cc:224] Handling call
0709 13:01:47.494199 (+   617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.588503 (+ 94304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.624405 (+ 35902us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.823377  3550 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.824106  3301 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:47.828065  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:01:47.752251  1637 rpcz_store.cc:269] 0709 13:01:45.390152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.063977 (+673825us) service_pool.cc:224] Handling call
0709 13:01:47.525406 (+1461429us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400461ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:47.525640 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3096018816}
I20260709 13:01:47.831506  3535 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:47.832585  3554 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:47.687052  3484 consensus_queue.cc:1059] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.208s
I20260709 13:01:47.674453  3490 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:47.883164  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 206765us
I20260709 13:01:47.885850  1714 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:47.806146  2303 rpcz_store.cc:276] 0709 13:01:45.766078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.468507 (+1702429us) service_pool.cc:224] Handling call
0709 13:01:47.469481 (+   974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.553074 (+ 83593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.796019 (+242945us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b54005d2ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:47.796228 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39849728}
I20260709 13:01:47.887094  2305 rpcz_store.cc:276] 0709 13:01:45.940033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.764698 (+1824665us) service_pool.cc:224] Handling call
0709 13:01:47.788555 (+ 23857us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.788862 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:47.788890 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:47.788950 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:47.795164 (+  6214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.795245 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.795399 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:47.805398  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:48.003118  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:48.003563  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.803850  1642 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:47.888409  2269 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:47.934451  1715 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:47.935161  1709 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.889362  1567 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:47.886482  3345 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:47.810896  1639 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:01:47.948251  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 163187us
I20260709 13:01:47.949162  3372 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:47.599370  3473 consensus_queue.cc:1059] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.288s
I20260709 13:01:47.950485  3483 raft_consensus.cc:3055] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:01:47.951166  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 143841us
I20260709 13:01:47.956002  3497 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:47.956704  2304 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:47.956454  3352 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.939960  1562 rpcz_store.cc:276] 0709 13:01:45.977733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.982499 (+1004766us) service_pool.cc:224] Handling call
0709 13:01:47.540829 (+558330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:47.773601 (+232772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.811806 (+ 38205us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54000a78d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:47.811999 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1236086144}
I20260709 13:01:47.960728  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:48.023888  3535 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:47.940377  3508 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.025065  3508 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.982556  3538 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:47.962373   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:48.027902  3538 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:48.029704  1682 leader_election.cc:400] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4.
I20260709 13:01:47.871387  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:47.835181  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.802654  3444 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:48.032538  3444 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.032943  3444 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:47.984027  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 346) took 1307 ms. Trace:
I20260709 13:01:48.033735  3301 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:48.033890  1563 rpcz_store.cc:276] 0709 13:01:46.672805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.740853 (+ 68048us) service_pool.cc:224] Handling call
0709 13:01:47.947534 (+1206681us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.937450  3449 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:47.988095  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:47.834843  1635 rpcz_store.cc:276] 0709 13:01:46.291744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.061269 (+769525us) service_pool.cc:224] Handling call
0709 13:01:47.623043 (+561774us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.035511  2303 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:47.792958  3509 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.036257  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 646) took 2093 ms. Trace:
I20260709 13:01:48.036604  2303 rpcz_store.cc:276] 0709 13:01:45.943631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.002440 (+2058809us) service_pool.cc:224] Handling call
0709 13:01:48.035280 (+ 32840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.035969 (+   689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.036197 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.037827  3509 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:47.977816  3541 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:48.000909  1644 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:48.039547  1609 leader_election.cc:400] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3.
I20260709 13:01:47.966027  3471 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.041074  1609 leader_election.cc:400] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4.
I20260709 13:01:48.041369  1609 leader_election.cc:403] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:47.969785  3517 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.042752  3301 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:48.043077  3480 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4.
I20260709 13:01:47.980890  1717 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:48.043934  3517 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:48.044333  3301 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:47.982837  2297 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:48.044524  3345 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:48.044809  3301 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:48.009632  2298 raft_consensus.cc:2393] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:48.045749  3345 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.045867  3301 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.046638  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 682) took 1067 ms. Trace:
I20260709 13:01:48.047129  3542 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:01:48.047091  2298 rpcz_store.cc:276] 0709 13:01:46.979534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.153141 (+173607us) service_pool.cc:224] Handling call
0709 13:01:48.046406 (+893265us) spinlock_profiling.cc:243] Waited 831 ms on lock 0x7b5400708354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:48.046578 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1911556480}
I20260709 13:01:48.047955  3542 consensus_queue.cc:1243] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:48.048874  2298 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:48.049113  3542 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:48.049548  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 678) took 1504 ms. Trace:
I20260709 13:01:48.049659  3542 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:48.050199  1640 raft_consensus.cc:2393] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:48.051250  1713 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:01:48.051352  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1564) took 2039 ms (client timeout 2085 ms). Trace:
I20260709 13:01:48.051828  3471 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:48.054643  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:01:48.016316  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 358601us
I20260709 13:01:48.020479  3473 raft_consensus.cc:3055] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:48.056967  3473 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:48.053011  2299 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:48.058168  3473 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:47.912482  1560 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:48.063553  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 684) took 1077 ms. Trace:
I20260709 13:01:48.038194  2303 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:48.052718  3471 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:48.027904  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 499) took 2014 ms. Trace:
I20260709 13:01:48.062172  1643 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:48.063959  2304 rpcz_store.cc:276] 0709 13:01:46.985891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.160327 (+174436us) service_pool.cc:224] Handling call
0709 13:01:48.054630 (+894303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.030578  1636 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:48.083374  3360 consensus_meta.cc:287] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.003s
W20260709 13:01:48.084546  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 68967us
I20260709 13:01:48.058984  3490 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.078989  3451 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.052456  1711 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:48.101558  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 504) took 1822 ms. Trace:
I20260709 13:01:48.104198  2303 rpcz_store.cc:276] 0709 13:01:46.278794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.037387 (+1758593us) service_pool.cc:224] Handling call
0709 13:01:48.037765 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.065272 (+ 27507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.101478 (+ 36206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.103971  1644 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:48.051748  1637 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:48.051836  1640 rpcz_store.cc:269] 0709 13:01:46.011805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.049612 (+1037807us) service_pool.cc:224] Handling call
0709 13:01:48.051166 (+1001554us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b5400124b54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:48.051305 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993790336}
I20260709 13:01:48.086242  3544 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:01:48.012271  3483 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:48.092111  2305 rpcz_store.cc:276] 0709 13:01:46.013816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.008430 (+1994614us) service_pool.cc:224] Handling call
0709 13:01:48.027253 (+ 18823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.027547 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.027571 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:48.027607 (+    36us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:48.027639 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.027710 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:48.027828 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.097348  1638 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:48.099607  3301 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:48.058992  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 632) took 2259 ms. Trace:
W20260709 13:01:48.085014  2302 consensus_meta.cc:287] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.005s
I20260709 13:01:48.049942  2298 rpcz_store.cc:276] 0709 13:01:46.545156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.048296 (+1503140us) service_pool.cc:224] Handling call
0709 13:01:48.048683 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.049287 (+   604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.049496 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.132117  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:48.092941  3392 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:48.100533  3517 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.063683  3414 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.111935  1642 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
W20260709 13:01:48.131520  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 336996us
I20260709 13:01:48.140978  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 339) took 4950 ms. Trace:
I20260709 13:01:48.144798  1638 rpcz_store.cc:276] 0709 13:01:43.161960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.083179 (+4921219us) service_pool.cc:224] Handling call
0709 13:01:48.097181 (+ 14002us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.111761 (+ 14580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.111944 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280}
I20260709 13:01:48.141359  2302 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:48.146328  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 639) took 2209 ms. Trace:
I20260709 13:01:48.146653  2302 rpcz_store.cc:276] 0709 13:01:45.937196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.581766 (+1644570us) service_pool.cc:224] Handling call
0709 13:01:47.582219 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.146059 (+563840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.146274 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.092455  3471 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:48.149605  1640 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:48.150439  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 341) took 4957 ms. Trace:
I20260709 13:01:48.150770  1640 rpcz_store.cc:276] 0709 13:01:43.193031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.108327 (+4915296us) service_pool.cc:224] Handling call
0709 13:01:48.143350 (+ 35023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.150148 (+  6798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.150380 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.140997  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 332) took 5513 ms. Trace:
I20260709 13:01:48.151717  1644 rpcz_store.cc:276] 0709 13:01:42.598412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.078857 (+5480445us) service_pool.cc:224] Handling call
0709 13:01:48.103796 (+ 24939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.111447 (+  7651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.111647 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16128}
W20260709 13:01:48.132361  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 180543us
I20260709 13:01:48.153954  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:01:48.142316  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.156026  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:48.156803  1710 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:01:48.134505  1607 leader_election.cc:336] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.562s (SENT)
I20260709 13:01:48.158676  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:48.160717  1568 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.157457  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 1993) took 6230 ms. Trace:
I20260709 13:01:48.161725  1642 rpcz_store.cc:276] 0709 13:01:41.913898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.057389 (+6143491us) service_pool.cc:224] Handling call
0709 13:01:48.098857 (+ 41468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.144249 (+ 45392us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.144447 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.162533  1567 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.161144  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:48.165205  1562 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:48.166028  2300 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:01:48.166695  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 634) took 2366 ms. Trace:
W20260709 13:01:48.168030  3563 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:01:48.135300  2299 rpcz_store.cc:276] 0709 13:01:45.799821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.513734 (+1713913us) service_pool.cc:224] Handling call
0709 13:01:48.052714 (+538980us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.058059 (+  5345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.058740 (+   681us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b5400708350. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:48.058892 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1238022400}
I20260709 13:01:48.162539  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:48.167467  3483 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.167467  2304 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:48.138621  3484 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:48.161260  3414 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.102982  3431 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:48.178718  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 124168us
I20260709 13:01:48.180377  1643 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:48.182595  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:48.169781  1683 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:48.183843  2296 consensus_meta.cc:287] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.968s	user 0.007s	sys 0.005s
I20260709 13:01:48.185236  3451 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:48.185995  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:48.187436  1638 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:48.187270  1681 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:48.050765  3542 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.187857  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:48.105674  3360 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.142642  3547 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:48.189251  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 681) took 1210 ms. Trace:
I20260709 13:01:48.218389  1567 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:48.194664  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:48.276907  1709 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:48.396576  2302 rpcz_store.cc:276] 0709 13:01:46.979239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.156060 (+1176821us) service_pool.cc:224] Handling call
0709 13:01:48.188556 (+ 32496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.188860 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.188884 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:48.188938 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:48.188996 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.189060 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:48.189182 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.277661  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:48.352334  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:48.389899  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 210444us
I20260709 13:01:48.190861  2303 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:48.195079  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 508) took 1625 ms. Trace:
I20260709 13:01:48.399849  2304 rpcz_store.cc:276] 0709 13:01:46.569888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.139434 (+1569546us) service_pool.cc:224] Handling call
0709 13:01:48.167234 (+ 27800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.179380 (+ 12146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.195016 (+ 15636us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.195690  1713 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:48.396216  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 342) took 4982 ms. Trace:
I20260709 13:01:48.400892  1638 rpcz_store.cc:276] 0709 13:01:43.414044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.145670 (+4731626us) service_pool.cc:224] Handling call
0709 13:01:48.187258 (+ 41588us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.390665 (+203407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.396151 (+  5486us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.392160  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:48.191792  1642 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:48.161618  3473 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:48.288411  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 226511us
I20260709 13:01:48.222774  1712 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:01:48.401737  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 340) took 5219 ms. Trace:
I20260709 13:01:48.288416  1640 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:48.167075  2300 rpcz_store.cc:276] 0709 13:01:45.808877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.562425 (+1753548us) service_pool.cc:224] Handling call
0709 13:01:47.585824 (+ 23399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.166422 (+580598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.166636 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.411489  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 1996) took 4774 ms. Trace:
W20260709 13:01:48.401098  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.403347  1710 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:48.403471  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:01:48.402984  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 303741us
I20260709 13:01:48.411962  1640 rpcz_store.cc:276] 0709 13:01:43.637283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.212824 (+4575541us) service_pool.cc:224] Handling call
0709 13:01:48.288176 (+ 75352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.411232 (+123056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.411430 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.405818  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:48.288180  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:48.408258  1643 rpcz_store.cc:276] 0709 13:01:43.170521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.101196 (+4930675us) service_pool.cc:224] Handling call
0709 13:01:48.162268 (+ 61072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.389105 (+226837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.389373 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.410374  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 1997) took 4772 ms. Trace:
I20260709 13:01:48.413347  1642 rpcz_store.cc:276] 0709 13:01:43.637526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.191133 (+4553607us) service_pool.cc:224] Handling call
0709 13:01:48.191550 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.409728 (+218178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.410248 (+   520us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.277246  3550 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:48.417166  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:48.276523  3348 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:48.418572  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:48.169219  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 1991) took 6460 ms. Trace:
I20260709 13:01:48.428303  3420 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:01:48.428479  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 140400us
I20260709 13:01:48.429400  1566 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:48.398423  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:48.429930  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:01:48.433526  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 246371us
I20260709 13:01:48.190448  3567 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:48.403030  3483 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.175221  3554 raft_consensus.cc:697] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:48.289444  3517 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.160068  3538 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.254s
I20260709 13:01:48.408921  1635 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:48.442518  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 283290us
I20260709 13:01:48.405694  2296 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:48.448793  1636 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:48.437559  3352 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:48.419656  3470 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:48.499702  3451 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:01:48.500521  2296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 683) took 1520 ms (client timeout 1744 ms). Trace:
I20260709 13:01:48.453016  1563 raft_consensus.cc:2393] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 4: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:01:48.500982  2296 rpcz_store.cc:269] 0709 13:01:46.979790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.121649 (+141859us) service_pool.cc:224] Handling call
0709 13:01:48.500421 (+1378772us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.439656  2269 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:48.455952  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:48.457592  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 1764) took 1003 ms. Trace:
W20260709 13:01:48.502605  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 89677.6us
I20260709 13:01:48.458701  1644 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:01:48.459074  1565 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:48.456223  3360 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.462898  3414 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.477185  3554 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:48.478371  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.479396  1607 leader_election.cc:400] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3.
I20260709 13:01:48.503253  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:48.435683  2298 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:48.467949  3431 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.454281  1717 raft_consensus.cc:2368] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3.
I20260709 13:01:48.454416  1560 raft_consensus.cc:2410] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:01:48.457084  1534 leader_election.cc:336] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 2.063s (SENT)
W20260709 13:01:48.513408  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 68939us
W20260709 13:01:48.513857  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:48.514429  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:48.485000  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.281299  3404 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.456974  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:48.454766  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 1998) took 4817 ms. Trace:
I20260709 13:01:48.517315  1638 rpcz_store.cc:276] 0709 13:01:43.637765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.401909 (+4764144us) service_pool.cc:224] Handling call
0709 13:01:48.412388 (+ 10479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.454377 (+ 41989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.454409 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:48.454478 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:48.454513 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.454575 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:48.454695 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.504921  3345 raft_consensus.cc:3055] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:48.524561  3577 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.524704  3345 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:48.526203  1641 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:48.526885  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.468619  1639 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:48.491683  3472 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:48.529277  3472 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:48.529912  3517 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:48.469794  3444 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:48.492770  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 47774.5us
I20260709 13:01:48.468513  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:01:48.518831  3528 consensus_meta.cc:287] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.000s
I20260709 13:01:48.496181  2301 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 1. Current term is 2. Ops: []
I20260709 13:01:48.492137  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:48.499867  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:48.440430  3516 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:48.475205  2305 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.481050  3471 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.504109  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1537) took 3335 ms. Trace:
I20260709 13:01:48.536521  1644 rpcz_store.cc:276] 0709 13:01:45.168687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.458002 (+3289315us) service_pool.cc:224] Handling call
0709 13:01:48.458363 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.503911 (+ 45548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.504067 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:48.471657  3536 consensus_meta.cc:287] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.504s	user 0.005s	sys 0.005s
I20260709 13:01:48.502848  1712 rpcz_store.cc:276] 0709 13:01:47.453761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.465908 (+ 12147us) service_pool.cc:224] Handling call
0709 13:01:48.457416 (+991508us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54003db8d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:48.457537 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268919680}
I20260709 13:01:48.462117  2297 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:01:48.507566  1607 leader_election.cc:403] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:01:48.435527  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:48.195019  3527 consensus_queue.cc:1059] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.394s
I20260709 13:01:48.509219  1643 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:01:48.559340  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 80172.1us
I20260709 13:01:48.559751  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1534) took 3391 ms. Trace:
I20260709 13:01:48.560247  1643 rpcz_store.cc:276] 0709 13:01:45.168003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.437604 (+3269601us) service_pool.cc:224] Handling call
0709 13:01:48.508980 (+ 71376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.559326 (+ 50346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.559668 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.502957  3449 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:48.511725  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.480720  3508 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.469513  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:48.519932  1710 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:48.520037  1715 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:48.520503  1565 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:01:48.520875  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 65055.6us
I20260709 13:01:48.521941  1642 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:48.524060  3404 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.575295  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:48.575953  2303 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.534942  3345 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:48.577915  3345 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:48.488377  1568 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:48.466096  3541 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.580265  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 636) took 2770 ms. Trace:
I20260709 13:01:48.580984  3541 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:48.528304  3431 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:48.511757  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 327) took 5912 ms. Trace:
I20260709 13:01:48.528544  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1535) took 3360 ms. Trace:
I20260709 13:01:48.530889  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:48.473400  3484 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.562536  3470 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:48.589802  1639 rpcz_store.cc:276] 0709 13:01:45.168231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.438125 (+3269894us) service_pool.cc:224] Handling call
0709 13:01:48.468404 (+ 30279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.528249 (+ 59845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.528469 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.590718  3484 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.525350  3577 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.530311  3472 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.591601  3517 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:48.591998  3472 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:48.592818  3577 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.436364  1637 rpcz_store.cc:276] 0709 13:01:41.698835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.051069 (+6352234us) service_pool.cc:224] Handling call
0709 13:01:48.051539 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.111157 (+ 59618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.157950 (+ 46793us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:48.533224  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:48.593961  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.533861  1561 raft_consensus.cc:2368] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4.
I20260709 13:01:48.533908  1682 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:48.535076  1716 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:48.539364  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.536090  3528 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.547247  3444 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.549443  1640 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:48.551455  1644 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:48.552757  3471 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.552919  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1542) took 3383 ms. Trace:
I20260709 13:01:48.545019  3536 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.558075  1641 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
W20260709 13:01:48.511186  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 92060.4us
I20260709 13:01:48.552999  1464 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45150 (request call id 323) took 3167 ms. Trace:
I20260709 13:01:48.535019  3414 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.567137  3516 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:48.567215  3394 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:48.568763  3490 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:48.572042  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:48.604365  3392 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:48.604773  3392 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:48.605139  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:48.605566  3392 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.606181  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:48.606863  2302 raft_consensus.cc:2368] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4.
I20260709 13:01:48.572553  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 643) took 2631 ms. Trace:
I20260709 13:01:48.572855  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:48.607362  1638 raft_consensus.cc:2368] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3.
I20260709 13:01:48.607633  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:48.608213  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:48.608561  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:48.596571  1636 rpcz_store.cc:276] 0709 13:01:45.169785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.551870 (+3382085us) service_pool.cc:224] Handling call
0709 13:01:48.552236 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.552490 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.552524 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:48.552581 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:48.552623 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.552700 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:48.552835 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.609877  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:48.610064  2302 raft_consensus.cc:2368] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3.
I20260709 13:01:48.610415  3508 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:48.573464  3497 consensus_queue.cc:1059] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:48.573921  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:48.574465  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:48.574685  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:48.578645  3404 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.563175  2305 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:48.586786  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1538) took 3417 ms. Trace:
I20260709 13:01:48.608240  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:48.612638  1638 raft_consensus.cc:2368] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3.
I20260709 13:01:48.580637  2301 rpcz_store.cc:276] 0709 13:01:45.809655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.579536 (+1769881us) service_pool.cc:224] Handling call
0709 13:01:48.494015 (+914479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.534953 (+ 40938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.580063 (+ 45110us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b5400600ed0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:48.580203 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2102422400}
I20260709 13:01:48.591055  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:48.596714  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1541) took 3427 ms. Trace:
I20260709 13:01:48.581820  3431 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:48.621640  1639 raft_consensus.cc:2393] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:48.622716  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:01:48.600198  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.623340  3528 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:48.600427  1718 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:48.623956  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:48.572947  3348 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3.
W20260709 13:01:48.624647  3572 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:01:48.624842  1564 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:48.607383  2297 rpcz_store.cc:276] 0709 13:01:45.940920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.976867 (+2035947us) service_pool.cc:224] Handling call
0709 13:01:47.982679 (+  5812us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.545556 (+562877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.572489 (+ 26933us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:48.625643  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51271.5us
W20260709 13:01:48.627080  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 95968.2us
I20260709 13:01:48.627384  3567 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:48.627813  3567 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.628181  3567 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.628382  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:48.597105  1464 rpcz_store.cc:276] 0709 13:01:45.385592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.397792 (+ 12200us) service_pool.cc:224] Handling call
0709 13:01:46.456864 (+1059072us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b540013f954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0b090 00007fd7a1ca9f3a 00007fd7a7a601cc 00007fd7a7b5e37e 00007fd7a601f2a4 00007fd7a601f232 00007fd7a601f161 00007fd7a601f0dc 00007fd7a601e392 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:48.552756 (+2095892us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:47.006335 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:47.142855 (+136520us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:47.142888 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:01:47.142906 (+    18us) tablet.cc:662] Decoding operations
0709 13:01:47.155467 (+ 12561us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:47.155547 (+    80us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:47.155577 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:01:47.155764 (+   187us) tablet.cc:701] Row locks acquired
0709 13:01:47.155780 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:01:47.155906 (+   126us) write_op.cc:270] Start()
0709 13:01:47.156021 (+   115us) write_op.cc:276] Timestamp: P: 1783602107155869 usec, L: 0
0709 13:01:47.156042 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:01:47.156738 (+   696us) log.cc:844] Serialized 541 byte log entry
0709 13:01:47.506465 (+349727us) op_driver.cc:464] REPLICATION: finished
0709 13:01:47.957538 (+451073us) write_op.cc:301] APPLY: starting
0709 13:01:47.957675 (+   137us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:47.957819 (+   144us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:47.957964 (+   145us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:47.958300 (+   336us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:47.958627 (+   327us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:01:47.958689 (+    62us) write_op.cc:312] APPLY: finished
0709 13:01:48.013079 (+ 54390us) log.cc:844] Serialized 23 byte log entry
0709 13:01:48.078162 (+ 65083us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:48.105536 (+ 27374us) write_op.cc:454] Released schema lock
0709 13:01:48.132848 (+ 27312us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:48.191052 (+ 58204us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:48.191092 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"spinlock_wait_cycles":8802560,"child_traces":[["op",{"apply.queue_time_us":383023,"apply.run_cpu_time_us":4616,"apply.run_wall_time_us":233465,"num_ops":1,"prepare.queue_time_us":145789,"prepare.run_cpu_time_us":26230,"prepare.run_wall_time_us":322892,"raft.queue_time_us":2800,"raft.run_cpu_time_us":31638,"raft.run_wall_time_us":443164,"replication_time_us":348327,"thread_start_us":483322,"threads_started":4,"wal-append.queue_time_us":152021}]]}
I20260709 13:01:48.629161  1682 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:01:48.629499  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:48.603296  2269 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:48.600855  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1540) took 3431 ms. Trace:
I20260709 13:01:48.603353  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:48.612334  1642 rpcz_store.cc:276] 0709 13:01:45.168935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.521390 (+3352455us) service_pool.cc:224] Handling call
0709 13:01:48.521749 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.583338 (+ 61589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.586732 (+  3394us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:48.554776  3446 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.607s	user 0.000s	sys 0.009s
I20260709 13:01:48.610491  1636 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:48.615794  1640 rpcz_store.cc:276] 0709 13:01:45.169583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.545002 (+3375419us) service_pool.cc:224] Handling call
0709 13:01:48.549265 (+  4263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.595498 (+ 46233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.596663 (+  1165us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:48.599610  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 107558us
I20260709 13:01:48.619879  3404 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.620486  3444 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:48.607103  3414 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.612625  3571 raft_consensus.cc:2764] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:01:48.629938  3348 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:01:48.659014  3348 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:48.659238  3404 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:48.645669  2298 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:48.659727  3348 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:48.660887  3348 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.661629  3414 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:48.663672  3573 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:01:48.653059  3449 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.622356  3431 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:48.623554  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.554329  1462 catalog_manager.cc:5697] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 2 to 3, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 3 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:48.624094  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 1768) took 1116 ms. Trace:
I20260709 13:01:48.668491  1716 rpcz_store.cc:276] 0709 13:01:47.510596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.976085 (+465489us) service_pool.cc:224] Handling call
0709 13:01:47.976395 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.594619 (+618224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.623845 (+ 29226us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b54003dbdd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:48.624021 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1284497664}
I20260709 13:01:48.668246  1462 catalog_manager.cc:5697] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 2 to 3, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:48.634236  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:48.670113  1638 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:01:48.666796  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.671118  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:01:48.671754  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.672096  1638 raft_consensus.cc:2368] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5.
W20260709 13:01:48.672230  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.673307  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:48.674144  1638 raft_consensus.cc:2393] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:48.643868  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2025) took 1227 ms. Trace:
I20260709 13:01:48.675455  1641 rpcz_store.cc:276] 0709 13:01:47.417009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.772147 (+355138us) service_pool.cc:224] Handling call
0709 13:01:48.643682 (+871535us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b54001c7354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:48.643815 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1357806976}
I20260709 13:01:48.677273  1641 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:48.608364  3536 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.677814  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:48.679277  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:48.679934  1712 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:48.680977  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:48.682871  1532 leader_election.cc:400] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3.
I20260709 13:01:48.630792  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:48.690201  1532 leader_election.cc:403] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:01:48.690496  3301 raft_consensus.cc:3055] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:01:48.690754  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 60077.1us
I20260709 13:01:48.691144  3301 raft_consensus.cc:740] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:48.691722  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:48.692440  3517 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3.
I20260709 13:01:48.646082  2303 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:48.607100  3392 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.602005  3577 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.648587  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:48.650357  1643 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:48.669983  1462 catalog_manager.cc:5697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 2, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:48.707589  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1543) took 3492 ms. Trace:
I20260709 13:01:48.708145  1643 rpcz_store.cc:276] 0709 13:01:45.215261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.564854 (+3349593us) service_pool.cc:224] Handling call
0709 13:01:48.647724 (+ 82870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.707252 (+ 59528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.707512 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.621055  3497 consensus_queue.cc:1243] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:48.710109  3577 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.705008  3508 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:48.710985  3360 raft_consensus.cc:3055] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:48.652549  3444 raft_consensus.cc:697] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:48.711620  3360 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:01:48.711982  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 63481.5us
I20260709 13:01:48.712349  3444 consensus_queue.cc:237] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.712741  3360 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.714222  3360 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:48.714365  3516 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:48.586551  1635 rpcz_store.cc:276] 0709 13:01:42.596953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.078767 (+5481814us) service_pool.cc:224] Handling call
0709 13:01:48.200044 (+121277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.494090 (+294046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.509633 (+ 15543us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.655252  1637 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 2: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:48.611747  3484 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 13:01:48.655557  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:48.650439  3394 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:01:48.715051  3570 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:48.630096  3352 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:48.659080  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 55936.1us
I20260709 13:01:48.621637  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:48.677943  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1549) took 3290 ms. Trace:
I20260709 13:01:48.683914  3414 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.684751  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:48.682546  3567 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.723780  2269 leader_election.cc:400] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3.
I20260709 13:01:48.728602  2269 leader_election.cc:403] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:01:48.721611  1640 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:48.714944  3528 consensus_queue.cc:1059] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:01:48.733894  3517 raft_consensus.cc:3055] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:48.734359  3517 raft_consensus.cc:740] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:48.722865  3542 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:01:48.735385  3517 consensus_queue.cc:260] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:48.720403  3352 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:48.689206  2296 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:48.648154  3446 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.736831  3352 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.691794  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:48.627099  1683 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:01:48.644117  1644 rpcz_store.cc:276] 0709 13:01:45.169380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.544848 (+3375468us) service_pool.cc:224] Handling call
0709 13:01:48.551190 (+  6342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.598407 (+ 47217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.600783 (+  2376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.738972  3446 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.739058  1644 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:48.728150  3414 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.692581  3301 consensus_queue.cc:260] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.684746  1638 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 4. Ops: []
I20260709 13:01:48.740845  3301 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.741066  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1544) took 3409 ms. Trace:
I20260709 13:01:48.741624  1638 rpcz_store.cc:276] 0709 13:01:45.331321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.674737 (+3343416us) service_pool.cc:224] Handling call
0709 13:01:48.684563 (+  9826us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.740772 (+ 56209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.741002 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.692531  1718 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:48.723893  3557 consensus_queue.cc:1059] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:01:48.725354  1641 rpcz_store.cc:276] 0709 13:01:45.387786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.676596 (+3288810us) service_pool.cc:224] Handling call
0709 13:01:48.677090 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.677693 (+   603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.677885 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.611435  2302 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:48.704219  3483 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:48.725384  3571 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:48.705089  1532 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:48.722061  3470 consensus_queue.cc:1059] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:01:48.725836  1709 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
W20260709 13:01:48.744596  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 49974.6us
I20260709 13:01:48.720201  3394 raft_consensus.cc:697] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:48.720408  3570 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:48.745857  3394 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.746233  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:48.748991  1532 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:48.711028  1462 catalog_manager.cc:5697] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 2, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:48.725056  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 113372us
I20260709 13:01:48.749526  3575 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:48.742280  1643 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 4. Ops: []
I20260709 13:01:48.702908  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:48.751353  1642 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:48.719075  1639 raft_consensus.cc:2393] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:01:48.711028  1534 leader_election.cc:336] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.860s (SENT)
W20260709 13:01:48.752796  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 45397.7us
I20260709 13:01:48.750077  3451 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:48.630618  3490 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:01:48.753691  3451 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.754117  3451 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:48.754288  3490 raft_consensus.cc:3055] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:48.721284  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 352) took 2695 ms. Trace:
I20260709 13:01:48.754806  3490 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:48.745059  3550 raft_consensus.cc:3055] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:48.725219  3541 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:01:48.719944  1635 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: []
W20260709 13:01:48.755200  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 63478.3us
I20260709 13:01:48.747663  3483 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:48.743834  2300 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:48.749059  3567 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.718183  3497 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:01:48.736274  3471 raft_consensus.cc:3055] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:48.743737  3557 raft_consensus.cc:3055] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:48.752832  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:48.746112  3570 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.688521  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1550) took 3300 ms. Trace:
I20260709 13:01:48.755206  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:48.811528  1718 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:01:48.774757  3449 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:01:48.774415  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 147381us
I20260709 13:01:48.724442  3577 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:48.775297  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.774906  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1551) took 3386 ms. Trace:
I20260709 13:01:48.825371  1643 rpcz_store.cc:276] 0709 13:01:45.388318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.709770 (+3321452us) service_pool.cc:224] Handling call
0709 13:01:48.742077 (+ 32307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.751012 (+  8935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.774845 (+ 23833us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:48.786527  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 157185us
I20260709 13:01:48.785871  3483 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.775588  3571 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:01:48.826452  3483 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.776540  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1555) took 3387 ms. Trace:
I20260709 13:01:48.827206  1635 rpcz_store.cc:276] 0709 13:01:45.389346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.718285 (+3328939us) service_pool.cc:224] Handling call
0709 13:01:48.719767 (+  1482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.776251 (+ 56484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.776480 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.776520  3541 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:48.828433  3541 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:01:48.829969  1607 connection.cc:256] RPC call timeout handler was delayed by -1.1877314619999879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
I20260709 13:01:48.829644  3541 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.815558  3490 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.832003  3490 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:48.832343  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:48.833160  1565 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:48.833420  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:48.834172  3483 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.834540  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:48.776757  3542 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3.
I20260709 13:01:48.835273  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:48.835855  1561 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.836828  3541 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.793038  3517 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.750088  3575 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.837710  3575 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:48.775374  3550 raft_consensus.cc:740] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:48.742318  3414 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.839358  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:48.840230  3550 consensus_queue.cc:260] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.840827  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:48.841612  3550 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.842093  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:01:48.842509  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:48.819583  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 105695us
I20260709 13:01:48.852447  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:48.853091  1709 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.853456  1681 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:48.855430  1567 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:48.855995  1681 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:48.856647  3575 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.857887  1568 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:01:48.842713  1563 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.859295  1635 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:48.834203  1564 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:48.865036  1715 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:48.866503  3550 consensus_queue.cc:1059] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:48.868280  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:48.868906  1715 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:48.836967  3483 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:48.874320  1683 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:01:48.830447  1607 connection.cc:256] RPC call timeout handler was delayed by -1.139131462000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
W20260709 13:01:48.875581  1607 connection.cc:256] RPC call timeout handler was delayed by -1.086031461999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20400000000000001s for a response
I20260709 13:01:48.872730  3431 consensus_queue.cc:1059] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:01:48.892354  1683 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:01:48.892867  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.810051  3557 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:48.810945  3471 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:48.896714  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:48.899211  1642 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:48.903669  1563 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:48.905865  1567 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:01:48.819950  3449 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:48.911144  3557 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.912015  3517 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.912748  3557 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:48.916896  3508 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:48.917575  3508 raft_consensus.cc:697] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:48.912772  3517 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.919029  1709 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
W20260709 13:01:48.919135  1534 leader_election.cc:336] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 2.346s (SENT)
I20260709 13:01:48.920461  3517 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.921003  1532 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:48.921873  3517 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:48.922427  3517 raft_consensus.cc:697] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:01:48.953234  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.945644  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:48.926100  1561 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:48.928435  1715 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:01:48.955868  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:48.875430  3575 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:48.966895  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.959129  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:48.968845  1643 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:48.971793  3484 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.958181  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:48.972929  1635 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:48.974009  1640 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:48.975782  3541 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:48.976802  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.979566  3554 raft_consensus.cc:3055] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:48.980087  3554 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:48.981281  3472 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:48.981072  3554 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.982229  3472 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:48.982771  3554 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:48.983374  3580 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:01:48.923359  3517 consensus_queue.cc:237] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.965587  3348 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.984997  3484 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.985031  3348 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.987236  3497 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:48.987421  3484 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:48.987979  3497 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:48.988459  3497 raft_consensus.cc:697] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:48.989152  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:48.989351  3497 consensus_queue.cc:237] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.985008  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 685) took 1805 ms. Trace:
I20260709 13:01:48.991253  3472 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:48.991832  2300 rpcz_store.cc:276] 0709 13:01:47.179315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.445136 (+1265821us) service_pool.cc:224] Handling call
0709 13:01:48.495385 (+ 50249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.793591 (+298206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.984719 (+191128us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b5400642154. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:48.984942 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":531740544}
I20260709 13:01:48.897383  3444 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:01:48.999418  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:49.003010  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:48.922847  3578 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.206s	user 0.003s	sys 0.000s
I20260709 13:01:49.004980  1569 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:49.004417  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:01:49.014333  1534 leader_election.cc:336] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 2.573s (SENT)
I20260709 13:01:48.929535  3490 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.889078  3577 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 13:01:48.918502  3508 consensus_queue.cc:237] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:49.012266  3449 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:49.022881  3585 raft_consensus.cc:3055] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:49.023660  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:49.027225  3471 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:48.755162  1637 rpcz_store.cc:276] 0709 13:01:46.026231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.720280 (+2694049us) service_pool.cc:224] Handling call
0709 13:01:48.720639 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.720947 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.720975 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:48.721025 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:48.721057 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.721118 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:48.721232 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:49.030936  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 175040us
I20260709 13:01:49.039723  1643 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:49.012943  3554 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:49.004769  3516 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:01:49.049382  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.051595  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:49.032582  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:49.052915  1534 leader_election.cc:336] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 2.779s (SENT)
W20260709 13:01:49.054723  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.056551  1638 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:49.057809  3471 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.061792  3352 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:49.063221  1715 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:49.063335  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:49.064946  3352 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:49.067150  1639 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:49.070185  3584 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:49.075915  3550 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.076573  3550 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:49.079866  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 187606us
I20260709 13:01:48.811194  1636 rpcz_store.cc:276] 0709 13:01:45.388050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.687012 (+3298962us) service_pool.cc:224] Handling call
0709 13:01:48.687564 (+   552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.687948 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.687992 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:48.688081 (+    89us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:48.688144 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.688233 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:48.688422 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:49.092056  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:49.092730  2302 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:49.038298  3444 raft_consensus.cc:697] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:49.040549  3394 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:49.095317  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 43828.7us
I20260709 13:01:49.038081  3585 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:01:49.102566  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 208948us
W20260709 13:01:49.104347  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.084375  3348 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:49.079061  3550 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:49.128022  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 88342.4us
I20260709 13:01:49.130121  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:49.130191  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:49.131387  2303 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:49.135728  1681 leader_election.cc:400] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4.
I20260709 13:01:49.136090  1681 leader_election.cc:403] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:01:49.136268  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:49.040771  3490 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:49.137503  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 321531us
I20260709 13:01:49.093387  3589 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:49.142902  2302 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:49.133796  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:49.144999  2301 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.124675  3352 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:49.149310  3581 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:49.141628  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:49.146857  3392 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:49.148115  2298 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:49.145223  3550 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:49.163439  3360 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4.
I20260709 13:01:49.164516  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:49.166697  1682 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:49.171552  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 45130.3us
I20260709 13:01:49.163681  2300 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 3: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:49.134260  3451 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.173082  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:49.174800  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:49.175091  1607 leader_election.cc:336] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.806s (SENT)
I20260709 13:01:49.175547  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:49.163504  2296 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:01:49.198757  2268 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:49.199008  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:49.201198  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 69972us
W20260709 13:01:49.201953  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.161051  3352 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:49.202858  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.192683  3444 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:49.206025  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:49.210656  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.204124  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:49.225085  1639 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:49.226189  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:49.173820  3516 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:49.227442  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:49.228155  1639 raft_consensus.cc:2368] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4.
I20260709 13:01:49.229101  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:49.222230  3472 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.174755  3585 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:49.206386  3392 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:49.224390  3577 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:49.217939  3590 log.cc:927] Time spent T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:01:49.214851  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:49.237905  1533 leader_election.cc:336] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.837s (SENT)
I20260709 13:01:49.244463  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:49.241250  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:49.241647  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:49.246472  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:49.215698  3541 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:49.172714  3508 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:01:49.245610  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:49.251116  1709 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:49.246220  1682 leader_election.cc:400] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4.
I20260709 13:01:49.246240  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:49.239449  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:49.254127  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:49.248172  1641 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:01:49.247991  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:49.244570  1638 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:01:49.249540  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:49.246940  1713 raft_consensus.cc:2376] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4. Re-sending same reply.
I20260709 13:01:49.256429  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:49.205148  3595 consensus_queue.cc:1059] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
W20260709 13:01:49.263501  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.265182  1714 raft_consensus.cc:2376] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3. Re-sending same reply.
I20260709 13:01:49.264182  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:49.275931  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:49.276587  1711 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:49.279026  3490 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:49.279208  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:49.238425  3472 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:49.280409  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:49.281174  3472 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:49.281522  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:01:49.286141  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.287312  1642 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:01:49.288267  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:49.288743  1642 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.292004  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:49.292433  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 58076.9us
I20260709 13:01:49.292737  1712 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:01:49.293763  3348 consensus_queue.cc:1059] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:01:49.295011  3348 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:49.295392  3348 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:01:49.295449  3597 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.145s	user 0.002s	sys 0.000s
I20260709 13:01:49.296428  3348 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:49.297768  3348 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:49.298689  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:49.300252  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:49.300736  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:49.302451  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:49.303382  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:49.305261  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.243427  3589 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:49.255220  3517 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:49.314208  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.316956  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:49.323750  2296 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:49.326483  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.273972  3585 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:49.343262  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:01:49.380107  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 128019us
I20260709 13:01:49.245903  3577 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.398869  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:49.402041  3497 raft_consensus.cc:3055] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:01:49.402428  2296 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:49.402614  3497 raft_consensus.cc:740] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:49.403294  3577 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:49.403790  3497 consensus_queue.cc:260] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:49.405601  3497 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:01:49.349681  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.318593  3490 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:49.410393  3490 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:49.411295  3541 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:49.411437  3490 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.412550  3541 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:49.412983  3541 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:49.413988  3541 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.415126  1569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41262
I20260709 13:01:49.415886  1569 raft_consensus.cc:606] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Received request to transfer leadership
I20260709 13:01:49.415934  1638 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:49.416266  1569 raft_consensus.cc:614] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:49.416348  1561 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:49.417608  1711 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:01:49.285594  3392 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:49.285297  3581 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:49.418404  3392 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:49.320878  1716 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:49.349085  3592 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:49.400277  1533 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:01:49.367230  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.316226  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:49.420503  1464 catalog_manager.cc:5697] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 2 to 3, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 3 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:49.370535  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.317178  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:49.386598  1639 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.387542  3571 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:49.372151  2269 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:49.435325  1638 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:01:49.435634  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 63559.5us
I20260709 13:01:49.379509  3557 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:01:49.392905  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:49.395227  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:49.397830  2298 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:01:49.417533  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:49.454723  1568 raft_consensus.cc:2410] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
W20260709 13:01:49.363693  3596 log.cc:927] Time spent T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.134s	user 0.000s	sys 0.004s
I20260709 13:01:49.428821  1713 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:49.429186  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:49.430099  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.430204  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:01:49.432222  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 289932us
I20260709 13:01:49.438481  1642 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:49.328116  1637 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:49.439282  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 123203us
I20260709 13:01:49.462843  2302 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.463776  1682 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:49.440933  1533 leader_election.cc:400] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3.
W20260709 13:01:49.440959  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:49.461901  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.465746  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:49.466662  1713 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:49.462081  1635 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.467926  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:49.452608  3517 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:49.441968  3592 raft_consensus.cc:697] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:49.452097  3550 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:49.488603  2301 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:49.477319  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:49.483165  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.483383  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:49.457876  3451 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:49.458187  2297 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.496127  1609 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:49.460181  3449 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:49.469388  3541 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.401340  3589 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.487010  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:49.547557  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:49.487278  3444 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:49.521559  2300 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:01:49.534768  2302 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
W20260709 13:01:49.569669  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 73626.8us
I20260709 13:01:49.571928  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:49.529466  3431 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:49.574728  1609 leader_election.cc:400] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4.
I20260709 13:01:49.575325  1609 leader_election.cc:403] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:01:49.497145  3602 consensus_queue.cc:1059] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 13:01:49.581837  3602 raft_consensus.cc:3055] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:49.582458  3602 raft_consensus.cc:740] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:49.585074  3571 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.585711  3571 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.586341  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:49.587025  1715 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:01:49.588114  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:49.589361  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:49.590533  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:49.595638  3592 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:49.536087  3550 raft_consensus.cc:697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:49.601742  2301 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:49.602394  1683 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:49.611969  3550 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.613305  3584 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:49.613945  3584 raft_consensus.cc:697] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:49.614346  3571 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:49.614113  3444 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:49.615154  3584 consensus_queue.cc:237] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.615814  3444 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:49.571233  3609 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:49.619259  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:01:49.619729  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 48930.8us
I20260709 13:01:49.622282  1609 leader_election.cc:400] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3.
W20260709 13:01:49.623396  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 75960.2us
I20260709 13:01:49.603294  1715 raft_consensus.cc:2376] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3. Re-sending same reply.
I20260709 13:01:49.605235  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:49.539743  3595 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:01:49.603230  1533 leader_election.cc:400] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5.
I20260709 13:01:49.634940  1533 leader_election.cc:403] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:01:49.627277  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:49.628947  1683 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:49.639349  1640 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:49.641605  3581 raft_consensus.cc:697] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:49.642220  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:49.643388  3517 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5.
I20260709 13:01:49.643859  1533 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:49.644217  3517 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:49.645000  3517 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:49.645427  3517 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.645802  3517 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.659015  1609 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:49.666628  1609 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:49.668586  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:49.670991  2301 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:01:49.634093  1717 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:01:49.642879  3581 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.636049  3394 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4.
I20260709 13:01:49.628314  3431 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:49.637573  3490 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:01:49.641099  3508 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:01:49.683818  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.537388  3392 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:49.662137  3585 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:49.682029  3609 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:01:49.689064  3598 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.479s	user 0.000s	sys 0.004s
I20260709 13:01:49.696928  3585 raft_consensus.cc:697] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:49.699131  3585 consensus_queue.cc:237] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.699476  1682 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:49.701522  3508 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:49.696924  3431 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.701797  3589 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:49.702497  3589 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:01:49.702898  1534 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 3.152s (SENT) [suppressed 53 similar messages]
W20260709 13:01:49.703939  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.703567  3589 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:49.664311  3497 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
W20260709 13:01:49.704581  1534 leader_election.cc:336] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 2.754s (SENT)
I20260709 13:01:49.710857  1711 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:01:49.712960  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.712482  3517 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:49.713538  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.717022  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:49.675274  3538 consensus_queue.cc:1059] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:01:49.726135  1640 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:01:49.733564  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:49.742972  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 43601.5us
W20260709 13:01:49.746208  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:49.746899  1534 leader_election.cc:336] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.641s (SENT)
W20260709 13:01:49.749029  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 53262us
W20260709 13:01:49.750957  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 82483.7us
I20260709 13:01:49.747334  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:49.752089  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:49.735423  3301 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:49.702783  3508 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.746740  3490 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:01:49.770241  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.771297  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:49.769515  1609 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:49.780323  3581 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:49.788345  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:01:49.804921  3611 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:01:49.805842  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:49.746546  3394 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:01:49.813357  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:49.759270  3301 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.746208  3577 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:01:49.750049  3497 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:01:49.739674  3613 consensus_queue.cc:1059] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:01:49.834801  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:49.832113  2303 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:49.839627  1640 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
W20260709 13:01:49.835117  3618 log.cc:927] Time spent T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:01:49.822137  3538 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:49.859086  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 112217us
I20260709 13:01:49.768405  3615 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 13:01:49.818897  3394 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:49.823726  3301 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:49.820402  3609 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:49.856619  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:49.838449  3626 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:49.877663  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:49.853858  3571 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.879076  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:49.821146  3581 consensus_queue.cc:1059] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:01:49.857175  3471 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.857914  3490 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:01:49.875015  1713 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
W20260709 13:01:49.889240  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 143999us
I20260709 13:01:49.880932  3517 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:49.913621  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 100357us
I20260709 13:01:49.916057  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:49.917012  2303 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:49.917454  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:49.897766  1717 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:49.899062  3627 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.919813  3627 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.922057  3627 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:49.907210  3394 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.906796  2299 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:49.905596  3609 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.924911  3629 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:49.915357  3577 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:49.912571  3451 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:01:49.925698  3626 raft_consensus.cc:3055] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:01:49.958068  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 45690.4us
I20260709 13:01:49.907773  3471 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.897564  3571 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:49.928303  3630 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.929931  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:49.961002  3577 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:49.935014  3490 raft_consensus.cc:3055] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:49.968209  3352 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:49.968946  3631 consensus_queue.cc:1059] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:49.970441  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:01:49.960659  2296 consensus_meta.cc:287] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.001s
I20260709 13:01:49.963438  3627 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:49.975834  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 58733.7us
I20260709 13:01:49.977504  1713 raft_consensus.cc:2368] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3.
I20260709 13:01:49.936102  3639 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:49.939364  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:49.957037  3394 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:49.991286  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 133270us
I20260709 13:01:49.940966  1532 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:49.991626  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:49.988366  3490 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:49.959264  1637 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:49.999521  2304 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:49.964114  3626 raft_consensus.cc:740] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:50.000439  2303 raft_consensus.cc:2393] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:01:49.996798  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 82103.3us
I20260709 13:01:50.001644  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:01:49.934482  3610 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:01:49.925182  3613 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:01:50.030174  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:50.003289  3639 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:01:50.034468  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 102060us
I20260709 13:01:50.000706  2296 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:01:50.006502  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:49.986829  3630 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.036422  3637 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:50.034090  3490 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.035701  3629 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:50.047111  3631 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:50.074946  3629 raft_consensus.cc:697] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:50.075156  3631 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:50.034267  3577 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:50.075964  3629 consensus_queue.cc:237] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:50.047963  1644 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:01:50.049997  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.085106  3613 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:50.085510  3557 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.076575  3631 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.086105  3626 consensus_queue.cc:260] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.051486  3472 raft_consensus.cc:3055] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:50.087749  3472 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:50.087985  3626 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:50.050537  2304 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:01:50.053854  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.090168  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.090616  3471 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:50.090763  2302 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:50.028064  3517 consensus_queue.cc:1059] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
W20260709 13:01:50.055675  1635 consensus_meta.cc:287] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.587s	user 0.010s	sys 0.002s
I20260709 13:01:50.055568  3571 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:50.043221  1714 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:01:50.040416  3394 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:50.057629  1533 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:50.007906  2298 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:50.032850  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:50.028548  3612 raft_consensus.cc:2764] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:01:50.065018  3642 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:50.065673  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.066190  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.072234  1711 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 4: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:50.056239  3627 raft_consensus.cc:3055] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:50.029498  3636 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:50.034094  3607 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.086416  3557 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.092424  1609 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:01:50.052211  1639 consensus_meta.cc:287] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.595s	user 0.008s	sys 0.004s
I20260709 13:01:50.095964  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.112515  2302 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
W20260709 13:01:50.084328  2297 consensus_meta.cc:287] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.003s
I20260709 13:01:50.121832  2297 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:50.065055  3639 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.075292  3630 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:50.104183  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 128681us
I20260709 13:01:50.095386  1635 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
W20260709 13:01:50.106062  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 48550.1us
I20260709 13:01:50.108264  2303 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:50.093344  3538 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:50.119086  3635 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:50.113850  1639 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:50.109107  3394 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:50.135756  1638 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:01:50.119534  3607 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.118659  3557 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:50.055382  3602 consensus_queue.cc:260] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:50.105057  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:50.129390  3642 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:50.108922  3517 consensus_queue.cc:1243] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:50.133648  1533 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:50.111445  3636 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:50.111528  3627 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:01:50.130996  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 98206.9us
I20260709 13:01:50.142342  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:50.114895  2299 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:50.130473  3348 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:50.131747  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.151690  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:50.157590  1640 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:50.134011  3635 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.153228  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.157874  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:50.138481  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.153951  3392 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:50.154491  1718 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:50.159902  1711 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:50.161077  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.137214  1717 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:50.166685  2304 raft_consensus.cc:2410] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:50.163774  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:50.168340  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 104962us
I20260709 13:01:50.156889  3639 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:50.172731  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:50.174104  1637 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:50.174413  2297 raft_consensus.cc:2393] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:50.175423  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.176311  2297 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:50.177201  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.177258  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.178167  2296 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:50.178675  2297 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:50.174608  3517 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:50.179080  2298 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 3: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:01:50.179677  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.179101  3627 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.162896  3301 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.164924  3635 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:50.184405  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:50.188594  3392 raft_consensus.cc:2764] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:01:50.170969  3613 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:50.190682  2304 raft_consensus.cc:2410] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:50.195961  2298 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:01:50.196422  2298 pending_rounds.cc:85] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 4
I20260709 13:01:50.196664  2298 pending_rounds.cc:107] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:01:50.196895  2298 raft_consensus.cc:2889] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:50.198200  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:50.201213  1713 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:50.200924  3636 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.202382  3636 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:50.202840  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:50.203737  1712 raft_consensus.cc:2393] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:50.205044  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.205721  2305 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:50.208897  1534 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:50.185014  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:50.218353  2299 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:50.218483  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:50.220736  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:50.222272  2304 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:50.215381  3627 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:50.225775  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.230712  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:50.211794  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:50.226258  1534 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:50.219857  2298 raft_consensus.cc:2393] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:50.235404  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:50.236382  2299 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:01:50.211308  3538 consensus_queue.cc:1175] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" responder_term: 4 status { last_received { term: 3 index: 6 } last_committed_idx: 6 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:50.224813  3352 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.224598  3360 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.248157  3352 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.248306  3360 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:50.230376  3644 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:50.250590  3352 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:50.251247  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.251796  1637 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:50.243867  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:50.253021  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:50.253171  3360 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:50.238133  3584 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.253597  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:50.254470  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:50.236496  3589 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:50.247570  3301 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:50.256556  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.268810  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:01:50.271633  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 95811.4us
W20260709 13:01:50.237432  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.220712  1636 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:01:50.270531  3392 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:50.283269  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 52660.7us
I20260709 13:01:50.283605  1533 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:50.242697  3394 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:50.274057  3630 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:50.286710  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:50.298323  3631 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:50.280156  3471 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:50.302686  3471 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.303225  3471 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:50.308257  1682 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:50.311102  3360 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:50.311028  3471 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.311756  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:50.316689  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1586) took 1136 ms. Trace:
I20260709 13:01:50.317117  1641 rpcz_store.cc:276] 0709 13:01:49.180163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.191628 (+ 11465us) service_pool.cc:224] Handling call
0709 13:01:50.316524 (+1124896us) spinlock_profiling.cc:243] Waited 48.1 ms on lock 0x7b5400125054. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:50.316630 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110601600}
I20260709 13:01:50.301433  1534 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:50.322564  2304 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:50.286840  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:50.335163  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:50.338171  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:50.341631  3360 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:50.341854  3449 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:50.342514  3360 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:50.342907  3471 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:50.302726  3651 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 13:01:50.346259  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 44906.1us
I20260709 13:01:50.285645  3392 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:50.287873  3630 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.363706  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.280347  3571 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:01:50.300688  3575 raft_consensus.cc:2764] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:01:50.362703  1643 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:50.286041  3589 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:50.367120  3630 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:50.367298  3649 log.cc:927] Time spent T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:01:50.291368  3607 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:50.363310  3392 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:50.387187  3472 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.388546  3623 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:50.390700  3589 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:50.359375  3636 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:50.396303  3636 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.396836  3636 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:50.397603  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:50.398379  1637 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:50.403082  2269 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:01:50.368244  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.368748  3653 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.368300  3541 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:01:50.373127  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:50.375053  1641 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:50.376245  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:50.287432  3584 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.359398  3629 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:50.445438  1643 raft_consensus.cc:2410] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:01:50.395871  3654 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.120s	user 0.003s	sys 0.000s
I20260709 13:01:50.343572  3592 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:01:50.359001  3508 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:01:50.482450  1641 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:50.385960  3360 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:50.534673  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 251121us
I20260709 13:01:50.535955  1533 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:50.537099  3631 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:50.537642  3631 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.538014  3631 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:50.542587  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 184719us
W20260709 13:01:50.542817  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 256170us
I20260709 13:01:50.541397  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:50.543476  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:50.546523  2304 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:01:50.535535  3610 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:01:50.551280  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.553617  2301 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:50.554858  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.461935  3653 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:50.555751  2304 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:50.495215  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:50.561161  1636 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:50.562763  3301 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:50.544219  3626 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:50.565938  3626 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.566460  3626 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:50.572724  3472 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:50.488968  3541 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
W20260709 13:01:50.575701  3657 log.cc:927] Time spent T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:01:50.576472  1639 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:50.577155  1462 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45142 (request call id 347) took 3891 ms. Trace:
I20260709 13:01:50.579246  3653 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:50.538720  2296 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:50.559444  1715 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:50.587275  1464 catalog_manager.cc:5697] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 3, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 3 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:50.575408  3635 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.587457  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:50.555760  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:50.605368  1464 catalog_manager.cc:5697] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 2, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:50.607872  1464 catalog_manager.cc:5697] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 2 to 4, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 4 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:50.610351  1464 catalog_manager.cc:5697] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 3 to 4, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 4 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:50.557266  3660 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:50.595443  1683 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:50.544735  3575 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.633627  2304 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:01:50.616617  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 173849us
W20260709 13:01:50.636952  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:50.640194  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 95619.7us
I20260709 13:01:50.641325  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:50.667202  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 380429us
I20260709 13:01:50.668164  1641 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:01:50.674147  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.577538  1462 rpcz_store.cc:276] 0709 13:01:46.685518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.685954 (+   436us) service_pool.cc:224] Handling call
0709 13:01:50.577093 (+3891139us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:48.812082 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:48.978198 (+166116us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:48.978250 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:01:48.978270 (+    20us) tablet.cc:662] Decoding operations
0709 13:01:48.978970 (+   700us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:48.979086 (+   116us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:48.979113 (+    27us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:01:48.979600 (+   487us) tablet.cc:701] Row locks acquired
0709 13:01:48.979650 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:01:48.979812 (+   162us) write_op.cc:270] Start()
0709 13:01:48.979959 (+   147us) write_op.cc:276] Timestamp: P: 1783602108979752 usec, L: 0
0709 13:01:48.979982 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:01:48.980645 (+   663us) log.cc:844] Serialized 1880 byte log entry
0709 13:01:49.161997 (+181352us) op_driver.cc:464] REPLICATION: finished
0709 13:01:49.274466 (+112469us) write_op.cc:301] APPLY: starting
0709 13:01:49.287945 (+ 13479us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:49.350623 (+ 62678us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:49.386667 (+ 36044us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:49.387876 (+  1209us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:49.388061 (+   185us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:01:49.388102 (+    41us) write_op.cc:312] APPLY: finished
0709 13:01:49.388502 (+   400us) log.cc:844] Serialized 41 byte log entry
0709 13:01:49.388704 (+   202us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:49.388799 (+    95us) write_op.cc:454] Released schema lock
0709 13:01:49.388837 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:49.389310 (+   473us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:49.389346 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":106902,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":85982,"apply.run_cpu_time_us":3910,"apply.run_wall_time_us":132508,"num_ops":4,"prepare.queue_time_us":36657,"prepare.run_cpu_time_us":3964,"prepare.run_wall_time_us":168883,"replication_time_us":163200,"thread_start_us":67680,"threads_started":2}]]}
I20260709 13:01:50.615197  3642 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:50.617869  3571 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:50.632661  3301 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.583293  3508 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:01:50.634524  3644 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.617719  3660 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:50.705926  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 64683.2us
W20260709 13:01:50.707038  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 151374us
I20260709 13:01:50.692833  2268 leader_election.cc:400] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3.
I20260709 13:01:50.733606  1534 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:50.669744  3607 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:50.615689  3630 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.677984  3653 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:50.683334  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:50.744267  2268 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:50.682595  3431 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:01:50.718276  3352 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.671118  3575 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:50.750023  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:50.741014  3607 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.753362  3668 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:01:50.759680  3575 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:50.761452  3575 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:01:50.762287  3575 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.745846  3644 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:50.771142  3613 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:50.772442  3630 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:50.719962  3301 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:50.740669  3660 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:50.779860  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:50.780827  1714 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:50.792028  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:50.774746  3644 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:50.762863  3575 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.797756  3626 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:50.800073  3636 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.801026  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:50.752254  1639 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:50.803704  2296 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:50.806247  3575 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:50.812758  1609 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:01:50.815071  1642 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:01:50.817761  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:50.818436  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:50.818898  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.756834  3607 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:50.813925  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:50.865875  1714 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:50.818909  2302 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:50.866142  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:50.774298  1635 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:01:50.788915  3584 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:50.867125  1715 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:01:50.876678  3571 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.877364  3571 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:50.878093  3653 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:50.867466  3669 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:01:50.881621  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.881821  2296 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:01:50.883288  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 133377us
I20260709 13:01:50.884166  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:01:50.753000  3352 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:50.868973  2304 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:50.887749  1715 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:01:50.892462  3602 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:50.852047  3394 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:50.893011  3352 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:50.890334  3392 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.895973  3301 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:50.897710  3392 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:50.899241  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:50.902153  1714 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:01:50.908425  3667 consensus_queue.cc:1059] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:50.893931  3621 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:50.917934  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:50.914399  3394 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:50.920610  3394 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:50.922142  3648 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.706s	user 0.000s	sys 0.002s
I20260709 13:01:50.922755  3348 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.923322  3348 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:50.924000  3626 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:50.925550  3348 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:50.935433  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:50.936281  1714 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:50.937515  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.949748  3615 raft_consensus.cc:3055] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:50.953331  3615 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:50.954316  3615 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:50.930361  3621 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.957610  3615 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:01:50.958158  3636 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:50.958595  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 522) took 1593 ms. Trace:
I20260709 13:01:50.947613  3508 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.950336  3392 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:50.960075  3670 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:50.956228  3352 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:01:50.916296  3635 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:51.261338  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.010674  1714 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
W20260709 13:01:51.043494  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.995330  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2047) took 1393 ms. Trace:
I20260709 13:01:51.276935  1642 rpcz_store.cc:276] 0709 13:01:49.602212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.698308 (+ 96096us) service_pool.cc:224] Handling call
0709 13:01:50.130579 (+432271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.815713 (+685134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.994177 (+178464us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b540032cdd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:50.994994 (+   817us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b540032cdd0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:50.995203 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2548391552}
I20260709 13:01:51.090318  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1517) took 1680 ms. Trace:
I20260709 13:01:51.282119  1716 rpcz_store.cc:276] 0709 13:01:49.409388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.479089 (+ 69701us) service_pool.cc:224] Handling call
0709 13:01:50.245740 (+766651us) spinlock_profiling.cc:243] Waited 8.15 ms on lock 0x7b58001e4598. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:01:50.247201 (+  1461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.400195 (+152994us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.400232 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:50.400278 (+    46us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:50.400313 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.089919 (+689606us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54001478d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:51.090001 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.090197 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370141952}
I20260709 13:01:51.305872  3644 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.059199  3665 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:50.978585  3621 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:51.306524  3644 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:51.305902  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.309587  1638 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:01:51.216766  3449 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.026s
W20260709 13:01:51.321000  3449 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.001s	sys 0.028s
W20260709 13:01:51.095094  3675 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 13:01:51.132053  1637 consensus_meta.cc:287] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.733s	user 0.013s	sys 0.000s
I20260709 13:01:51.322141  1637 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:01:50.977679  3348 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.323486  3348 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:51.326663  3348 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:51.328266  3627 raft_consensus.cc:2764] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:01:51.316789  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:51.329264  3637 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:51.328819  3644 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:51.330488  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:51.331532  1638 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:50.989356  3577 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:51.333601  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:51.335140  3301 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.012917  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:51.336119  2299 raft_consensus.cc:2376] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4. Re-sending same reply.
I20260709 13:01:51.335893  3301 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:51.336689  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 682) took 1140 ms. Trace:
I20260709 13:01:51.337055  2299 rpcz_store.cc:276] 0709 13:01:50.196399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.238250 (+ 41851us) service_pool.cc:224] Handling call
0709 13:01:51.012739 (+774489us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:51.336628 (+323889us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40820480}
I20260709 13:01:51.338569  3541 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:51.339231  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:51.339725  1564 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 4. Ops: []
W20260709 13:01:51.096220  3629 consensus_meta.cc:287] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.634s	user 0.007s	sys 0.004s
I20260709 13:01:51.341673  1569 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:51.339361  3653 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:51.329484  1641 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:51.317085  3392 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:51.340126  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:01:51.064957  3631 consensus_meta.cc:287] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.000s
I20260709 13:01:51.339227  3640 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.363976  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:51.377868  3653 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:51.378566  1563 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:51.380391  3653 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:51.339242  3541 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:51.363826  3666 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.338455  3301 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:51.368230  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:51.427944  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 81641.7us
I20260709 13:01:51.429891  1682 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:51.269124  3490 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:51.340576  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:51.431167  1561 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 2.
W20260709 13:01:51.364626  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 55028.3us
I20260709 13:01:51.369705  1567 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:01:51.375767  1642 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:01:51.443475  1532 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:51.444031  1642 pending_rounds.cc:85] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 6
I20260709 13:01:51.444482  1642 pending_rounds.cc:107] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:01:51.444852  1642 raft_consensus.cc:2889] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:51.369874  3449 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:01:51.378368  1562 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:51.363085  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:51.449539  1560 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 4: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:51.365350  3629 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:51.383663  1568 raft_consensus.cc:2368] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 3. Current term is 4.
I20260709 13:01:51.384536  1641 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:01:51.382822  3571 consensus_meta.cc:287] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.505s	user 0.001s	sys 0.007s
W20260709 13:01:51.371622  3660 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.000s
I20260709 13:01:51.316810  3662 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.479333  3541 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:51.481149  1560 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:51.384770  3640 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:51.382948  3392 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:51.410267  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:51.501312  1568 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:01:51.487036  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 53667.5us
I20260709 13:01:51.307116  3665 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:51.491369  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:51.491582  1532 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:51.383572  1635 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:51.319949  3508 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:51.489737  3301 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:51.500612  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:51.525355  1567 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:51.386425  3607 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:51.494580  3592 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:51.506203  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:51.507309  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:51.271023  2300 rpcz_store.cc:276] 0709 13:01:49.364743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.364877 (+   134us) service_pool.cc:224] Handling call
0709 13:01:49.463559 (+ 98682us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:49.569225 (+105666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.958252 (+1389027us) spinlock_profiling.cc:243] Waited 94.8 ms on lock 0x7b54005dcd50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:50.958482 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218057088}
I20260709 13:01:51.488605  3567 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:01:51.509480  1569 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:51.509562  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:51.491075  3449 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:01:51.514679  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:51.488965  3348 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.487677  3589 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:01:51.383069  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.519127  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:51.533298  3301 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.534214  1718 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:51.548856  3577 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:51.390964  3631 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:51.485898  3602 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:01:51.588362  3348 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:51.516127  3670 consensus_queue.cc:1059] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:01:51.610078  3670 raft_consensus.cc:3055] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:51.610539  3670 raft_consensus.cc:740] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:51.582553  1562 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:51.567474  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:51.611572  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:51.612316  1712 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:51.496866  3660 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:51.569936  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.534008  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:01:51.585597  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 72254.2us
I20260709 13:01:51.584847  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:51.495590  3571 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:51.537433  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:51.574960  3615 consensus_meta.cc:287] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.000s
I20260709 13:01:51.591739  3684 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.593189  3681 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.597577  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:51.475250  3666 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:51.548632  3592 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:51.496640  3662 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:51.606547  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:51.607368  1717 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:51.608037  1560 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:01:51.608623  3631 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:51.608698  3508 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:51.661657  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:51.617448  1716 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
W20260709 13:01:51.618718  1534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.025s	sys 0.017s
W20260709 13:01:51.671708  1534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.936s	user 0.025s	sys 0.017s
W20260709 13:01:51.672246  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:51.619813  2270 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.006s	sys 0.014s
W20260709 13:01:51.673326  2270 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.006s	sys 0.014s
W20260709 13:01:51.620414  2301 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.873s	user 0.006s	sys 0.014s
W20260709 13:01:51.675339  2301 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.006s	sys 0.014s
I20260709 13:01:51.630607  1564 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:01:51.629606  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.629920  1567 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:51.596658  3301 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:51.695561  1561 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:01:51.659708  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.006s	sys 0.019s
I20260709 13:01:51.605049  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1524) took 1560 ms. Trace:
I20260709 13:01:51.632019  3635 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.669618  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:51.708917  1682 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:51.644685  1644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.001s	sys 0.014s
I20260709 13:01:51.654075  3592 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:51.726300  1644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.004s	sys 0.014s
W20260709 13:01:51.728475  1681 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.013s	sys 0.007s
W20260709 13:01:51.730108  1681 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.013s	sys 0.007s
I20260709 13:01:51.605742  3577 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:51.697682  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.006s	sys 0.019s
W20260709 13:01:51.737447  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 768290us
I20260709 13:01:51.698451  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:51.704829  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:51.705277  2300 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:51.568800  1641 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:51.706279  3653 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.633390  3392 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:51.722141  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 123234us
I20260709 13:01:51.706682  3637 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.654807  3602 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.723819  1715 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
W20260709 13:01:51.653303  1639 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.015s	sys 0.040s
W20260709 13:01:51.653259  3585 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.018s	sys 0.005s
W20260709 13:01:51.724872  2271 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.019s	sys 0.022s
W20260709 13:01:51.802647  2271 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.020s	sys 0.022s
W20260709 13:01:51.671240  3557 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.005s	sys 0.012s
I20260709 13:01:51.652760  3567 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:51.623560  2297 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.333s	user 0.038s	sys 0.092s
I20260709 13:01:51.732882  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:01:51.734122  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 694960us
I20260709 13:01:51.735023  1568 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:51.690001  1713 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
W20260709 13:01:51.714805  3471 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.027s	sys 0.030s
I20260709 13:01:51.742300  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:51.707677  3517 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.731343  3631 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:01:51.722743  3644 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.752627  1567 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:01:51.827759  1567 pending_rounds.cc:85] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 6
I20260709 13:01:51.828141  1567 pending_rounds.cc:107] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:01:51.724690  3348 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:51.725649  3640 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:51.771458  1569 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:51.707073  1711 rpcz_store.cc:276] 0709 13:01:50.035770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.374224 (+338454us) service_pool.cc:224] Handling call
0709 13:01:50.395701 (+ 21477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:50.471447 (+ 75746us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.480339 (+  8892us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:50.577437 (+ 97098us) log.cc:844] Serialized 26 byte log entry
0709 13:01:50.732605 (+155168us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:50.742544 (+  9939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.746293 (+  3749us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:51.578083 (+831790us) raft_consensus.cc:1685] finished
0709 13:01:51.578121 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.595748 (+ 17627us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67940,"raft.queue_time_us":141041,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":121,"thread_start_us":140810,"threads_started":1,"wal-append.queue_time_us":993743}
I20260709 13:01:51.774993  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:51.611550  3670 consensus_queue.cc:260] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:51.751044  3661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.006s	sys 0.020s
I20260709 13:01:51.740710  1635 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:51.741210  3577 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:51.740278  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:51.816011  1532 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:01:51.912276  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 96395.2us
I20260709 13:01:51.903159  2298 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
W20260709 13:01:51.764207  1639 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.017s	sys 0.041s
I20260709 13:01:51.818703  3567 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:51.652532  3669 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.007s	sys 0.018s
I20260709 13:01:51.727703  3635 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:51.915154  3669 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.008s	sys 0.018s
I20260709 13:01:51.915447  3669 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:51.916133  3567 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:51.916649  3635 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:51.916388  3669 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:51.917165  3635 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.656848  3681 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:51.807474  3585 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.019s	sys 0.005s
I20260709 13:01:51.922777  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:51.923036  3585 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:51.923547  3585 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.923959  3585 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:51.821141  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 78970.6us
I20260709 13:01:51.824331  3490 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:51.828601  1567 raft_consensus.cc:2889] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:51.811858  2297 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.521s	user 0.038s	sys 0.093s
W20260709 13:01:51.807511  3557 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.005s	sys 0.012s
I20260709 13:01:51.763724  3637 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:51.757902  3653 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:51.894356  3348 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.838415  2302 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Advancing to term 4
I20260709 13:01:51.839357  3571 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:51.839870  3660 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:51.758006  3508 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:01:51.843745  3665 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:51.943673  3348 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:51.962064  3665 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.962620  3665 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:51.933346  3490 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:51.963562  1568 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:51.656765  3684 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:51.820724  3471 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.027s	sys 0.030s
I20260709 13:01:51.809806  3612 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:51.821112  3451 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:01:51.896962  3661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.007s	sys 0.021s
I20260709 13:01:51.911105  3593 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:51.913712  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:51.919546  1569 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:51.812520  3392 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.086s
I20260709 13:01:51.799662  3602 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:51.812552  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 79823.4us
I20260709 13:01:51.926573  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:51.837517  2300 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:01:52.079489  3621 consensus_meta.cc:287] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.771s	user 0.007s	sys 0.005s
I20260709 13:01:52.079934  3621 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:52.082026  3621 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:51.917642  3635 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:52.084287  3635 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:51.917162  3640 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:52.084995  3640 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.085443  3640 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:52.085492  3571 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:52.086324  3571 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:52.087559  3571 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:51.901019  3607 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:51.925494  3680 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 13:01:51.940905  1643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.020s	sys 0.027s
W20260709 13:01:51.939700  3623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.470s	user 0.021s	sys 0.044s
I20260709 13:01:51.845777  3517 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:51.977975  2302 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:01:52.126050  2302 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:01:51.894665  3644 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:51.965204  3667 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.055s	user 0.011s	sys 0.040s
I20260709 13:01:51.981917  1562 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:51.986184  3681 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:51.965034  3394 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.036s	user 0.018s	sys 0.027s
I20260709 13:01:51.931631  3592 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:52.075264  3662 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:52.076066  1568 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:52.084393  3637 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:52.091434  1643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.020s	sys 0.027s
I20260709 13:01:51.979122  3451 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.094364  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:52.119738  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.120416  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:51.983961  3613 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.041s	user 0.022s	sys 0.040s
I20260709 13:01:51.988425  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 378) took 1345 ms. Trace:
I20260709 13:01:51.978657  3612 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:52.065053  3660 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.140638  1709 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 13:01:52.118520  3623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.653s	user 0.021s	sys 0.045s
W20260709 13:01:52.147815  1609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.233s	user 0.016s	sys 0.038s
I20260709 13:01:52.069525  3593 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.156360  1714 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
W20260709 13:01:52.163120  3613 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.218s	user 0.022s	sys 0.040s
W20260709 13:01:52.162459  1533 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.375s	user 0.006s	sys 0.051s
W20260709 13:01:52.240206  1533 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.453s	user 0.006s	sys 0.051s
W20260709 13:01:52.240599  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 1.46372e+06us
W20260709 13:01:52.150527  3394 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.221s	user 0.020s	sys 0.032s
I20260709 13:01:52.093884  2298 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:52.143002  3490 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:52.208998  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 114728us
I20260709 13:01:52.264146  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:52.209477  1568 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:01:52.210615  1609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.296s	user 0.016s	sys 0.038s
W20260709 13:01:52.134960  3667 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.225s	user 0.011s	sys 0.040s
I20260709 13:01:52.216275  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:01:52.218775  3683 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.623s	user 0.000s	sys 0.002s
I20260709 13:01:52.219678  1565 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:52.220319  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:52.221594  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:52.306797  1569 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 1.
I20260709 13:01:52.223824  1567 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:01:52.231516  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:52.234283  3665 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:52.244304  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:52.205839  3449 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.000s	sys 0.046s
W20260709 13:01:52.245177  1640 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.078s	sys 0.151s
I20260709 13:01:52.133472  3631 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 13:01:52.133458  3689 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.117s
W20260709 13:01:52.250252  2303 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.016s	sys 0.014s
W20260709 13:01:52.389159  2303 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.017s	sys 0.014s
I20260709 13:01:52.160661  3451 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:52.162814  3666 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:52.294958  1607 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.007s	sys 0.014s
W20260709 13:01:52.397034  1607 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.007s	sys 0.014s
I20260709 13:01:52.214995  3662 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.138s
I20260709 13:01:52.207382  3697 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.190s
I20260709 13:01:52.310002  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:52.266144  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:52.312543  1638 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.015s	sys 0.019s
I20260709 13:01:52.210521  3623 raft_consensus.cc:697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:52.162691  3681 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 13:01:52.318447  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:52.349260  3676 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.000s	sys 0.033s
W20260709 13:01:52.376467  1640 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.078s	sys 0.151s
I20260709 13:01:52.238956  3593 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:52.365162  1710 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.007s	sys 0.016s
W20260709 13:01:52.479481  1710 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.007s	sys 0.018s
W20260709 13:01:52.378969  1561 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.007s	sys 0.014s
W20260709 13:01:52.381121  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.009s	sys 0.026s
I20260709 13:01:52.165109  3508 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.152s
I20260709 13:01:52.265075  3687 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:52.300716  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 84557.1us
I20260709 13:01:52.415050  3701 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.144s
I20260709 13:01:52.402339  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:52.250059  3612 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:52.376891  3449 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.000s	sys 0.047s
I20260709 13:01:52.300096  3538 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:01:52.315205  3637 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:01:52.424300  1638 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.015s	sys 0.019s
W20260709 13:01:52.327075  2299 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.008s	sys 0.012s
I20260709 13:01:52.372908  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 715) took 1769 ms. Trace:
W20260709 13:01:52.430825  1683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.012s	sys 0.018s
I20260709 13:01:52.350049  3615 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:52.435479  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.009s	sys 0.035s
I20260709 13:01:52.423733  1641 raft_consensus.cc:2410] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:52.375742  3621 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:52.213927  3660 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:52.324906  3661 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:52.306109  3348 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:52.325184  2296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.001s	sys 0.020s
I20260709 13:01:52.431749  3471 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.109s
W20260709 13:01:52.856841  2271 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.053s	user 0.014s	sys 0.055s
W20260709 13:01:52.935020  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.127s	user 0.016s	sys 0.044s
W20260709 13:01:52.935593  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.123s	user 0.009s	sys 0.052s
I20260709 13:01:52.901681  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 356) took 1524 ms. Trace:
I20260709 13:01:52.936635  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2069) took 1322 ms. Trace:
W20260709 13:01:52.937196  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 1.12491e+06us
I20260709 13:01:52.931146  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2064) took 1653 ms. Trace:
I20260709 13:01:52.939698  1643 rpcz_store.cc:276] 0709 13:01:51.277310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.299980 (+ 22670us) service_pool.cc:224] Handling call
0709 13:01:51.339411 (+ 39431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:51.339789 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.339822 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:51.340360 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 13:01:52.145371 (+805011us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:52.145433 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.145533 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:52.749428 (+603895us) raft_consensus.cc:1685] finished
0709 13:01:52.749453 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.931073 (+181620us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37878,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":30241,"thread_start_us":804559,"threads_started":1,"wal-append.queue_time_us":1179133}
W20260709 13:01:52.943401  2271 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.139s	user 0.016s	sys 0.055s
W20260709 13:01:52.944141  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:52.435925  3676 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.071s	user 0.001s	sys 0.034s
I20260709 13:01:52.936326  1564 rpcz_store.cc:276] 0709 13:01:51.342807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.342994 (+   187us) service_pool.cc:224] Handling call
0709 13:01:51.343330 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:51.676394 (+333064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.856309 (+1179915us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54001ff754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:52.859419 (+  3110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659659008}
I20260709 13:01:52.433128  3653 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:52.755208  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2063) took 1478 ms. Trace:
W20260709 13:01:52.494001  2270 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.004s	sys 0.029s
W20260709 13:01:52.983817  2270 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.114s	user 0.015s	sys 0.044s
W20260709 13:01:52.976362  1683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.235s	user 0.026s	sys 0.029s
W20260709 13:01:52.986325  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 1.31166e+06us
W20260709 13:01:52.495033  2304 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.021s	sys 0.012s
W20260709 13:01:52.988211  2304 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.209s	user 0.033s	sys 0.022s
W20260709 13:01:52.975745  1561 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.155s	user 0.022s	sys 0.033s
I20260709 13:01:52.990111  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2074) took 1268 ms. Trace:
I20260709 13:01:52.937166  1638 rpcz_store.cc:276] 0709 13:01:51.614362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.616208 (+  1846us) service_pool.cc:224] Handling call
0709 13:01:51.616630 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:51.630781 (+ 14151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.630812 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:51.631366 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 13:01:52.936268 (+1304902us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:52.936310 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.936382 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:52.936425 (+    43us) raft_consensus.cc:1685] finished
0709 13:01:52.936437 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.936583 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13724,"raft.queue_time_us":15342,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9050,"thread_start_us":1228642,"threads_started":1,"wal-append.queue_time_us":1231473}
I20260709 13:01:52.977277  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:52.959746  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:52.934028  2305 rpcz_store.cc:276] 0709 13:01:50.603726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.603897 (+   171us) service_pool.cc:224] Handling call
0709 13:01:50.605083 (+  1186us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:50.617818 (+ 12735us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.617847 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:50.683554 (+ 65707us) log.cc:844] Serialized 26 byte log entry
0709 13:01:51.193224 (+509670us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:51.193286 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.193368 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:51.924287 (+730919us) spinlock_profiling.cc:243] Waited 4.66 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:01:52.289391 (+365104us) raft_consensus.cc:1685] finished
0709 13:01:52.289411 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.372827 (+ 83416us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8124,"raft.queue_time_us":71853,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":34233,"spinlock_wait_cycles":10715904,"thread_start_us":459729,"threads_started":1,"wal-append.queue_time_us":961930}
W20260709 13:01:52.716483  1515 auto_rebalancer.cc:893] Removing replace marker failed: Replica 3f64b76180a54492bb015e405caeacec is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:01:52.995425  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:52.747387  3696 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
W20260709 13:01:52.945713  2299 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.189s	user 0.016s	sys 0.020s
W20260709 13:01:53.004180  1515 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:01:52.773998  3557 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.013s	sys 0.040s
I20260709 13:01:53.003561  1564 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:52.976814  1568 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
W20260709 13:01:52.909368  3703 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.373s	user 0.000s	sys 0.000s
W20260709 13:01:52.717824  3694 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.481s	user 0.000s	sys 0.000s
W20260709 13:01:52.982833  2268 connection.cc:256] RPC call timeout handler was delayed by -1.6930791639999825s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20400000000000001s for a response
W20260709 13:01:52.983255  1711 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.009s	sys 0.079s
W20260709 13:01:53.068256  2268 connection.cc:256] RPC call timeout handler was delayed by -1.658879163999984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
W20260709 13:01:53.068926  2268 connection.cc:256] RPC call timeout handler was delayed by -1.625579163999987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21150000000000002s for a response
W20260709 13:01:52.859442  3607 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.022s	sys 0.030s
W20260709 13:01:53.069664  2268 connection.cc:256] RPC call timeout handler was delayed by -1.5004791640000121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22540000000000002s for a response
I20260709 13:01:52.700593  3684 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:52.699256  3392 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.607676  3702 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:01:52.973593  3577 consensus_meta.cc:287] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.066s	user 0.005s	sys 0.006s
I20260709 13:01:52.948812  3615 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:53.074431  3607 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.024s	sys 0.031s
W20260709 13:01:52.717229  3695 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.510s	user 0.000s	sys 0.001s
I20260709 13:01:52.990633  1561 rpcz_store.cc:276] 0709 13:01:51.721696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.723553 (+  1857us) service_pool.cc:224] Handling call
0709 13:01:51.737068 (+ 13515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:51.749979 (+ 12911us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.750031 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:51.750581 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:01:52.989598 (+1239017us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:52.989654 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.989751 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:52.989864 (+   113us) raft_consensus.cc:1685] finished
0709 13:01:52.989881 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.990043 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3054,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":10396,"thread_start_us":1168555,"threads_started":1,"wal-append.queue_time_us":1099548}
I20260709 13:01:53.095587  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:53.096189  1561 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:53.010612  2302 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Advancing to term 5
I20260709 13:01:52.607363  3613 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.076378  3713 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:52.994881  3612 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
W20260709 13:01:53.013484  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.017066  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:52.976017  3687 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:52.700285  3631 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.983327  1640 rpcz_store.cc:276] 0709 13:01:51.276958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.299892 (+ 22934us) service_pool.cc:224] Handling call
0709 13:01:51.531053 (+231161us) spinlock_profiling.cc:243] Waited 8.81 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7ce9c 00007fd79eb7ca5b 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:01:51.531320 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:51.531932 (+   612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.531975 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:51.559064 (+ 27089us) log.cc:844] Serialized 26 byte log entry
0709 13:01:52.436179 (+877115us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:52.436236 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.436330 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:52.718875 (+282545us) raft_consensus.cc:1685] finished
0709 13:01:52.718902 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.755119 (+ 36217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3486,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":24991,"spinlock_wait_cycles":20270848,"thread_start_us":824393,"threads_started":1,"wal-append.queue_time_us":1145733}
W20260709 13:01:52.718360  3690 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.623s	user 0.000s	sys 0.001s
I20260709 13:01:52.912030  3602 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:52.747071  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:52.756101  3644 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:52.436204  3666 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.573s
I20260709 13:01:52.648754  3665 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.363s
W20260709 13:01:52.978838  3301 consensus_meta.cc:287] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.254s	user 0.012s	sys 0.000s
I20260709 13:01:52.699393  3490 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.071265  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:52.545318  1635 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
W20260709 13:01:53.016618  1515 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:01:53.071941  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:01:53.013366  3593 consensus_meta.cc:287] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.000s
I20260709 13:01:53.081310  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:53.013907  3681 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.012005  3550 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:53.070757  1711 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.009s	sys 0.079s
I20260709 13:01:53.012177  3653 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:52.997007  2297 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.020s	sys 0.036s
I20260709 13:01:53.082571  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:53.209321  1635 pending_rounds.cc:85] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 1
I20260709 13:01:53.018705  3538 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.979038  3348 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 13:01:52.973555  3621 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:53.019214  3682 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.017s	sys 0.028s
I20260709 13:01:53.005880  3693 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:53.071806  3557 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.100s	user 0.015s	sys 0.049s
I20260709 13:01:53.073518  3394 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.066622  3592 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:53.072924  3629 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:52.937357  2296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.143s	user 0.008s	sys 0.029s
I20260709 13:01:53.108381  2302 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:01:53.291313  2268 connection.cc:256] RPC call timeout handler was delayed by -1.052882344000011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
I20260709 13:01:52.748783  3589 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.328s
W20260709 13:01:52.716109  3705 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.480s	user 0.002s	sys 0.000s
I20260709 13:01:53.114733  1561 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
W20260709 13:01:53.117097  1642 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.892s	user 0.038s	sys 0.047s
I20260709 13:01:53.281764  3653 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.802145  3623 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.283703  3684 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.257087  3681 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.093334  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2058) took 2713 ms. Trace:
W20260709 13:01:53.168938  1609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.003s	sys 0.012s
W20260709 13:01:53.323024  1609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.003s	sys 0.012s
W20260709 13:01:53.323486  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 661735us
I20260709 13:01:52.994467  3661 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.092870  3577 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.233053  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:53.332736  1640 raft_consensus.cc:2410] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:53.118239  3701 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:53.334836  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.085459  3392 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.232398  1564 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 4: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:53.244138  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:53.236521  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 377) took 2599 ms. Trace:
I20260709 13:01:53.258035  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1765) took 1006 ms. Trace:
W20260709 13:01:52.746743  3541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.018s	sys 0.031s
W20260709 13:01:53.258468  2303 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.015s	sys 0.028s
W20260709 13:01:53.343153  2303 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.016s	sys 0.028s
I20260709 13:01:53.340845  1568 rpcz_store.cc:276] 0709 13:01:52.251986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.082292 (+830306us) service_pool.cc:224] Handling call
0709 13:01:53.257947 (+175655us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:53.261780  1535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.023s	sys 0.016s
I20260709 13:01:53.184320  3602 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:53.265565  1684 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.020s	sys 0.020s
I20260709 13:01:53.186515  3709 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:01:53.257422  2297 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.009s	user 0.020s	sys 0.037s
I20260709 13:01:53.130856  3631 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.262811  1635 pending_rounds.cc:107] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:01:53.090198  3702 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.234829  3717 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.111940  3613 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:53.261274  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1766) took 1004 ms. Trace:
I20260709 13:01:53.289131  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 721) took 1578 ms. Trace:
I20260709 13:01:53.258082  3721 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:53.272457  3682 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.127s	user 0.017s	sys 0.029s
I20260709 13:01:53.014667  3692 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.293687  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:01:53.427166  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 133549us
I20260709 13:01:53.427333  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1546) took 1350 ms. Trace:
W20260709 13:01:53.294210  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.368s	user 0.015s	sys 0.085s
W20260709 13:01:53.428758  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.502s	user 0.015s	sys 0.086s
W20260709 13:01:53.429109  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 1.50293e+06us
I20260709 13:01:53.294720  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 716) took 2681 ms. Trace:
I20260709 13:01:53.430341  2301 rpcz_store.cc:276] 0709 13:01:50.607564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.607716 (+   152us) service_pool.cc:224] Handling call
0709 13:01:50.632253 (+ 24537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:50.632696 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.632734 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:50.728216 (+ 95482us) log.cc:844] Serialized 26 byte log entry
0709 13:01:51.678671 (+950455us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:51.678738 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.678840 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:53.288885 (+1610045us) raft_consensus.cc:1685] finished
0709 13:01:53.288902 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.289026 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2955,"raft.queue_time_us":97631,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":9940,"thread_start_us":930840,"threads_started":1,"wal-append.queue_time_us":1221861}
W20260709 13:01:53.297049  1534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.872s	user 0.024s	sys 0.022s
I20260709 13:01:53.237207  3490 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:53.299134  1533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.929s	user 0.032s	sys 0.070s
W20260709 13:01:53.433842  1533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.064s	user 0.032s	sys 0.070s
W20260709 13:01:53.301043  1642 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.076s	user 0.038s	sys 0.047s
I20260709 13:01:53.081804  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:53.232426  3301 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:53.257695  3508 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:01:53.262288  3550 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.092537  3615 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:53.312940  1681 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.063s	user 0.017s	sys 0.063s
W20260709 13:01:53.475522  1681 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.226s	user 0.018s	sys 0.063s
I20260709 13:01:53.314191  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:53.475929  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 1.22626e+06us
I20260709 13:01:53.127358  3713 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:53.238691  3593 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:53.273077  3538 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.336088  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:53.338236  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:53.487977  1564 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:01:53.488773  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1760) took 1238 ms. Trace:
I20260709 13:01:53.489250  1564 rpcz_store.cc:276] 0709 13:01:52.250536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.337979 (+1087443us) service_pool.cc:224] Handling call
0709 13:01:53.488693 (+150714us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:53.328142  2300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.008s	sys 0.033s
I20260709 13:01:53.321985  3735 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:53.344045  1535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.023s	sys 0.016s
W20260709 13:01:53.343817  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 99617.5us
W20260709 13:01:53.346830  1684 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.021s	sys 0.020s
I20260709 13:01:53.347554  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:53.505023  1568 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:53.505569  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1764) took 1253 ms. Trace:
I20260709 13:01:53.506008  1568 rpcz_store.cc:276] 0709 13:01:52.251674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.347234 (+1095560us) service_pool.cc:224] Handling call
0709 13:01:53.505502 (+158268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.506134  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:53.472018  3687 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:53.296576  3629 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:53.291570  3394 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:53.342931  3541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.300s	user 0.019s	sys 0.041s
I20260709 13:01:53.312374  3666 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:01:52.163481  1639 rpcz_store.cc:276] 0709 13:01:50.642688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.804634 (+161946us) service_pool.cc:224] Handling call
0709 13:01:50.805015 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:50.805441 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.805486 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:50.806001 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 13:01:51.914321 (+1108320us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:51.914378 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.914462 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:51.964671 (+ 50209us) raft_consensus.cc:1685] finished
0709 13:01:51.967360 (+  2689us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.982536 (+ 15176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22471,"raft.queue_time_us":14460,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":27350,"thread_start_us":1107893,"threads_started":1,"wal-append.queue_time_us":1156865,"wal-append.run_cpu_time_us":1671,"wal-append.run_wall_time_us":1001693}
W20260709 13:01:53.384150  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.396400  2304 rpcz_store.cc:276] 0709 13:01:51.710656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.743079 (+ 32423us) service_pool.cc:224] Handling call
0709 13:01:51.753247 (+ 10168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:51.753515 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:52.988795 (+1235280us) log.cc:844] Serialized 14 byte log entry
0709 13:01:53.288834 (+300039us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:53.288884 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:53.288922 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.288984 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.289081 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":669879,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":932,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":3994,"thread_start_us":1509163,"threads_started":2}
I20260709 13:01:53.320401  3653 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:53.286782  3693 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.341084  3660 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:53.325309  3684 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.344352  3348 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.407374  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1537) took 1770 ms. Trace:
I20260709 13:01:53.346879  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 723) took 1553 ms. Trace:
W20260709 13:01:53.425289  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:53.425863  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 101516us
I20260709 13:01:53.339425  1644 rpcz_store.cc:276] 0709 13:01:50.636768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.772749 (+135981us) service_pool.cc:224] Handling call
0709 13:01:50.773383 (+   634us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:50.774068 (+   685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.774126 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:50.851485 (+ 77359us) log.cc:844] Serialized 52 byte log entry
0709 13:01:51.728202 (+876717us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:51.728247 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.728333 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:53.212851 (+1484518us) raft_consensus.cc:1685] finished
0709 13:01:53.212887 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.236424 (+ 23537us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31620,"raft.queue_time_us":11984,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":23373,"thread_start_us":827004,"threads_started":1,"wal-append.queue_time_us":1261931}
I20260709 13:01:53.404599  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:01:53.432289  1534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.024s	sys 0.022s
I20260709 13:01:53.344552  3739 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.442350  1641 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:53.402595  1635 raft_consensus.cc:2889] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:53.456049  3640 consensus_meta.cc:287] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.370s	user 0.010s	sys 0.004s
I20260709 13:01:53.456118  3692 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.406672  3721 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:53.460140  3723 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
W20260709 13:01:53.461057  3722 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 13:01:53.396421  3709 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.406363  3631 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:53.466188  1533 connection.cc:256] RPC call timeout handler was delayed by -1.2153339830000505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
I20260709 13:01:53.342986  3701 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:53.356753  3602 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.346019  3607 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 13:01:53.396662  1560 rpcz_store.cc:276] 0709 13:01:52.252256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.548481 (+296225us) service_pool.cc:224] Handling call
0709 13:01:53.256740 (+708259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.479128  1638 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:53.486577  3682 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:53.495333  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:53.453692  3636 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.496836  2305 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:53.463282  3718 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.506848  1564 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:53.343888  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 720) took 1656 ms. Trace:
I20260709 13:01:53.507177  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:53.505429  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 77060.7us
I20260709 13:01:53.421631  3717 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.481060  3592 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:01:53.429278  3712 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.268s
I20260709 13:01:53.322852  1636 rpcz_store.cc:276] 0709 13:01:50.372327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.561913 (+189586us) service_pool.cc:224] Handling call
0709 13:01:50.562252 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:50.605043 (+ 42791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.612728 (+  7685us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:50.728149 (+115421us) log.cc:844] Serialized 26 byte log entry
0709 13:01:51.216689 (+488540us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:51.254703 (+ 38014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.271997 (+ 17294us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:52.218184 (+946187us) spinlock_profiling.cc:243] Waited 64.8 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:01:52.975133 (+756949us) raft_consensus.cc:1685] finished
0709 13:01:52.975160 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.080460 (+105300us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25672,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":20241,"spinlock_wait_cycles":149051264,"thread_start_us":403901,"threads_started":1,"wal-append.queue_time_us":841366}
W20260709 13:01:53.718228  3451 consensus_meta.cc:287] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.286s	user 0.007s	sys 0.006s
I20260709 13:01:53.718708  3451 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.719105  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:53.427897  1711 rpcz_store.cc:276] 0709 13:01:52.077335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.077515 (+   180us) service_pool.cc:224] Handling call
0709 13:01:52.077834 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.078049 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:53.270726 (+1192677us) log.cc:844] Serialized 14 byte log entry
0709 13:01:53.426789 (+156063us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:53.426866 (+    77us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:53.426951 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.427056 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.427250 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":389902,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":27088,"raft.queue_time_us":148,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":1312588,"threads_started":2}
W20260709 13:01:53.717648  3727 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.343s	user 0.001s	sys 0.001s
W20260709 13:01:53.721087  1533 connection.cc:256] RPC call timeout handler was delayed by -1.188071438999998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
W20260709 13:01:53.721946  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 1.45589e+06us
I20260709 13:01:53.723520  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:53.723433  2303 rpcz_store.cc:276] 0709 13:01:51.687542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.687698 (+   156us) service_pool.cc:224] Handling call
0709 13:01:51.688139 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:51.688432 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:52.389623 (+701191us) log.cc:844] Serialized 14 byte log entry
0709 13:01:53.343562 (+953939us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:53.343611 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:53.343654 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.343724 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.343836 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9477,"prepare.queue_time_us":474507,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":3058,"raft.queue_time_us":1123,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":10117,"thread_start_us":1635215,"threads_started":2,"wal-append.queue_time_us":1034303}
I20260709 13:01:53.724545  1710 rpcz_store.cc:276] 0709 13:01:51.636661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.636827 (+   166us) service_pool.cc:224] Handling call
0709 13:01:51.637242 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:52.480118 (+842876us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:52.480208 (+    90us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:52.538905 (+ 58697us) log.cc:844] Serialized 26 byte log entry
0709 13:01:53.184079 (+645174us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:53.184115 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.184957 (+   842us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:53.407120 (+222163us) raft_consensus.cc:1685] finished
0709 13:01:53.407157 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.407310 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37298,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":10723,"thread_start_us":1238862,"threads_started":2,"wal-append.queue_time_us":659902}
I20260709 13:01:53.725745  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:53.727151  1640 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:53.727190  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:53.728433  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1595) took 1419 ms. Trace:
I20260709 13:01:53.728365  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:53.728727  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:53.729655  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:53.596596  3733 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:53.730497  3733 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.731097  3684 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:53.732231  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:53.643047  3681 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:53.733126  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1603) took 1413 ms. Trace:
I20260709 13:01:53.733388  3681 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:53.735836  3681 raft_consensus.cc:2764] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:01:53.752388  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1540) took 2113 ms. Trace:
I20260709 13:01:53.753026  1709 rpcz_store.cc:276] 0709 13:01:51.638868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.639040 (+   172us) service_pool.cc:224] Handling call
0709 13:01:51.662183 (+ 23143us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.208280 (+546097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.752105 (+1543825us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54000483d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:53.752292 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1087955200}
I20260709 13:01:53.736351  2296 rpcz_store.cc:276] 0709 13:01:51.791313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.791525 (+   212us) service_pool.cc:224] Handling call
0709 13:01:51.792081 (+   556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:51.792643 (+   562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.792693 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:51.793488 (+   795us) log.cc:844] Serialized 26 byte log entry
0709 13:01:53.304216 (+1510728us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:53.312022 (+  7806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.329141 (+ 17119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:53.333725 (+  4584us) raft_consensus.cc:1685] finished
0709 13:01:53.337429 (+  3704us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.344845 (+  7416us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83527,"raft.queue_time_us":414736,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":51944,"thread_start_us":1497236,"threads_started":1,"wal-append.queue_time_us":1102116}
I20260709 13:01:53.739293  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:53.739979  1643 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:53.546032  3621 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:53.741698  3702 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:53.741861  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:53.742043  3692 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:53.742722  3640 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.758436  1560 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 1.
I20260709 13:01:53.758685  3629 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:53.759068  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1770) took 1335 ms. Trace:
I20260709 13:01:53.759246  3629 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.597182  3732 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:53.760133  3732 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.759936  3629 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.760984  3732 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:53.761787  3640 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:53.596735  3734 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.744227  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2088) took 1524 ms. Trace:
I20260709 13:01:53.763335  1564 rpcz_store.cc:276] 0709 13:01:52.219878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.504867 (+1284989us) service_pool.cc:224] Handling call
0709 13:01:53.744158 (+239291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.763046  3734 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.764984  1564 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:53.765833  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1758) took 1515 ms. Trace:
I20260709 13:01:53.766294  1564 rpcz_store.cc:276] 0709 13:01:52.250045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.764316 (+1514271us) service_pool.cc:224] Handling call
0709 13:01:53.764744 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.765519 (+   775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.765759 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.770046  3629 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:53.588958  1717 raft_consensus.cc:2393] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:53.746778  3541 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:53.778349  3541 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:53.736577  3682 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.782078  3682 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:53.783253  3541 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.739226  3653 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:53.731928  3684 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:53.730851  1635 rpcz_store.cc:276] 0709 13:01:52.309210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.313598 (+  4388us) service_pool.cc:224] Handling call
0709 13:01:52.427973 (+114375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.728174 (+1300201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.728374 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.786509  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:53.787380  1635 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:53.787844  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1605) took 1467 ms. Trace:
I20260709 13:01:53.733578  3733 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:53.756764  3721 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:53.796684  1640 raft_consensus.cc:2410] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:53.759109  1561 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 4: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:53.596442  3740 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:53.625801  3679 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 2.003s	user 0.002s	sys 0.003s
I20260709 13:01:53.745385  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1539) took 2107 ms. Trace:
I20260709 13:01:53.756026  3621 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.767294  3692 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:53.759639  1560 rpcz_store.cc:276] 0709 13:01:52.423686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.741550 (+1317864us) service_pool.cc:224] Handling call
0709 13:01:53.759007 (+ 17457us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.745968  1566 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:01:53.804517  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2087) took 1584 ms. Trace:
I20260709 13:01:53.802561  3740 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.804960  1566 rpcz_store.cc:276] 0709 13:01:52.219608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.745231 (+1525623us) service_pool.cc:224] Handling call
0709 13:01:53.745534 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.804222 (+ 58688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.804455 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:53.722728  1533 connection.cc:256] RPC call timeout handler was delayed by -1.0467714389999969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2033s for a response
I20260709 13:01:53.802940  1715 rpcz_store.cc:276] 0709 13:01:51.638324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.638445 (+   121us) service_pool.cc:224] Handling call
0709 13:01:53.744795 (+2106350us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b5400474354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:53.745304 (+   509us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2630912}
I20260709 13:01:53.726136  3602 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:53.508385  3735 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:53.809150  3740 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:53.788312  1635 rpcz_store.cc:276] 0709 13:01:52.320694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.786229 (+1465535us) service_pool.cc:224] Handling call
0709 13:01:53.787785 (+  1556us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.597494  3713 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:53.811580  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:53.812261  3735 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:53.813131  1635 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:53.813898  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1609) took 1444 ms (client timeout 1735 ms). Trace:
I20260709 13:01:53.803913  3692 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.797560  3684 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:53.811956  1636 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:53.816670  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1600) took 1506 ms. Trace:
I20260709 13:01:53.817497  1636 rpcz_store.cc:276] 0709 13:01:52.310642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.718768 (+1408126us) service_pool.cc:224] Handling call
0709 13:01:53.816427 (+ 97659us) spinlock_profiling.cc:243] Waited 92 ms on lock 0x7b540032bed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:53.816605 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211581824}
I20260709 13:01:53.787743  3702 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:01:53.819097  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:53.820509  1636 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:01:53.821079  3667 raft_consensus.cc:3055] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:53.821496  3667 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:53.821697  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:53.822746  1636 raft_consensus.cc:2410] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:53.823251  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1606) took 1502 ms. Trace:
I20260709 13:01:53.799321  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:53.814286  1643 rpcz_store.cc:269] 0709 13:01:52.369188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.002801 (+633613us) service_pool.cc:224] Handling call
0709 13:01:53.813566 (+810765us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b540032c3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:53.813676 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1161675136}
I20260709 13:01:53.803664  3621 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.755932  1644 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:53.833513  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51753.2us
W20260709 13:01:53.848951  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.816609  3684 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.757897  1569 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:53.826087  3517 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:53.826531  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2089) took 1606 ms. Trace:
W20260709 13:01:53.496147  2300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.008s	sys 0.033s
I20260709 13:01:53.827978  1566 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:01:53.852345  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1757) took 1602 ms. Trace:
I20260709 13:01:53.816705  3692 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:53.867205  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1768) took 1614 ms. Trace:
I20260709 13:01:53.874938  1568 rpcz_store.cc:276] 0709 13:01:52.252782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.506759 (+1253977us) service_pool.cc:224] Handling call
0709 13:01:53.867097 (+360338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.734194  1639 rpcz_store.cc:276] 0709 13:01:52.320160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.731943 (+1411783us) service_pool.cc:224] Handling call
0709 13:01:53.733064 (+  1121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.876681  2303 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:53.877619  1569 rpcz_store.cc:276] 0709 13:01:52.249747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.757210 (+1507463us) service_pool.cc:224] Handling call
0709 13:01:53.757641 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.852068 (+ 94427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.852285 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.809999  1637 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:01:53.853008  1561 rpcz_store.cc:276] 0709 13:01:52.220145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.310703 (+1090558us) service_pool.cc:224] Handling call
0709 13:01:53.826324 (+515621us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54001cf054. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:53.826468 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":650247424}
I20260709 13:01:53.882625  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1761) took 1631 ms. Trace:
I20260709 13:01:53.892335  1566 rpcz_store.cc:276] 0709 13:01:52.250824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.805786 (+1554962us) service_pool.cc:224] Handling call
0709 13:01:53.827669 (+ 21883us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.882234 (+ 54565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.882539 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:53.829018  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1610) took 1459 ms (client timeout 1894 ms). Trace:
I20260709 13:01:53.557495  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 719) took 1859 ms. Trace:
I20260709 13:01:53.899132  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:53.589587  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:53.661554  3742 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 13:01:53.893994  1640 rpcz_store.cc:269] 0709 13:01:52.369562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.346563 (+977001us) service_pool.cc:224] Handling call
0709 13:01:53.828957 (+482394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.895548  1568 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:53.948995  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1763) took 1697 ms. Trace:
W20260709 13:01:53.672845  3744 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:01:53.659214  3729 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.330s	user 0.002s	sys 0.000s
I20260709 13:01:53.899894  1643 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
W20260709 13:01:53.901201  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 52371.6us
I20260709 13:01:53.717355  3754 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:53.827839  3577 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:53.711874  3301 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:53.831614  1562 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:53.697428  3661 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:53.946170  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2070) took 2331 ms. Trace:
I20260709 13:01:53.556922  3613 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:53.832603  3348 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.832144  3636 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.822501  3667 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.681401  3748 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.492s
W20260709 13:01:54.018632  1535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.008s	sys 0.026s
W20260709 13:01:54.205614  1535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.014s	sys 0.037s
I20260709 13:01:54.206944  1642 rpcz_store.cc:276] 0709 13:01:51.614677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.615033 (+   356us) service_pool.cc:224] Handling call
0709 13:01:52.223117 (+608084us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:01:52.223312 (+   195us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:52.223710 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:52.223738 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:52.224347 (+   609us) log.cc:844] Serialized 52 byte log entry
0709 13:01:53.434942 (+1210595us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:53.435009 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.435127 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:53.945797 (+510670us) raft_consensus.cc:1685] finished
0709 13:01:53.945818 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.946103 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1395,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"spinlock_wait_cycles":3566336,"thread_start_us":1210089,"threads_started":1,"wal-append.queue_time_us":1073286}
I20260709 13:01:53.909428  2303 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:53.949369  1568 rpcz_store.cc:276] 0709 13:01:52.251414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.894686 (+1643272us) service_pool.cc:224] Handling call
0709 13:01:53.895133 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.948711 (+ 53578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.948937 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:54.186318  1641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1601) took 1875 ms (client timeout 1570 ms). Trace:
W20260709 13:01:54.213390  1641 rpcz_store.cc:269] 0709 13:01:52.319359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.976987 (+657628us) service_pool.cc:224] Handling call
0709 13:01:54.186028 (+1209041us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b54001973d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:54.186212 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1012711168}
I20260709 13:01:53.867945  3602 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:54.219794  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.880820  3451 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:54.223572  1640 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:54.227473  3743 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.229816  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1597) took 1919 ms. Trace:
I20260709 13:01:54.230439  1640 rpcz_store.cc:276] 0709 13:01:52.309890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.222939 (+1913049us) service_pool.cc:224] Handling call
0709 13:01:54.223329 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.229518 (+  6189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.229742 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.943964  1635 raft_consensus.cc:2410] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:53.879544  3640 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.766708  3734 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:53.847231  1636 rpcz_store.cc:276] 0709 13:01:52.320963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.821399 (+1500436us) service_pool.cc:224] Handling call
0709 13:01:53.823176 (+  1777us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:54.204082  3747 log.cc:927] Time spent T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.139s	user 0.002s	sys 0.000s
I20260709 13:01:53.905791  3682 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:54.208477  3392 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:53.905184  3490 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:53.947381  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:53.946419  3550 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:53.946892  3593 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:54.251569  3743 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:54.252249  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 531) took 4159 ms. Trace:
I20260709 13:01:54.409513  2297 rpcz_store.cc:276] 0709 13:01:50.092597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.289570 (+196973us) service_pool.cc:224] Handling call
0709 13:01:50.290010 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:51.937539 (+1647529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.942869 (+  5330us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:52.121965 (+179096us) log.cc:844] Serialized 26 byte log entry
0709 13:01:53.398649 (+1276684us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:53.411747 (+ 13098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.421240 (+  9493us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:54.251993 (+830753us) raft_consensus.cc:1685] finished
0709 13:01:54.252019 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.252177 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77632,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":2745366,"threads_started":2,"wal-append.queue_time_us":1115831}
I20260709 13:01:54.253171  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:54.412809  1643 raft_consensus.cc:2410] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:54.413306  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 386) took 1406 ms. Trace:
I20260709 13:01:54.413622  1643 rpcz_store.cc:276] 0709 13:01:53.006582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.221170 (+1214588us) service_pool.cc:224] Handling call
0709 13:01:54.413244 (+192074us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:53.854764  3585 consensus_meta.cc:287] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.924s	user 0.008s	sys 0.008s
I20260709 13:01:54.203842  3749 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:54.414582  3585 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:54.289109  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:54.364285  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:54.205039  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1542) took 2481 ms (client timeout 1735 ms). Trace:
W20260709 13:01:54.392800  1638 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1598) took 2082 ms (client timeout 1724 ms). Trace:
I20260709 13:01:54.381762  3661 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:54.313673  3640 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:54.417132  3585 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:54.419410  1638 rpcz_store.cc:269] 0709 13:01:52.310121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.016726 (+706605us) service_pool.cc:224] Handling call
0709 13:01:54.392498 (+1375772us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54001e8f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:54.392717 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838334208}
I20260709 13:01:54.436655  1568 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:01:54.448535  1567 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:54.393225  1640 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 4. Ops: []
W20260709 13:01:54.018970  3449 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.012s	sys 0.023s
W20260709 13:01:54.419320  1717 rpcz_store.cc:269] 0709 13:01:51.723506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.724659 (+  1153us) service_pool.cc:224] Handling call
0709 13:01:54.204718 (+2480059us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b5400239e54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:54.204941 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3678632704}
W20260709 13:01:54.463141  3449 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.985s	user 0.013s	sys 0.024s
W20260709 13:01:54.463801  3745 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.316s	user 0.000s	sys 0.000s
I20260709 13:01:54.431835  1560 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:01:54.464924  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1767) took 2212 ms. Trace:
I20260709 13:01:54.245565  3592 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:54.404579  1564 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:54.469914  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:54.470809  1569 raft_consensus.cc:2368] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 3. Current term is 4.
I20260709 13:01:53.876050  3718 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:53.944965  3670 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:54.461470  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1602) took 2141 ms. Trace:
I20260709 13:01:54.414821  3749 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:54.464970  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 731) took 1398 ms. Trace:
I20260709 13:01:54.476523  2303 rpcz_store.cc:276] 0709 13:01:53.066546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.725503 (+658957us) service_pool.cc:224] Handling call
0709 13:01:54.464828 (+739325us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b5400600254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:54.464918 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":344769408}
I20260709 13:01:54.415736  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:54.478459  1565 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:54.244154  3667 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:54.471810  1640 rpcz_store.cc:276] 0709 13:01:52.319858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.236655 (+1916797us) service_pool.cc:224] Handling call
0709 13:01:54.392992 (+156337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.461106 (+ 68114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.461388 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.469632  3592 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:54.433431  3661 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:54.473907  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:54.494510  1569 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
W20260709 13:01:54.482818  3738 log.cc:927] Time spent T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:01:54.424536  3348 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:54.437218  1639 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 2: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:01:54.392745  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 382) took 1438 ms (client timeout 1598 ms). Trace:
I20260709 13:01:54.394761  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 543) took 1311 ms. Trace:
W20260709 13:01:54.449258  3615 consensus_meta.cc:287] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.959s	user 0.010s	sys 0.002s
W20260709 13:01:54.550907  1608 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.068s	user 0.025s	sys 0.055s
W20260709 13:01:54.551388  1608 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.069s	user 0.025s	sys 0.055s
W20260709 13:01:54.553829  1532 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.000s	sys 0.049s
W20260709 13:01:54.554199  1532 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.043s	user 0.000s	sys 0.049s
W20260709 13:01:54.554487  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 1.04532e+06us
I20260709 13:01:54.490840  1640 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:53.891904  3739 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:54.518783  3752 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.893s	user 0.019s	sys 0.013s
W20260709 13:01:54.519522  1684 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.011s	user 0.014s	sys 0.019s
I20260709 13:01:54.419108  3636 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:54.244419  3602 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:54.427615  3593 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:54.494246  3637 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:54.545739  2301 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.015s	sys 0.020s
W20260709 13:01:54.601404  2301 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.015s	sys 0.021s
W20260709 13:01:54.553422  3623 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.006s	sys 0.009s
W20260709 13:01:54.573125  1684 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.065s	user 0.020s	sys 0.020s
I20260709 13:01:54.436167  3743 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:54.520126  3728 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.833s	user 0.001s	sys 0.005s
W20260709 13:01:54.582823  3736 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.871s	user 0.021s	sys 0.025s
W20260709 13:01:54.536031  1635 rpcz_store.cc:269] 0709 13:01:52.950403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.811288 (+860885us) service_pool.cc:224] Handling call
0709 13:01:54.389021 (+577733us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:54.600818  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.139s	user 0.018s	sys 0.028s
I20260709 13:01:54.558647  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 388) took 1536 ms. Trace:
I20260709 13:01:54.662488  1640 rpcz_store.cc:276] 0709 13:01:53.021976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.490231 (+1468255us) service_pool.cc:224] Handling call
0709 13:01:54.490605 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.558280 (+ 67675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.558572 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.533567  3394 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:54.618494  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1769) took 2218 ms. Trace:
I20260709 13:01:54.465391  1560 rpcz_store.cc:276] 0709 13:01:52.252500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.951027 (+1698527us) service_pool.cc:224] Handling call
0709 13:01:54.431157 (+480130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.464635 (+ 33478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.464868 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.539170  3684 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:54.546805  3621 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:54.563576  3752 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.019s	sys 0.014s
I20260709 13:01:54.639146  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 538) took 1625 ms. Trace:
I20260709 13:01:54.669309  2301 rpcz_store.cc:276] 0709 13:01:53.013838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.733759 (+719921us) service_pool.cc:224] Handling call
0709 13:01:53.734194 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.734469 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:53.735358 (+   889us) log.cc:844] Serialized 14 byte log entry
0709 13:01:54.638717 (+903359us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.638781 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:54.638825 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.638902 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.639074 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21951,"prepare.queue_time_us":299,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":22930,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":17346,"thread_start_us":866064,"threads_started":1}
I20260709 13:01:54.673327   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:54.578286  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1607) took 2240 ms. Trace:
I20260709 13:01:54.675725  1639 rpcz_store.cc:276] 0709 13:01:52.321232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.947059 (+1625827us) service_pool.cc:224] Handling call
0709 13:01:54.561723 (+614664us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.676946  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:54.677685  1639 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:54.561573  3717 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:54.640126  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:54.679878  3717 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:54.680011  2269 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:54.680384  3717 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:54.680780  3717 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:54.635615  3691 raft_consensus.cc:3055] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:54.681568  3691 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:54.682428  3691 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:54.678808  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:54.683476  3691 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:54.581575  3348 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.538735  2298 rpcz_store.cc:276] 0709 13:01:53.080521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.547102 (+466581us) service_pool.cc:224] Handling call
0709 13:01:54.392105 (+845003us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:54.651991  3728 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.001s	sys 0.006s
I20260709 13:01:54.690224  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2094) took 2435 ms. Trace:
I20260709 13:01:54.664021  1565 rpcz_store.cc:276] 0709 13:01:52.400971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.538314 (+2137343us) service_pool.cc:224] Handling call
0709 13:01:54.581275 (+ 42961us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.617996 (+ 36721us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:54.618061 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.618123 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:54.618167 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.618265 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.618422 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15189,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":3939}
W20260709 13:01:54.662438  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.201s	user 0.019s	sys 0.028s
I20260709 13:01:54.668753  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:54.692106  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 1.24856e+06us
I20260709 13:01:54.692868  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:54.693007  3684 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:54.693825  3684 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:54.637229  3701 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:54.696851  3684 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:54.684486  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:54.616829  3757 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:54.562973  3615 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:54.698552  1639 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
W20260709 13:01:54.652367  3732 consensus_meta.cc:287] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.891s	user 0.005s	sys 0.009s
W20260709 13:01:54.656934  3736 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.022s	sys 0.025s
W20260709 13:01:54.645116  3623 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.008s	sys 0.009s
I20260709 13:01:54.690793  1568 rpcz_store.cc:276] 0709 13:01:52.276137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.223425 (+1947288us) service_pool.cc:224] Handling call
0709 13:01:54.223814 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.456338 (+232524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.689863 (+233525us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b5400216e54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:54.690139 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488377216}
I20260709 13:01:54.718873  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.706861  3757 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:54.545532  3592 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:54.631608  3593 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:54.633927  3637 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:54.766530  2269 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.598s (SENT) [suppressed 20 similar messages]
I20260709 13:01:54.768793  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:54.771872  1709 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:54.757391  1681 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.281s	user 0.030s	sys 0.026s
I20260709 13:01:54.770339  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:54.776631  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:54.777420  1639 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:54.778084  1563 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 6: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:01:54.778477  1681 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.302s	user 0.031s	sys 0.026s
I20260709 13:01:54.778287  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:54.779385  1639 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:54.780599  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:54.704048  3471 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:54.689603  3640 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:54.782406  3743 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.797397  3743 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:54.798770  3593 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:54.799139  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:54.800185  3743 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:54.582693  2304 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.000s	sys 0.015s
I20260709 13:01:54.787127  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:54.802240  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:54.802801  2304 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.047s	user 0.000s	sys 0.015s
W20260709 13:01:54.778772  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 1.3022e+06us
I20260709 13:01:54.806900  1709 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:01:54.807971  1683 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:54.721657  3764 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:54.820144  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.820904  1712 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:01:54.820525  3764 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:54.823503  3718 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:54.823853  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.825019  3615 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:54.825570  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.826463  1710 raft_consensus.cc:2393] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:53.899565  2299 rpcz_store.cc:276] 0709 13:01:51.687106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.697891 (+ 10785us) service_pool.cc:224] Handling call
0709 13:01:51.706088 (+  8197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:51.706346 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:53.071094 (+1364748us) log.cc:844] Serialized 14 byte log entry
0709 13:01:53.483878 (+412784us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:53.500510 (+ 16632us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:53.503505 (+  2995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.509765 (+  6260us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.545952 (+ 36187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1628,"prepare.queue_time_us":556789,"prepare.run_cpu_time_us":3026,"prepare.run_wall_time_us":18329,"raft.queue_time_us":316449,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":159,"thread_start_us":1638012,"threads_started":3,"wal-append.queue_time_us":830258}
I20260709 13:01:54.826690  3661 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:54.827952  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.828548  3602 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:54.828540  1710 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:54.829138  3637 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:54.830302  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:54.786547  3471 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.830832  1710 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:54.831137  3471 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:54.835279  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.835728  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.785920  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 726) took 2948 ms. Trace:
I20260709 13:01:54.837283  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.838999  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.837226  2300 rpcz_store.cc:276] 0709 13:01:51.834606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.080090 (+245484us) service_pool.cc:224] Handling call
0709 13:01:52.080445 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:52.080793 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:52.080837 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:52.800348 (+719511us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b2c00047a38. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:01:52.800946 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 13:01:53.891775 (+1090829us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:53.901362 (+  9587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.968200 (+ 66838us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:54.689385 (+721185us) raft_consensus.cc:1685] finished
0709 13:01:54.697854 (+  8469us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.782866 (+ 85012us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":290352,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":52035,"spinlock_wait_cycles":93668736,"thread_start_us":1052020,"threads_started":1,"wal-append.queue_time_us":1409141}
I20260709 13:01:54.839534  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1762) took 2588 ms. Trace:
I20260709 13:01:54.839923  1567 rpcz_store.cc:276] 0709 13:01:52.251139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.806311 (+1555172us) service_pool.cc:224] Handling call
0709 13:01:54.206465 (+400154us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:01:54.206655 (+   190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.457262 (+250607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.839307 (+382045us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54002986d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:54.839468 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557875328}
I20260709 13:01:54.840376  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:54.841054  3739 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:54.753321  3732 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:54.842738  3693 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:54.843344  1718 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:54.844617  3732 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:54.844748  2305 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 540) took 1773 ms (client timeout 1590 ms). Trace:
W20260709 13:01:54.845328  2305 rpcz_store.cc:269] 0709 13:01:53.071562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.303983 (+232421us) service_pool.cc:224] Handling call
0709 13:01:54.844544 (+1540561us) spinlock_profiling.cc:243] Waited 9.13 ms on lock 0x7b540059b5d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:54.844673 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20999040}
I20260709 13:01:54.845783  3749 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:54.702369  3348 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:54.848518  3757 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:54.848860  3348 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:54.850085  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:54.785672  3712 consensus_queue.cc:1175] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5184c101be204393adf3ac8b31f25ad6" responder_term: 4 status { last_received { term: 4 index: 6 } last_committed_idx: 5 last_received_current_leader { term: 4 index: 6 } }
W20260709 13:01:54.860173  1609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.139s	user 0.020s	sys 0.046s
W20260709 13:01:54.862288  1609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.142s	user 0.020s	sys 0.046s
I20260709 13:01:54.862593  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:54.863492  1639 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:54.864678  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:54.864862  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 1.14432e+06us
I20260709 13:01:54.865278  1639 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:54.866389  1639 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:54.867715  2305 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:01:54.867822  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.871302  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.871784  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.872731  1714 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 2.
I20260709 13:01:54.874508  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:54.875341  1714 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:54.877133  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.877828  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.878248  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:54.878559  1716 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:54.878861  2305 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:01:54.880968  1644 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.033s	user 0.007s	sys 0.006s
I20260709 13:01:54.882063  1644 raft_consensus.cc:2410] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:01:54.884974  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1611) took 2515 ms (client timeout 2152 ms). Trace:
W20260709 13:01:54.857571  3541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.015s	sys 0.023s
W20260709 13:01:54.885433  1644 rpcz_store.cc:269] 0709 13:01:52.369883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.738902 (+1369019us) service_pool.cc:224] Handling call
0709 13:01:54.884899 (+1145997us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:54.851052  1711 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
W20260709 13:01:54.889326  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 85669.2us
W20260709 13:01:54.901423  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.889645  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.892168  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:54.899129  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 91283.8us
W20260709 13:01:54.914234  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.184s	user 0.010s	sys 0.036s
W20260709 13:01:54.915160  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.185s	user 0.011s	sys 0.036s
I20260709 13:01:54.915825  1710 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
W20260709 13:01:54.916757  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 1.18719e+06us
I20260709 13:01:54.918701  2268 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:01:54.881830  3352 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.022s	sys 0.043s
I20260709 13:01:54.890208  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:54.943017  1639 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:01:54.944465  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:01:54.889770  3607 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.013s	sys 0.029s
I20260709 13:01:54.950372  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:54.956519  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:54.882481  3669 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.071s	user 0.037s	sys 0.031s
I20260709 13:01:54.963400  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1759) took 2713 ms. Trace:
W20260709 13:01:54.889703  3541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.985s	user 0.016s	sys 0.024s
I20260709 13:01:54.861435  3592 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:54.941962  2268 leader_election.cc:400] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 3. Current term is 4.
I20260709 13:01:54.889588  1532 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:54.882578  3593 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.962208  1533 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:54.939548  1714 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
W20260709 13:01:54.972740  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 83238.4us
I20260709 13:01:54.972787  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:54.973858  1644 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:54.965929  1681 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:01:54.942849  3352 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.996s	user 0.022s	sys 0.044s
I20260709 13:01:54.969358  2268 leader_election.cc:403] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:01:54.969738  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:54.889305  3756 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.941488  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:54.889549  3764 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:54.956349  2296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.153s	user 0.008s	sys 0.060s
I20260709 13:01:54.978116  1681 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:55.042536  1639 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:55.025771  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 63655.1us
I20260709 13:01:54.963930  1564 rpcz_store.cc:276] 0709 13:01:52.250289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.794701 (+1544412us) service_pool.cc:224] Handling call
0709 13:01:54.187640 (+392939us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ceee 0000000000375939 00007fd7a6bf01e0 00007fd7a6cb90c4 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:01:54.187863 (+   223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.469966 (+282103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.963035 (+493069us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54003e7454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:54.963292 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551967872}
I20260709 13:01:55.043828  3541 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 3. Current term is 4.
W20260709 13:01:55.044251  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 103072us
W20260709 13:01:54.964886  3669 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.151s	user 0.037s	sys 0.031s
I20260709 13:01:54.959560  3701 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:01:54.962462  3607 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.013s	sys 0.029s
W20260709 13:01:55.036550  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.949218  3693 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:54.990415  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.938201  2305 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:55.057796  3758 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:54.942625  3732 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:01:54.968379  3746 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.470s	user 0.001s	sys 0.001s
I20260709 13:01:54.901779  3615 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.090s
I20260709 13:01:54.959741  3567 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.109s
W20260709 13:01:55.370230  1607 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.649s	user 0.014s	sys 0.020s
W20260709 13:01:55.821286  1607 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 2.100s	user 0.021s	sys 0.031s
W20260709 13:01:55.326180  1638 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.015s	sys 0.013s
W20260709 13:01:55.821909  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 2.10076e+06us
I20260709 13:01:55.072790  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:01:55.824488  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 751833us
I20260709 13:01:55.046538  3637 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:01:55.230268  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1608) took 2897 ms. Trace:
W20260709 13:01:55.507174  3585 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.001s	sys 0.022s
I20260709 13:01:55.827647  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:55.828779  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:55.830024  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:55.877584  1561 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:55.879387  1568 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:55.880244  1564 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 4: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:55.027194  3665 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.803s
W20260709 13:01:55.209154  3717 consensus_meta.cc:287] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.003s
I20260709 13:01:55.204941  1639 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:55.831224  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:55.831501  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:55.832806  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:55.886284  1567 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:01:55.887022  1565 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:55.886251  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:55.179695  1534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.302s	user 0.023s	sys 0.022s
W20260709 13:01:55.890379  1534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.013s	user 0.027s	sys 0.031s
W20260709 13:01:55.891170  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 2.16773e+06us
W20260709 13:01:55.231020  1643 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.021s	sys 0.062s
W20260709 13:01:55.892280  1643 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.477s	user 0.023s	sys 0.074s
W20260709 13:01:55.834532  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.210099  1711 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:01:55.058377  3736 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.778s
W20260709 13:01:55.822590  1638 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.277s	user 0.024s	sys 0.013s
W20260709 13:01:55.206293  3667 consensus_meta.cc:287] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.709s	user 0.010s	sys 0.000s
W20260709 13:01:55.204753  3449 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.028s	sys 0.049s
I20260709 13:01:55.850726  3701 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:55.852209  1682 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:01:55.850795  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2081) took 3633 ms. Trace:
W20260709 13:01:55.096330  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 118302us
I20260709 13:01:55.858218  1642 rpcz_store.cc:276] 0709 13:01:52.332840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.243897 (+1911057us) service_pool.cc:224] Handling call
0709 13:01:54.388707 (+144810us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.536369 (+147662us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.536411 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.536474 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:54.536520 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.230008 (+693488us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b5400247554. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:55.230062 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.230202 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339059072}
I20260709 13:01:55.956228  1681 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:01:55.875537  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 52328.3us
I20260709 13:01:55.876142  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:55.824980  3693 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:01:55.857414  3637 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:55.868544  3640 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:01:55.896621  3717 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:55.960760  3760 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:55.965788  3637 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:01:55.941458  3741 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:55.961122  3693 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:55.968269  3717 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:55.881125  3585 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.296s	user 0.009s	sys 0.033s
I20260709 13:01:55.900041  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:55.969751  1639 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:55.970710  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:55.971151  3508 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:55.965813  3538 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:55.111436  3764 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:55.138556  3623 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:55.902462  1563 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2084) took 3683 ms (client timeout 1640 ms). Trace:
W20260709 13:01:55.138105  3589 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.027s	sys 0.045s
I20260709 13:01:55.908794  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1555) took 1080 ms. Trace:
W20260709 13:01:55.068114  2296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.264s	user 0.009s	sys 0.061s
I20260709 13:01:55.986342  1718 rpcz_store.cc:276] 0709 13:01:54.798678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.798813 (+   135us) service_pool.cc:224] Handling call
0709 13:01:55.879160 (+1080347us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b54001356d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:55.879290 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99731200}
W20260709 13:01:55.946087  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 54082.6us
I20260709 13:01:55.849797  3778 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:55.952493  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 100411us
I20260709 13:01:55.960047  2305 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:55.971343  1639 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:55.990237  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 744) took 1028 ms. Trace:
I20260709 13:01:55.990315  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:55.990623  2305 rpcz_store.cc:276] 0709 13:01:54.962407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.875788 (+913381us) service_pool.cc:224] Handling call
0709 13:01:55.990176 (+114388us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:55.991722  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:55.992604  2305 raft_consensus.cc:2393] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:01:55.992977  2269 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.133s	user 0.011s	sys 0.015s
W20260709 13:01:55.993311  2269 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.133s	user 0.011s	sys 0.015s
W20260709 13:01:55.993587  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 1.22718e+06us
W20260709 13:01:55.179495  3571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.633s	user 0.030s	sys 0.050s
I20260709 13:01:55.230405  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:55.869585  3709 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:55.985683  1532 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.958s	user 0.007s	sys 0.020s
W20260709 13:01:56.017290  1532 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.008s	sys 0.021s
W20260709 13:01:55.976460  3471 consensus_meta.cc:287] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.145s	user 0.008s	sys 0.000s
W20260709 13:01:55.979274  1563 rpcz_store.cc:269] 0709 13:01:52.218650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.248667 (+ 30017us) service_pool.cc:224] Handling call
0709 13:01:55.902138 (+3653471us) spinlock_profiling.cc:243] Waited 2.35 s on lock 0x7b54000a78d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:55.902327 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5396631936}
I20260709 13:01:55.205307  3765 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.845s
W20260709 13:01:55.950477  3449 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.486s	user 0.030s	sys 0.060s
W20260709 13:01:55.997750  3571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.460s	user 0.030s	sys 0.063s
I20260709 13:01:56.000344  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 547) took 1108 ms. Trace:
W20260709 13:01:56.003997  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.971678  3508 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:01:56.017795  1641 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.521s	user 0.058s	sys 0.112s
W20260709 13:01:56.116019  1641 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.619s	user 0.058s	sys 0.112s
W20260709 13:01:56.017843  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 1.027e+06us
I20260709 13:01:55.966836  3774 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.021888  1639 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:56.120111  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2086) took 1130 ms. Trace:
I20260709 13:01:56.120524  1639 rpcz_store.cc:276] 0709 13:01:54.989701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.970428 (+980727us) service_pool.cc:224] Handling call
0709 13:01:56.120023 (+149595us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:56.022279  1683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.009s	sys 0.025s
I20260709 13:01:55.973659  3762 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:56.090129  3666 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:01:56.119766  3508 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:56.128165  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:56.128981  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 549) took 1083 ms. Trace:
I20260709 13:01:56.129294  2301 rpcz_store.cc:276] 0709 13:01:55.045973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.127854 (+1081881us) service_pool.cc:224] Handling call
0709 13:01:56.128911 (+  1057us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:56.130261  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:56.131069  3691 consensus_meta.cc:287] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.447s	user 0.009s	sys 0.002s
W20260709 13:01:56.131271  3780 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:01:56.131954  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:56.132714  1560 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:56.133747  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:56.134069  3693 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:56.020388  3623 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:55.953648  1562 rpcz_store.cc:276] 0709 13:01:52.217530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.263365 (+ 45835us) service_pool.cc:224] Handling call
0709 13:01:52.398463 (+135098us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.220989 (+1822526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.850448 (+1629459us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54000f6254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:55.850702 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3017757056}
I20260709 13:01:56.134759  3508 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:56.021526  3778 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:56.033108  3471 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:55.893095  3732 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:55.986259  3589 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.473s	user 0.033s	sys 0.064s
I20260709 13:01:55.909507  3712 raft_consensus.cc:3055] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:55.944540  3669 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:01:55.979415  3764 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:56.032241  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 50114.3us
W20260709 13:01:56.032722  1640 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.181s	user 0.017s	sys 0.011s
W20260709 13:01:56.189082  1640 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.338s	user 0.017s	sys 0.011s
W20260709 13:01:56.126300  1637 consensus_meta.cc:287] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 2.184s	user 0.014s	sys 0.000s
W20260709 13:01:55.994896  3763 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.168s	user 0.012s	sys 0.014s
W20260709 13:01:56.134884  3592 consensus_meta.cc:287] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.146s	user 0.005s	sys 0.005s
W20260709 13:01:56.029059  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.071426  3756 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:56.035256  2305 rpcz_store.cc:276] 0709 13:01:54.892112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.991473 (+1099361us) service_pool.cc:224] Handling call
0709 13:01:56.000156 (+  8683us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:56.056842  3585 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:56.247390  1712 raft_consensus.cc:2410] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:01:56.034417  2298 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.923s	user 0.018s	sys 0.038s
I20260709 13:01:56.147296  3541 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.152484  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:56.034159  3765 consensus_queue.cc:1243] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:01:56.113328  1607 connection.cc:256] RPC call timeout handler was delayed by -2.3568951009999637s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2868s for a response
W20260709 13:01:56.382874  1607 connection.cc:256] RPC call timeout handler was delayed by -2.2826988189999611s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28940000000000005s for a response
W20260709 13:01:55.999495  2297 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.153s	user 0.017s	sys 0.034s
I20260709 13:01:56.115715  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:56.383443  1607 connection.cc:256] RPC call timeout handler was delayed by -2.2826988189999611s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28940000000000005s for a response
I20260709 13:01:56.383951  1566 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:56.116555  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:56.117826  1644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.989s	user 0.004s	sys 0.067s
I20260709 13:01:56.022784  2299 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:55.046319  3593 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:56.122196  1683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.075s	user 0.009s	sys 0.025s
I20260709 13:01:56.126247  1532 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:56.112932  3567 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:01:56.151186  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:56.403786  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:01:56.018800  1534 connection.cc:256] RPC call timeout handler was delayed by -2.8784168760000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
W20260709 13:01:56.406149  1534 connection.cc:256] RPC call timeout handler was delayed by -2.8730168760000083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16060000000000002s for a response
I20260709 13:01:56.146814  3471 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:56.406689  1534 connection.cc:256] RPC call timeout handler was delayed by -2.84511687600002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
W20260709 13:01:56.407219  1534 connection.cc:256] RPC call timeout handler was delayed by -2.3267168760000345s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
W20260709 13:01:56.407714  1534 connection.cc:256] RPC call timeout handler was delayed by -2.1449168760000248s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24150000000000002s for a response
I20260709 13:01:56.149116  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 730) took 3083 ms. Trace:
I20260709 13:01:56.147210  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:56.024030  3753 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.202s	user 0.038s	sys 0.089s
I20260709 13:01:56.155781  2301 raft_consensus.cc:2393] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:56.212594  3691 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:56.143030  3784 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 13:01:56.190644  1635 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.354s	user 0.004s	sys 0.028s
W20260709 13:01:56.156083  3662 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.355s	user 0.000s	sys 0.028s
I20260709 13:01:56.136389  3623 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:55.989059  3665 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.099s
I20260709 13:01:56.149948  3762 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:56.203192  1636 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.112s	user 0.003s	sys 0.035s
I20260709 13:01:56.261782  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:56.380403  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 134605us
W20260709 13:01:56.385465  1644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.257s	user 0.005s	sys 0.067s
I20260709 13:01:56.385185  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2092) took 1157 ms. Trace:
I20260709 13:01:56.246299  2269 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:01:56.386092  1607 connection.cc:256] RPC call timeout handler was delayed by -1.8156576039999663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
I20260709 13:01:56.144758  3778 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:56.136385  3693 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:56.153944  3712 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:01:56.212107  3763 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.382s	user 0.012s	sys 0.014s
W20260709 13:01:56.389472  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 263336us
I20260709 13:01:56.388067  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:56.199314  3764 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:56.411386  2304 rpcz_store.cc:276] 0709 13:01:53.066223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.743714 (+677491us) service_pool.cc:224] Handling call
0709 13:01:53.743997 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.827064 (+1083067us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.827096 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:54.827140 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:54.827176 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.148808 (+1321632us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b5400406854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:56.148877 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.149033 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31965,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":151,"spinlock_wait_cycles":25600896,"thread_start_us":1047761,"threads_started":1}
W20260709 13:01:56.260118  3352 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.145s	user 0.018s	sys 0.022s
I20260709 13:01:56.148625  3637 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:56.411648  1563 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:56.143978  3508 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:56.213327  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 733) took 2426 ms. Trace:
I20260709 13:01:56.209272  3736 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.209354  3666 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.261920  3585 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:56.388972  2297 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.543s	user 0.019s	sys 0.036s
W20260709 13:01:56.387583  2298 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.297s	user 0.027s	sys 0.048s
I20260709 13:01:56.416013  3541 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:56.149310  3774 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:56.441323  3593 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:56.442807  3662 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.646s	user 0.000s	sys 0.029s
I20260709 13:01:56.378242  3709 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.126376  3701 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:56.464951  1642 rpcz_store.cc:276] 0709 13:01:55.228114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.955144 (+727030us) service_pool.cc:224] Handling call
0709 13:01:56.385128 (+429984us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:56.465278  1607 connection.cc:256] RPC call timeout handler was delayed by -1.8156576039999663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
I20260709 13:01:56.466104  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 548) took 1565 ms. Trace:
W20260709 13:01:56.466763  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 220566us
I20260709 13:01:56.407831  1637 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:01:56.418004  3753 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.596s	user 0.038s	sys 0.090s
W20260709 13:01:56.388343  2299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 552) took 1330 ms (client timeout 1602 ms). Trace:
I20260709 13:01:56.471577  1532 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:01:56.426107  1635 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.589s	user 0.004s	sys 0.028s
I20260709 13:01:56.399402  1560 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:01:56.471443  1566 raft_consensus.cc:2376] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3. Re-sending same reply.
I20260709 13:01:56.388875  3589 consensus_queue.cc:1059] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:01:56.517338  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:56.484254  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.519332  2305 rpcz_store.cc:276] 0709 13:01:54.901227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.261437 (+1360210us) service_pool.cc:224] Handling call
0709 13:01:56.466028 (+204591us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:56.552083  1563 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 6: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:01:56.552625  1609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.026s	sys 0.034s
I20260709 13:01:56.444193  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:01:56.519057  1607 connection.cc:256] RPC call timeout handler was delayed by -1.7446270049999839s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29910000000000003s for a response
I20260709 13:01:56.516491  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 397) took 1739 ms. Trace:
I20260709 13:01:56.528491  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2072) took 1827 ms. Trace:
W20260709 13:01:56.557410  2270 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.022s	sys 0.040s
W20260709 13:01:56.558018  2270 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.022s	sys 0.040s
I20260709 13:01:56.557557  1640 rpcz_store.cc:276] 0709 13:01:54.700996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.701589 (+   593us) service_pool.cc:224] Handling call
0709 13:01:54.772142 (+ 70553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.772734 (+   592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.772780 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.783194 (+ 10414us) log.cc:844] Serialized 26 byte log entry
0709 13:01:56.189479 (+1406285us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:56.189518 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.189586 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.528094 (+338508us) raft_consensus.cc:1685] finished
0709 13:01:56.528128 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.528420 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":170,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":7495,"thread_start_us":1338127,"threads_started":1,"wal-append.queue_time_us":1591769}
I20260709 13:01:56.516844  3637 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:56.532948  3712 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:56.536597  3774 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:56.539284  3736 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:56.530248  2299 rpcz_store.cc:269] 0709 13:01:55.058208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.229991 (+171783us) service_pool.cc:224] Handling call
0709 13:01:56.388278 (+1158287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:56.540916  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:01:56.469378  1636 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.398s	user 0.003s	sys 0.037s
I20260709 13:01:56.453817  3691 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:56.470402  3538 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:56.542646  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:56.520787  2296 rpcz_store.cc:276] 0709 13:01:53.783421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.793320 (+  9899us) service_pool.cc:224] Handling call
0709 13:01:53.794001 (+   681us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:53.802060 (+  8059us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:53.802105 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:53.802765 (+   660us) log.cc:844] Serialized 26 byte log entry
0709 13:01:56.021450 (+2218685us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:56.030942 (+  9492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.057379 (+ 26437us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.152423 (+ 95044us) raft_consensus.cc:1685] finished
0709 13:01:56.152455 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.210021 (+ 57566us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7536,"raft.queue_time_us":12574,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":2184375,"threads_started":1,"wal-append.queue_time_us":2179781}
I20260709 13:01:56.544020  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:56.524484  3585 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:56.544533  3709 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:56.545387  1712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 422) took 1705 ms (client timeout 1598 ms). Trace:
I20260709 13:01:56.545912  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1594) took 4237 ms. Trace:
I20260709 13:01:56.546382  3471 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:56.552842  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:56.513904  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:56.590672  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 554) took 1384 ms. Trace:
I20260709 13:01:56.590801  2304 raft_consensus.cc:2410] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:01:56.553843  1609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.026s	sys 0.034s
I20260709 13:01:56.591087  2305 rpcz_store.cc:276] 0709 13:01:55.205877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.552518 (+1346641us) service_pool.cc:224] Handling call
0709 13:01:56.590610 (+ 38092us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:56.549993  2269 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:56.592387  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 555) took 1335 ms. Trace:
I20260709 13:01:56.467687  3762 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:56.474239  3795 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:56.556735  1607 connection.cc:256] RPC call timeout handler was delayed by -1.7446270049999839s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29910000000000003s for a response
I20260709 13:01:56.556833  1644 rpcz_store.cc:276] 0709 13:01:54.776736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.974765 (+198029us) service_pool.cc:224] Handling call
0709 13:01:55.044579 (+ 69814us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:55.044964 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:55.045004 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:55.045501 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 13:01:56.464748 (+1419247us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:56.464787 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.464853 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.516270 (+ 51417us) raft_consensus.cc:1685] finished
0709 13:01:56.516297 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.516432 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":899,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":4426,"thread_start_us":1335718,"threads_started":1,"wal-append.queue_time_us":1417033}
I20260709 13:01:56.558943  1683 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:01:56.594595  1607 connection.cc:256] RPC call timeout handler was delayed by -1.6005576039999596s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2028s for a response
W20260709 13:01:56.595853  1607 connection.cc:256] RPC call timeout handler was delayed by -1.4277576039999644s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22200000000000003s for a response
W20260709 13:01:56.595999  2302 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.014s	sys 0.023s
W20260709 13:01:56.596335  2302 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.014s	sys 0.023s
W20260709 13:01:56.512995  3352 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.417s	user 0.035s	sys 0.039s
W20260709 13:01:56.596890  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 47029.5us
I20260709 13:01:56.597926  2269 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:56.534380  3666 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:56.548475  3607 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.019s	sys 0.040s
I20260709 13:01:56.573589  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:56.603055  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:56.603631  1563 raft_consensus.cc:2376] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3. Re-sending same reply.
I20260709 13:01:56.574174  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:56.574447  1681 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:56.604665  2303 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:01:56.586409  3538 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:56.581470  1568 raft_consensus.cc:2376] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3. Re-sending same reply.
I20260709 13:01:56.562206  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:56.564476  3765 raft_consensus.cc:3055] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:01:56.566620  3637 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:01:56.587988  1712 rpcz_store.cc:269] 0709 13:01:54.840074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.840183 (+   109us) service_pool.cc:224] Handling call
0709 13:01:56.545147 (+1704964us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b540033a9d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:56.545307 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3233859840}
I20260709 13:01:56.565881  3589 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:01:56.593127  1643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.014s	sys 0.023s
I20260709 13:01:56.565761  1562 raft_consensus.cc:2376] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3. Re-sending same reply.
I20260709 13:01:56.592813  2304 rpcz_store.cc:276] 0709 13:01:55.256899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.471802 (+1214903us) service_pool.cc:224] Handling call
0709 13:01:56.592301 (+120499us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:56.611328  1643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.015s	sys 0.023s
I20260709 13:01:56.612514  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:56.594154  1533 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.003s	sys 0.034s
I20260709 13:01:56.573695  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 384) took 3599 ms. Trace:
W20260709 13:01:56.596267  1607 connection.cc:256] RPC call timeout handler was delayed by -1.0911576039999318s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2594s for a response
W20260709 13:01:56.604372  3607 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.020s	sys 0.040s
I20260709 13:01:56.589331  1637 rpcz_store.cc:276] 0709 13:01:52.308843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.314729 (+  5886us) service_pool.cc:224] Handling call
0709 13:01:53.809710 (+1494981us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ceee 0000000000375939 00007fd7a6bf01e0 00007fd7a6cb90c4 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:01:53.809872 (+   162us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.528679 (+2718807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.545842 (+ 17163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22992256}
W20260709 13:01:56.624059  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.028s	sys 0.042s
W20260709 13:01:56.624428  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.028s	sys 0.042s
I20260709 13:01:56.604737  3665 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:01:56.606209  2303 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:56.626889  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 745) took 1636 ms. Trace:
I20260709 13:01:56.624648  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:56.628760  1716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.569s	user 0.024s	sys 0.079s
I20260709 13:01:56.608743  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:56.627502  2303 rpcz_store.cc:276] 0709 13:01:54.990170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.605514 (+1615344us) service_pool.cc:224] Handling call
0709 13:01:56.605798 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.626528 (+ 20730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.626800 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:56.630718  1568 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:56.584129  1715 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:56.613638  1533 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.004s	sys 0.034s
I20260709 13:01:56.632133  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:56.615103  1607 connection.cc:256] RPC call timeout handler was delayed by -1.0911576039999318s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2594s for a response
W20260709 13:01:56.632483  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 673357us
W20260709 13:01:56.633085  1607 connection.cc:256] RPC call timeout handler was delayed by -1.0794576039999697s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26070000000000004s for a response
I20260709 13:01:56.599557  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:56.597023  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:01:56.637916  1533 leader_election.cc:336] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.570s (SENT)
W20260709 13:01:56.624712  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 799156us
I20260709 13:01:56.613862  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 385) took 3634 ms. Trace:
W20260709 13:01:56.562654  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.640528  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:56.614482  3637 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.616737  3592 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:56.614646  3352 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:56.642455  1643 rpcz_store.cc:276] 0709 13:01:52.979195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.414222 (+1435027us) service_pool.cc:224] Handling call
0709 13:01:54.414529 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.414771 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:55.892861 (+1478090us) log.cc:844] Serialized 14 byte log entry
0709 13:01:56.613508 (+720647us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:56.613561 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:56.613612 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.613685 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.613808 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1489,"prepare.queue_time_us":1673708,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"thread_start_us":2196188,"threads_started":2}
I20260709 13:01:56.644572  3670 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:56.625924  3636 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.510s	user 0.016s	sys 0.055s
W20260709 13:01:56.635083  1716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.575s	user 0.024s	sys 0.079s
I20260709 13:01:56.646209  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:56.623756  3589 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:56.616542  3732 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:56.626579  3756 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:56.650990  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:56.653383  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:56.639765  3693 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:56.637355  3774 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.654246  1715 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
W20260709 13:01:56.647544  3787 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:01:56.644030  2299 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:01:56.645279  3637 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:56.658474  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.010s	sys 0.014s
W20260709 13:01:56.703583  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.010s	sys 0.014s
W20260709 13:01:56.703919  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 671288us
I20260709 13:01:56.704267  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:56.705075  1563 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 4: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:56.705852  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:56.659235  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:56.610462  3765 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:01:56.646209  1533 leader_election.cc:336] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.809s (SENT)
I20260709 13:01:56.643834  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2071) took 1943 ms. Trace:
I20260709 13:01:56.668380  1568 raft_consensus.cc:2410] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:01:56.649111  3665 raft_consensus.cc:697] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:01:56.659292  3786 log.cc:927] Time spent T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 13:01:56.652148  3592 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
W20260709 13:01:56.662748  3636 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.551s	user 0.017s	sys 0.055s
W20260709 13:01:56.708729  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 62582.4us
I20260709 13:01:56.614765  3795 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:56.712805  1682 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:56.662757  3670 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:01:56.712846  3766 log.cc:927] Time spent T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
W20260709 13:01:56.701903  2300 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.016s	sys 0.028s
W20260709 13:01:56.758499  2300 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.017s	sys 0.029s
I20260709 13:01:56.760262  3665 consensus_queue.cc:237] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:56.716267  3732 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:56.763283  3732 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.763727  3732 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:56.765722  3763 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:56.766170  3763 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:56.714172  3756 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:01:56.757139  3670 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:56.767094  3763 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:56.768316  3763 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:56.703081  3774 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:56.771099  3774 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:56.771965  3800 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:01:56.772163  3768 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 1.516s	user 0.004s	sys 0.009s
W20260709 13:01:56.772727  3801 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:01:56.774530  3795 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:56.671181  3693 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:56.675211  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:56.754187  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:56.783432  1568 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 4: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:01:56.659022  3538 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:56.755661  3790 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:01:56.775027  3712 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:56.775581  1637 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:56.756233  3691 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:56.823486  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:56.614810  1636 rpcz_store.cc:276] 0709 13:01:52.973922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.401109 (+1427187us) service_pool.cc:224] Handling call
0709 13:01:54.464836 (+ 63727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.465078 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:55.023476 (+558398us) log.cc:844] Serialized 14 byte log entry
0709 13:01:56.573319 (+1549843us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:56.573386 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:56.573430 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.573495 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.573634 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1663549,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":22396,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":6755,"thread_start_us":1997666,"threads_started":2,"wal-append.queue_time_us":1508550}
I20260709 13:01:56.775607  3765 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:56.723081  1641 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.004s	sys 0.037s
W20260709 13:01:56.799319  1533 leader_election.cc:336] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.637s (SENT)
W20260709 13:01:56.835126  1533 leader_election.cc:336] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.724s (SENT)
I20260709 13:01:56.835726  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:56.836638  1641 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.004s	sys 0.037s
I20260709 13:01:56.838182  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1599) took 4527 ms. Trace:
W20260709 13:01:56.842749  1533 leader_election.cc:336] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.735s (SENT)
W20260709 13:01:56.843875  1533 leader_election.cc:336] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.825s (SENT)
W20260709 13:01:56.844812  1533 leader_election.cc:336] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.876s (SENT)
I20260709 13:01:56.845146  1533 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:56.804730  1562 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:56.812752  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:56.813004  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:56.877040  1713 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:56.813691  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:56.838778  1641 rpcz_store.cc:276] 0709 13:01:52.310363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.224988 (+1914625us) service_pool.cc:224] Handling call
0709 13:01:54.226072 (+  1084us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.226757 (+   685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.226806 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.379791 (+152985us) log.cc:844] Serialized 26 byte log entry
0709 13:01:56.116539 (+1736748us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:56.837287 (+720748us) log.cc:844] Serialized 14 byte log entry
0709 13:01:56.837753 (+   466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.837879 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.837924 (+    45us) raft_consensus.cc:1685] finished
0709 13:01:56.837936 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.838109 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7686,"prepare.queue_time_us":12240,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":28,"raft.queue_time_us":107,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":152975,"thread_start_us":2330838,"threads_started":2,"wal-append.queue_time_us":1971495}
I20260709 13:01:56.815112  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:56.756829  3796 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:56.793838  3693 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:56.833648  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 387) took 3817 ms. Trace:
I20260709 13:01:56.881925  1638 rpcz_store.cc:276] 0709 13:01:53.015774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.451801 (+1436027us) service_pool.cc:224] Handling call
0709 13:01:54.452243 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.452733 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.452775 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.471368 (+ 18593us) log.cc:844] Serialized 26 byte log entry
0709 13:01:55.906882 (+1435514us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:55.906939 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.907004 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.833268 (+926264us) raft_consensus.cc:1685] finished
0709 13:01:56.833300 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.833562 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11083,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":124,"thread_start_us":1360826,"threads_started":1,"wal-append.queue_time_us":1746840}
W20260709 13:01:56.658859  3449 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.009s	sys 0.012s
W20260709 13:01:56.874619  1533 leader_election.cc:336] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.894s (SENT)
I20260709 13:01:56.817368  3736 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:56.852973  3795 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:01:56.884430  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.884899  1533 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:01:56.955973  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 81403.2us
W20260709 13:01:56.884805  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 413359us
I20260709 13:01:56.893302  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:57.034340  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 75884.7us
W20260709 13:01:57.035403  1533 leader_election.cc:336] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 2.033s (SENT)
I20260709 13:01:57.036144  1533 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:56.941022  1710 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:57.040283  1713 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
W20260709 13:01:57.042137  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.043848  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:56.889056  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:56.886955  3449 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.010s	sys 0.012s
I20260709 13:01:57.047789  1532 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:57.080982  1532 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:01:56.949617  3797 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:01:56.889132  3640 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.086259  3589 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:56.886052  3743 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:01:57.062906  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.929054  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:57.082557  3809 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.228s	user 0.000s	sys 0.000s
I20260709 13:01:57.063040  3637 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:57.061463  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 737) took 2264 ms. Trace:
I20260709 13:01:56.711036  1635 rpcz_store.cc:276] 0709 13:01:54.700480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.703814 (+  3334us) service_pool.cc:224] Handling call
0709 13:01:54.801477 (+ 97663us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.835509 (+ 34032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.835582 (+    73us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.836257 (+   675us) log.cc:844] Serialized 26 byte log entry
0709 13:01:56.559003 (+1722746us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:56.561642 (+  2639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.570752 (+  9110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.627877 (+ 57125us) raft_consensus.cc:1685] finished
0709 13:01:56.628391 (+   514us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:56.642343 (+ 13952us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6366,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":135,"thread_start_us":1694838,"threads_started":1,"wal-append.queue_time_us":1765943}
I20260709 13:01:56.909363  3765 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:01:57.084781  3808 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:01:57.085143  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:56.891765  3666 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:57.037652  3615 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:57.108202  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 1816) took 2206 ms. Trace:
I20260709 13:01:56.952726  3796 raft_consensus.cc:697] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:57.100703  1713 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:01:56.951174  3802 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.147s
I20260709 13:01:57.086431  3640 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:57.122713  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 480722us
I20260709 13:01:57.087095  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:57.127554  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:01:57.108702  1716 rpcz_store.cc:276] 0709 13:01:54.902032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.902225 (+   193us) service_pool.cc:224] Handling call
0709 13:01:54.902619 (+   394us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:54.952870 (+ 50251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.952899 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:54.971988 (+ 19089us) log.cc:844] Serialized 52 byte log entry
0709 13:01:56.660067 (+1688079us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:56.660120 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.660208 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.107912 (+447704us) raft_consensus.cc:1685] finished
0709 13:01:57.107936 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.108130 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58178,"raft.queue_time_us":63681,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":58444,"thread_start_us":1600003,"threads_started":1,"wal-append.queue_time_us":1819282}
I20260709 13:01:57.119705  1566 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 6: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:01:57.122599  3753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.027s	sys 0.003s
W20260709 13:01:57.148495  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 489340us
I20260709 13:01:57.149734  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:57.107048  3743 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:57.130221  1640 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.016s	sys 0.015s
W20260709 13:01:57.117436  3571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.010s	sys 0.020s
W20260709 13:01:57.151324  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.003s	sys 0.029s
W20260709 13:01:57.169682  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.005s	sys 0.030s
I20260709 13:01:57.170578  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:57.171080  2305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.030s	sys 0.032s
W20260709 13:01:57.171741  2305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.030s	sys 0.032s
W20260709 13:01:57.170627  1609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.010s	sys 0.026s
W20260709 13:01:57.172926  1609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.010s	sys 0.026s
W20260709 13:01:57.170089  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 544896us
I20260709 13:01:57.183956  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:57.155133  3753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.027s	sys 0.004s
W20260709 13:01:57.150317  3585 consensus_meta.cc:287] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.556s	user 0.007s	sys 0.003s
W20260709 13:01:57.170570  2303 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.011s	sys 0.018s
W20260709 13:01:57.227329  2303 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.011s	sys 0.018s
W20260709 13:01:57.169451  1640 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.020s	sys 0.016s
I20260709 13:01:57.115864  3804 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:57.228595  3804 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:57.172305  3571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.010s	sys 0.020s
I20260709 13:01:57.229080  3804 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:57.150617  3615 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:57.230808  3615 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.231246  3615 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:57.223212  3743 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:57.126588  1518 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.016s	sys 0.045s
I20260709 13:01:57.159775  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:57.173642  3702 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:01:57.251646  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.238541  1636 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:01:57.240394  1518 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.016s	sys 0.045s
I20260709 13:01:57.264161  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 735) took 3464 ms. Trace:
I20260709 13:01:57.264744  2302 rpcz_store.cc:276] 0709 13:01:53.799257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.879736 (+ 80479us) service_pool.cc:224] Handling call
0709 13:01:53.905630 (+ 25894us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:53.988650 (+ 83020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:53.988694 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:54.043084 (+ 54390us) log.cc:844] Serialized 52 byte log entry
0709 13:01:55.883762 (+1840678us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b180036dd20. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:01:56.596820 (+713058us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:56.596859 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.596938 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.263753 (+666815us) raft_consensus.cc:1685] finished
0709 13:01:57.263783 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.264086 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":157818,"raft.queue_time_us":62655,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":14116,"spinlock_wait_cycles":10570624,"thread_start_us":712847,"threads_started":1,"wal-append.queue_time_us":2666407}
I20260709 13:01:57.240564  3592 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:57.263818  3684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.032s	sys 0.045s
I20260709 13:01:57.260669  1635 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:57.245549  3702 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:57.237934  3585 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:57.244192  3743 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:57.314184  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2093) took 1998 ms. Trace:
I20260709 13:01:57.302109  3821 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:57.294077  3796 consensus_queue.cc:237] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:57.284291  3827 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:57.300225  3816 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
W20260709 13:01:57.305075  3684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.032s	sys 0.045s
I20260709 13:01:57.371416  1641 rpcz_store.cc:276] 0709 13:01:55.316290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.878097 (+1561807us) service_pool.cc:224] Handling call
0709 13:01:56.878446 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.878785 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:56.947042 (+ 68257us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.313665 (+366623us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:57.313746 (+    81us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:57.313811 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.313912 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.314095 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":67978,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":366161,"threads_started":1}
I20260709 13:01:57.375921  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:57.377386  2302 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
W20260709 13:01:57.375031  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 294173us
W20260709 13:01:57.328321  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 292976us
W20260709 13:01:57.380299  1533 leader_election.cc:336] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 2.152s (SENT)
I20260709 13:01:57.383280  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:01:57.386307  1533 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 2.191s (SENT)
I20260709 13:01:57.390976  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:57.345754  3702 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:57.393920  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:57.402720  1709 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:57.406378  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:57.407374  2302 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:57.415961  3640 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:57.417052  3637 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:57.417841  3815 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 13:01:57.419128  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 241866us
I20260709 13:01:57.419560  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2078) took 2560 ms. Trace:
I20260709 13:01:57.419625  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2087) took 2391 ms. Trace:
I20260709 13:01:57.419945  1639 rpcz_store.cc:276] 0709 13:01:54.859390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.405123 (+1545733us) service_pool.cc:224] Handling call
0709 13:01:56.405562 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:56.422055 (+ 16493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:56.422098 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:56.422694 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 13:01:56.843044 (+420350us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:56.843084 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.843148 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.419346 (+576198us) raft_consensus.cc:1685] finished
0709 13:01:57.419364 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.419505 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":164774,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":9072,"thread_start_us":403166,"threads_started":1,"wal-append.queue_time_us":799145}
I20260709 13:01:57.420251  1644 rpcz_store.cc:276] 0709 13:01:55.028566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.603807 (+1575241us) service_pool.cc:224] Handling call
0709 13:01:56.606089 (+  2282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:56.610547 (+  4458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:56.610590 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:56.627339 (+ 16749us) log.cc:844] Serialized 26 byte log entry
0709 13:01:57.107668 (+480329us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:57.107730 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.107843 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.419348 (+311505us) raft_consensus.cc:1685] finished
0709 13:01:57.419376 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.419551 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":280076,"raft.run_cpu_time_us":8064,"raft.run_wall_time_us":146065,"thread_start_us":456681,"threads_started":1,"wal-append.queue_time_us":773131}
I20260709 13:01:57.431711  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 1817) took 2470 ms. Trace:
I20260709 13:01:57.433501  1710 rpcz_store.cc:276] 0709 13:01:54.961136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.968490 (+  7354us) service_pool.cc:224] Handling call
0709 13:01:54.969177 (+   687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.038265 (+2069088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.431369 (+393104us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b54003db3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:57.431628 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4534559104}
I20260709 13:01:57.444983  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:57.446192  2302 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:57.427431  1532 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:57.451996  3702 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:57.454134  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 326712us
I20260709 13:01:57.455721  2268 leader_election.cc:400] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 3. Current term is 4.
I20260709 13:01:57.456126  2268 leader_election.cc:403] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:57.456972  2268 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:01:57.479984  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 93728us
I20260709 13:01:57.480098  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:57.480779  1533 leader_election.cc:336] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 2.213s (SENT)
I20260709 13:01:57.390178  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:57.486894  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:57.487139  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 742) took 2543 ms. Trace:
I20260709 13:01:57.480809  1639 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
W20260709 13:01:57.479332  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 52031.1us
I20260709 13:01:57.372359  3827 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:01:57.490846  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.429458  3636 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:57.487700  2305 rpcz_store.cc:276] 0709 13:01:54.943716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.591872 (+1648156us) service_pool.cc:224] Handling call
0709 13:01:56.592180 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.592424 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:56.597451 (+  5027us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.486009 (+888558us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:57.486070 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:57.486119 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.486902 (+   783us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.487068 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1547,"prepare.queue_time_us":8091,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":105364,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":9405,"thread_start_us":888115,"threads_started":2}
I20260709 13:01:57.495623  1710 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:57.495779  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:57.423374  3831 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:57.500334  3831 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:57.500519  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:57.502861  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:57.503605  1639 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:57.482580  3640 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:57.515215  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:57.516142  1710 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:57.517833  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 333978us
I20260709 13:01:57.507421  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:57.521850  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:57.523177  2305 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:57.522002  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:57.519057  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:57.525457  1644 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 4: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:57.491004  3637 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:57.492267  3827 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:57.571679  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.575114  1569 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:57.576084  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2113) took 1129 ms. Trace:
I20260709 13:01:57.576520  1569 rpcz_store.cc:276] 0709 13:01:56.446568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.574137 (+1127569us) service_pool.cc:224] Handling call
0709 13:01:57.574581 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.575729 (+  1148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.576009 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:57.577935  1710 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:01:57.494271  3805 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:57.508605  3802 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:57.588758  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:57.643644  2302 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:57.579794  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:57.645494  1710 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:01:57.644724  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:57.630398  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:57.649516  1644 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:57.533694  3821 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:57.654984  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:57.652436  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2090) took 2542 ms. Trace:
I20260709 13:01:57.655781  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:57.653955  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.502606  3684 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:01:57.546064  3816 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.655761  2302 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:01:57.578878  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2114) took 1125 ms. Trace:
I20260709 13:01:57.682814  1569 rpcz_store.cc:276] 0709 13:01:56.453529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.577774 (+1124245us) service_pool.cc:224] Handling call
0709 13:01:57.578241 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.578464 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.578504 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:57.578562 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:57.578607 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.578683 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.578825 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:57.545365  3640 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:01:57.676191  1609 connection.cc:256] RPC call timeout handler was delayed by -1.064067284000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 13:01:57.651324  3712 consensus_meta.cc:287] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.826s	user 0.010s	sys 0.000s
I20260709 13:01:57.498376  3804 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:57.688575  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:57.689435  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:57.535400  3841 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:57.663455  1643 rpcz_store.cc:276] 0709 13:01:55.109620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.662010 (+1552390us) service_pool.cc:224] Handling call
0709 13:01:56.662333 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.662546 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:56.703543 (+ 40997us) log.cc:844] Serialized 14 byte log entry
0709 13:01:56.770123 (+ 66580us) spinlock_profiling.cc:243] Waited 5.53 ms on lock 0x7b18003cef80. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a 00007fd7a1caba2a
0709 13:01:57.519790 (+749667us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.651587 (+131797us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:57.651710 (+   123us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:57.651826 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.651968 (+   142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.652294 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21061,"prepare.queue_time_us":46827,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":100,"spinlock_wait_cycles":12729600,"thread_start_us":857256,"threads_started":3}
I20260709 13:01:57.644752  3802 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.761053  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:57.669128  3817 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.427s	user 0.000s	sys 0.001s
I20260709 13:01:57.094884  2298 rpcz_store.cc:276] 0709 13:01:54.787002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.803233 (+ 16231us) service_pool.cc:224] Handling call
0709 13:01:54.908334 (+105101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.969571 (+ 61237us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:56.533591 (+1564020us) log.cc:844] Serialized 14 byte log entry
0709 13:01:56.886824 (+353233us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:56.917564 (+ 30740us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:56.933194 (+ 15630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.959247 (+ 26053us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.051158 (+ 91911us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8740,"prepare.queue_time_us":1543437,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":360812,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":58,"thread_start_us":1740100,"threads_started":2,"wal-append.queue_time_us":624165}
I20260709 13:01:57.754797  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:57.653843  3774 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:57.769789  2305 raft_consensus.cc:2410] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:01:57.788205  2268 connection.cc:256] RPC call timeout handler was delayed by -3.2542299720000187s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
W20260709 13:01:57.793668  2268 connection.cc:256] RPC call timeout handler was delayed by -3.0805299720000221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1791s for a response
W20260709 13:01:57.794138  2268 connection.cc:256] RPC call timeout handler was delayed by -2.6107299720000015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23130000000000003s for a response
I20260709 13:01:57.761344  1711 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:57.811982  3859 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:57.812182  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:57.812410  3855 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:57.812887  3855 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.812639  3859 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:01:57.813290  3855 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:57.813711  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:57.805462  3819 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.502s	user 0.001s	sys 0.001s
I20260709 13:01:57.815114  3859 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:57.777719  3831 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:57.705796  3816 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:57.804204  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:57.806283  3774 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:01:57.807761  2299 consensus_meta.cc:287] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.092s	user 0.013s	sys 0.000s
I20260709 13:01:57.808713  3743 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:57.809211  3732 consensus_meta.cc:287] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.045s	user 0.009s	sys 0.001s
W20260709 13:01:57.809587  3834 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 13:01:57.809727  3821 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:57.705544  3822 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:57.777289  3802 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:57.818270  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:57.818442  2305 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:01:57.776150  3841 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.854404  2302 raft_consensus.cc:2410] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:01:57.823349  3842 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
W20260709 13:01:57.812595  3764 consensus_meta.cc:287] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.288s	user 0.011s	sys 0.000s
W20260709 13:01:57.825222  3662 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.021s	sys 0.000s
W20260709 13:01:57.855046  3670 consensus_meta.cc:287] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.085s	user 0.012s	sys 0.000s
W20260709 13:01:57.862551  3840 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.325s	user 0.001s	sys 0.000s
I20260709 13:01:57.825956  3822 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.867151  3670 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:57.867735  3818 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.213s	user 0.000s	sys 0.000s
W20260709 13:01:57.868942  3823 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.238s	user 0.000s	sys 0.000s
I20260709 13:01:57.869575  3670 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:57.870280  3763 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.102s	user 0.010s	sys 0.003s
I20260709 13:01:57.825958  3847 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:01:57.870656  3693 consensus_meta.cc:287] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.941s	user 0.008s	sys 0.002s
W20260709 13:01:57.860790  3662 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.022s	sys 0.000s
I20260709 13:01:57.826179  3732 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:57.865737  3833 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.381s	user 0.001s	sys 0.000s
I20260709 13:01:57.857689  3841 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:57.879197  3852 log.cc:927] Time spent T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:01:57.876574  3847 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:57.863873  3702 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:57.871834  3822 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:57.861996  3764 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:57.885334  3816 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:57.876513  3709 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:57.852636  3640 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:57.886070  3762 consensus_meta.cc:287] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.271s	user 0.008s	sys 0.004s
I20260709 13:01:57.889869  3804 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:57.889765  3762 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:57.892761  3762 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:57.894907  3765 consensus_meta.cc:287] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.795s	user 0.009s	sys 0.001s
I20260709 13:01:57.896389  3855 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:57.898787  3855 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:57.898888  3802 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:57.902119  3827 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:57.905827  3844 log.cc:927] Time spent T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
W20260709 13:01:57.936439  1532 connection.cc:256] RPC call timeout handler was delayed by -1.3787930459999984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
I20260709 13:01:57.941337  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2103) took 1743 ms. Trace:
I20260709 13:01:57.943869  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2102) took 1746 ms. Trace:
I20260709 13:01:57.942057  1568 rpcz_store.cc:276] 0709 13:01:56.197960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.798667 (+600707us) service_pool.cc:224] Handling call
0709 13:01:56.799194 (+   527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:56.799749 (+   555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:56.799795 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:56.800481 (+   686us) log.cc:844] Serialized 26 byte log entry
0709 13:01:57.315378 (+514897us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:57.360499 (+ 45121us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.360836 (+   337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.360940 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.940526 (+579586us) raft_consensus.cc:1685] finished
0709 13:01:57.940610 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.941014 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44956,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":4676,"raft.run_cpu_time_us":401,"raft.run_wall_time_us":25684,"thread_start_us":430870,"threads_started":1,"wal-append.queue_time_us":1066980}
I20260709 13:01:57.944485  1563 rpcz_store.cc:276] 0709 13:01:56.197690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.714475 (+516785us) service_pool.cc:224] Handling call
0709 13:01:56.752960 (+ 38485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:56.776361 (+ 23401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:56.776395 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:56.776926 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:01:57.236830 (+459904us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:57.263033 (+ 26203us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.263240 (+   207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.263322 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.943160 (+679838us) raft_consensus.cc:1685] finished
0709 13:01:57.943221 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.943630 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27353,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":5637,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"spinlock_wait_cycles":13822208,"thread_start_us":459467,"threads_started":1,"wal-append.queue_time_us":1096481}
I20260709 13:01:57.949100  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:01:57.955710  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 436609us
I20260709 13:01:57.959362  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:57.962232  1607 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:57.965202  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:57.973599  1609 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:57.785257  1639 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:01:58.048699  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:01:57.974783  1562 consensus_meta.cc:287] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.080s	user 0.017s	sys 0.008s
I20260709 13:01:58.051391  1562 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:58.052487  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2107) took 1803 ms. Trace:
I20260709 13:01:58.053081  1562 rpcz_store.cc:276] 0709 13:01:56.249343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.644846 (+395503us) service_pool.cc:224] Handling call
0709 13:01:58.052356 (+1407510us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:57.975471  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2083) took 3020 ms. Trace:
I20260709 13:01:57.976749  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:58.056000  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 82797.3us
I20260709 13:01:58.056638  1643 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:01:58.057673  1609 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:58.059154  3853 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:58.059404  3816 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:57.977429  1644 raft_consensus.cc:2410] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:57.703953  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:57.987524  1567 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:01:58.061476  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2120) took 1523 ms. Trace:
I20260709 13:01:58.061614  1636 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:01:58.061834  1567 rpcz_store.cc:276] 0709 13:01:56.537767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.986429 (+1448662us) service_pool.cc:224] Handling call
0709 13:01:57.987013 (+   584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:58.061111 (+ 74098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.061401 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:58.062671  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2098) took 1179 ms. Trace:
W20260709 13:01:57.993716  1635 consensus_meta.cc:287] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.636s	user 0.002s	sys 0.013s
I20260709 13:01:57.996071  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2089) took 2922 ms. Trace:
I20260709 13:01:58.004889  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2111) took 1596 ms. Trace:
I20260709 13:01:58.017557  1682 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:58.018110  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:58.002506  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:58.020495  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 1825) took 1205 ms. Trace:
W20260709 13:01:58.025650  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.030773  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.031219  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 1823) took 1218 ms. Trace:
W20260709 13:01:57.886121  3615 consensus_meta.cc:287] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.654s	user 0.008s	sys 0.000s
I20260709 13:01:58.032085  1710 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
W20260709 13:01:58.032619  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.033768  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:57.875638  3585 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:01:57.817137  2296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.000s	sys 0.012s
I20260709 13:01:57.876591  3693 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:57.850481  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:58.042546  1712 raft_consensus.cc:2368] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4.
I20260709 13:01:58.043056  1713 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:58.043640  3855 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:01:58.044569  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.044853  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:58.063271  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:01:58.064002  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 99156.3us
I20260709 13:01:58.058383  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:58.050570  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:58.068930  1565 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
W20260709 13:01:58.069126  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 51939.8us
W20260709 13:01:58.072736  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.039144  3764 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:58.080343  3636 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:58.066802  1709 rpcz_store.cc:276] 0709 13:01:56.814616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.814776 (+   160us) service_pool.cc:224] Handling call
0709 13:01:58.019961 (+1205185us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b54003dc2d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:58.020268 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204491136}
I20260709 13:01:58.066233  1566 rpcz_store.cc:276] 0709 13:01:56.408460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.631808 (+223348us) service_pool.cc:224] Handling call
0709 13:01:58.004307 (+1372499us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b54002977d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:58.004604 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1110429952}
I20260709 13:01:58.063349  1636 rpcz_store.cc:276] 0709 13:01:56.882903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.651545 (+768642us) service_pool.cc:224] Handling call
0709 13:01:58.062555 (+411010us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:58.084642  1643 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:58.070367  3816 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:58.083643  1639 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.091725  3732 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:58.066008  1637 rpcz_store.cc:276] 0709 13:01:55.073514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.645126 (+1571612us) service_pool.cc:224] Handling call
0709 13:01:56.658543 (+ 13417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.825658 (+167115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.995526 (+1169868us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540032c8d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:57.995891 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258326400}
I20260709 13:01:58.093405  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:58.094461  1566 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:58.094791  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.096264  1637 raft_consensus.cc:2410] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:01:57.821102  2299 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:58.055172  1642 rpcz_store.cc:276] 0709 13:01:54.954872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.532037 (+1577165us) service_pool.cc:224] Handling call
0709 13:01:56.532437 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:56.853401 (+320964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:56.853467 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:56.881879 (+ 28412us) log.cc:844] Serialized 26 byte log entry
0709 13:01:57.316022 (+434143us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:57.316081 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.316184 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.974766 (+658582us) raft_consensus.cc:1685] finished
0709 13:01:57.974812 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.975176 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":52527,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":4539,"spinlock_wait_cycles":3456,"thread_start_us":602558,"threads_started":2,"wal-append.queue_time_us":553629}
W20260709 13:01:58.147032  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.093405  1567 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.145423  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.139926  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:58.160009  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 100868us
I20260709 13:01:58.150604  1682 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:58.152474  2296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.005s	sys 0.014s
I20260709 13:01:58.079905  1635 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:58.155853  3762 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:58.172227  1639 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:01:58.089291  3636 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.175623  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:58.176973  1716 raft_consensus.cc:2368] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3.
I20260709 13:01:58.154772  3795 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:58.097446  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:58.149540  3693 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:58.091166  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:58.086790  3855 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:58.181298  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.182052  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:58.182782  3637 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:01:58.185333  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.155416  3743 consensus_queue.cc:1059] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 13:01:58.152226  3804 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:58.187961  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2096) took 1305 ms. Trace:
I20260709 13:01:58.188884  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:58.189007  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.189429  2269 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:58.189916  3615 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:58.206082  3571 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:58.206991  3571 raft_consensus.cc:697] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:58.208540  3571 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:58.180197  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.211156  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.191294  1567 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
W20260709 13:01:58.191627  2299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 743) took 3235 ms (client timeout 3211 ms). Trace:
I20260709 13:01:58.193444  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 741) took 3291 ms. Trace:
I20260709 13:01:58.154532  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:58.197425  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 139966us
I20260709 13:01:58.174348  3762 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.170801  3732 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.068005  1717 rpcz_store.cc:276] 0709 13:01:56.812950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.825569 (+ 12619us) service_pool.cc:224] Handling call
0709 13:01:58.030667 (+1205098us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b5400239e54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:58.031016 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1106205568}
I20260709 13:01:58.179006  3636 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:58.204578  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:58.204972  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 133273us
W20260709 13:01:58.214494  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 64135.5us
I20260709 13:01:58.216316  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:58.217875  1643 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
W20260709 13:01:58.216015  2299 rpcz_store.cc:269] 0709 13:01:54.955635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.584681 (+1629046us) service_pool.cc:224] Handling call
0709 13:01:58.177876 (+1593195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:58.219062  3760 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.219379  1683 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:58.219918  1714 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:58.220382  1639 raft_consensus.cc:2410] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:58.219986  3841 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:58.202903  1635 rpcz_store.cc:276] 0709 13:01:56.882269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.119480 (+237211us) service_pool.cc:224] Handling call
0709 13:01:58.187859 (+1068379us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:58.221935  3765 raft_consensus.cc:2764] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:01:58.202020  3824 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 3. Current term is 4.
I20260709 13:01:58.213605  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:58.225142  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:58.328296  1567 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 4: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:01:58.328917  3832 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.522s	user 0.001s	sys 0.001s
I20260709 13:01:58.329540  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:58.330302  1567 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:58.330101  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:58.233906  1642 raft_consensus.cc:2393] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:58.199457  3855 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:58.232103  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:01:58.334611  3835 log.cc:927] Time spent T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.519s	user 0.001s	sys 0.001s
I20260709 13:01:58.330175  3743 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:58.335690  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:58.336794  3846 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.525s	user 0.002s	sys 0.000s
I20260709 13:01:58.337013  3615 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:58.330060  1562 raft_consensus.cc:2393] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 4: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:01:58.339111  1714 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 1829) took 1484 ms (client timeout 1900 ms). Trace:
I20260709 13:01:58.300689  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:58.338426  2298 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:01:58.236217  1638 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
W20260709 13:01:58.316211  3803 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 1.362s	user 0.008s	sys 0.000s
I20260709 13:01:58.356834  3804 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:01:58.327090  3830 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.517s	user 0.000s	sys 0.000s
W20260709 13:01:58.353434  1714 rpcz_store.cc:269] 0709 13:01:56.854854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.878130 (+ 23276us) service_pool.cc:224] Handling call
0709 13:01:58.096997 (+1218867us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b58001e4598. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:58.339035 (+242038us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7777536}
I20260709 13:01:58.339582  3762 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:58.343055  3760 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:58.378494  2301 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 553) took 3267 ms (client timeout 1893 ms). Trace:
I20260709 13:01:58.361666  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 550) took 3304 ms. Trace:
I20260709 13:01:58.405936  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.398129  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.364372  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:58.368453  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:01:58.426060  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 206842us
I20260709 13:01:58.361920  1682 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:58.412088  2301 rpcz_store.cc:269] 0709 13:01:55.110747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.130013 (+1019266us) service_pool.cc:224] Handling call
0709 13:01:58.378043 (+2248030us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b5400650254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:58.378278 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46740864}
I20260709 13:01:58.425715  2302 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.435097  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:58.432860  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 77534.7us
I20260709 13:01:58.438341  1643 raft_consensus.cc:2410] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:58.439963  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:58.377521  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 736) took 4148 ms. Trace:
I20260709 13:01:58.446336  1607 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:58.285842  3816 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.151s
I20260709 13:01:58.372370  3637 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:01:58.448553  1565 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 3: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:01:58.450085  3868 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:01:58.223771  2296 rpcz_store.cc:276] 0709 13:01:54.901926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.592721 (+1690795us) service_pool.cc:224] Handling call
0709 13:01:56.617480 (+ 24759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.639905 (+ 22425us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:56.663506 (+ 23601us) log.cc:844] Serialized 14 byte log entry
0709 13:01:58.192419 (+1528913us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:58.192616 (+   197us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:58.192760 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.192987 (+   227us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.193281 (+   294us) inbound_call.cc:177] Queueing success response
0709 13:01:58.327985 (+134704us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
Metrics: {"mutex_wait_us":3297,"prepare.queue_time_us":12640,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":1377,"raft.queue_time_us":150891,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":15272,"spinlock_wait_cycles":24599808,"thread_start_us":1399395,"threads_started":2,"wal-append.queue_time_us":1093877}
I20260709 13:01:58.450238  3773 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:58.370904  3802 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:58.454126  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 92425.6us
I20260709 13:01:58.453378  1639 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:01:58.412254  2304 rpcz_store.cc:276] 0709 13:01:55.057477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.625555 (+1568078us) service_pool.cc:224] Handling call
0709 13:01:56.642097 (+ 16542us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:57.100910 (+458813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.100939 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:57.101342 (+   403us) log.cc:844] Serialized 26 byte log entry
0709 13:01:57.492396 (+391054us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:57.492481 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.492604 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:58.335742 (+843138us) spinlock_profiling.cc:243] Waited 7.37 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:01:58.358447 (+ 22705us) raft_consensus.cc:1685] finished
0709 13:01:58.358475 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.361450 (+  2975us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":586912,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":121,"spinlock_wait_cycles":16948992,"thread_start_us":768773,"threads_started":2,"wal-append.queue_time_us":682220}
I20260709 13:01:58.380548  3853 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:01:58.430958  3732 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:58.466064  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:58.468899  1565 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.465363  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:58.471320  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:58.472536  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.445044  2297 rpcz_store.cc:276] 0709 13:01:54.229170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.411693 (+182523us) service_pool.cc:224] Handling call
0709 13:01:54.844696 (+433003us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.845207 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.845241 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.845831 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 13:01:56.539017 (+1693186us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:56.539085 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:56.539170 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:56.702497 (+163327us) raft_consensus.cc:1685] finished
0709 13:01:56.702522 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.376946 (+1674424us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b540059b5d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:58.377270 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4978,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":133,"spinlock_wait_cycles":994944768,"thread_start_us":1674586,"threads_started":1,"wal-append.queue_time_us":1793465,"wal-append.run_cpu_time_us":23430,"wal-append.run_wall_time_us":1119412}
I20260709 13:01:58.477005  1638 raft_consensus.cc:2410] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:01:58.478138  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.480049  3709 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:58.480600  3709 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:58.481029  3709 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:58.456955  3765 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:58.486778  1638 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:58.488034  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.488648  1638 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:58.489037  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2095) took 1607 ms. Trace:
I20260709 13:01:58.489254  1638 rpcz_store.cc:276] 0709 13:01:56.881384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.487687 (+1606303us) service_pool.cc:224] Handling call
0709 13:01:58.488990 (+  1303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:58.489931  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:58.490494  1638 raft_consensus.cc:2410] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:01:58.491151  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.491787  1638 raft_consensus.cc:2410] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:58.492313  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:58.492787  1638 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:58.493279  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.494125  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:58.496079  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:58.496739  1638 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:58.499071  3702 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:01:58.499542  3841 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:58.500865  3702 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:58.501241  3702 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:58.501595  3702 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.503361  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:58.504673  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 706) took 1008 ms. Trace:
I20260709 13:01:58.505177  2299 rpcz_store.cc:276] 0709 13:01:57.495748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.458502 (+962754us) service_pool.cc:224] Handling call
0709 13:01:58.504573 (+ 46071us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:58.486265  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 59527.8us
I20260709 13:01:58.451009  3773 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:58.512450  1639 raft_consensus.cc:2410] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:01:58.512902  3805 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:58.513518  3805 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.513764  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.514493  3773 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:58.457769  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:58.515393  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:58.515604  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 69327.6us
I20260709 13:01:58.467944  1683 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:58.516884  1638 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:01:58.516993  1566 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
W20260709 13:01:58.518031  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 53293us
I20260709 13:01:58.445837  3743 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:58.524075  2269 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:58.524271  1565 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
W20260709 13:01:58.525894  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 267758us
I20260709 13:01:58.526182  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:58.523880  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:58.528864  1643 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.530088  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:58.527050  3764 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:58.532027  3764 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.532684  3764 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:58.536100  1715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1551) took 3829 ms (client timeout 1716 ms). Trace:
I20260709 13:01:58.536371  3732 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:01:58.536554  1715 rpcz_store.cc:269] 0709 13:01:54.706443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.773483 (+ 67040us) service_pool.cc:224] Handling call
0709 13:01:56.540745 (+1767262us) spinlock_profiling.cc:243] Waited 7.52 ms on lock 0x7b58001e4598. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:58.535772 (+1995027us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b5400136ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:58.535913 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48836736}
I20260709 13:01:58.537869  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:58.540010  1711 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:01:58.540787  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1571) took 1040 ms. Trace:
I20260709 13:01:58.541368  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:58.542412  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 1826) took 1726 ms (client timeout 1935 ms). Trace:
I20260709 13:01:58.543972  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:58.519419  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:01:58.545570  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.546667  1639 raft_consensus.cc:2393] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:01:58.542815  1718 rpcz_store.cc:269] 0709 13:01:56.816063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.876229 (+ 60166us) service_pool.cc:224] Handling call
0709 13:01:58.541013 (+1664784us) spinlock_profiling.cc:243] Waited 7.28 ms on lock 0x7b58001e4598. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:58.542358 (+  1345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16745216}
I20260709 13:01:58.553187  1565 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:58.553639  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.554240  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:58.554332  3743 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:58.555181  1565 raft_consensus.cc:2410] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:01:58.556134  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:58.556208  3827 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:58.557682  3760 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:58.559098  3802 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:58.559300  1717 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:58.559288  3636 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:58.559960  3802 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:58.561149  3802 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:58.561623  3636 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:58.543228  3765 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:01:58.541133  1711 rpcz_store.cc:276] 0709 13:01:57.500272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.442997 (+942725us) service_pool.cc:224] Handling call
0709 13:01:58.539699 (+ 96702us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:58.540486 (+   787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.540708 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:58.570953  1638 raft_consensus.cc:2368] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3.
W20260709 13:01:58.597285  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 46222.8us
W20260709 13:01:58.597491  1533 leader_election.cc:336] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.636s (SENT)
I20260709 13:01:58.566880  3765 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:58.598286  1609 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:58.598591  3765 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:01:58.598928  3850 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:58.600023  1711 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:58.603354  2302 raft_consensus.cc:2410] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:01:58.603106  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 157302us
I20260709 13:01:58.603701  1715 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:01:58.603922  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 705) took 1108 ms. Trace:
W20260709 13:01:58.604010  3874 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:01:58.604344  2302 rpcz_store.cc:276] 0709 13:01:57.495434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.156704 (+661270us) service_pool.cc:224] Handling call
0709 13:01:58.603864 (+447160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:58.605062  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:58.605496  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:58.605907  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 551) took 3506 ms. Trace:
I20260709 13:01:58.606319  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 702) took 1115 ms. Trace:
I20260709 13:01:58.606693  1682 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:58.597534  3822 consensus_meta.cc:287] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.709s	user 0.012s	sys 0.004s
I20260709 13:01:58.607443  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:58.607628  3822 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:58.606746  2302 rpcz_store.cc:276] 0709 13:01:57.490405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.605169 (+1114764us) service_pool.cc:224] Handling call
0709 13:01:58.606235 (+  1066us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:58.607781  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 81076.2us
I20260709 13:01:58.621974  3855 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:58.623554  1641 raft_consensus.cc:2368] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 5. Current term is 6.
I20260709 13:01:58.624486  1638 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:01:58.627287  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 79189.5us
I20260709 13:01:58.543586  3762 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:58.634310  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 399) took 2636 ms. Trace:
I20260709 13:01:58.634814  1638 rpcz_store.cc:276] 0709 13:01:55.994568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.622721 (+2628153us) service_pool.cc:224] Handling call
0709 13:01:58.624276 (+  1555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.630550 (+  6274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.630821 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:58.632462  3820 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:58.640668  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 92861.4us
I20260709 13:01:58.646747  3665 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:01:58.647616  3665 raft_consensus.cc:697] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:01:58.648092  3820 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:58.649187  3541 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:58.649025  3665 consensus_queue.cc:237] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:58.696437  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2104) took 2439 ms. Trace:
I20260709 13:01:58.697072  1560 rpcz_store.cc:276] 0709 13:01:56.198228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.429784 (+1231556us) service_pool.cc:224] Handling call
0709 13:01:57.547196 (+117412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:58.224501 (+677305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.238521 (+ 14020us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:58.335339 (+ 96818us) log.cc:844] Serialized 26 byte log entry
0709 13:01:58.435891 (+100552us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:58.440956 (+  5065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.453924 (+ 12968us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:58.620486 (+166562us) raft_consensus.cc:1685] finished
0709 13:01:58.620524 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.637326 (+ 16802us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123791,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":30026,"thread_start_us":553297,"threads_started":2,"wal-append.queue_time_us":134813}
I20260709 13:01:58.699056  3728 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:58.606365  2303 rpcz_store.cc:276] 0709 13:01:55.057849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.630746 (+1572897us) service_pool.cc:224] Handling call
0709 13:01:56.631229 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:56.635900 (+  4671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:56.635954 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:56.640730 (+  4776us) log.cc:844] Serialized 26 byte log entry
0709 13:01:57.227876 (+587146us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:57.227941 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.228045 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:58.325698 (+1097653us) spinlock_profiling.cc:243] Waited 7.41 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:01:58.542980 (+217282us) raft_consensus.cc:1685] finished
0709 13:01:58.543002 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.564437 (+ 21435us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":241282,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":5700,"spinlock_wait_cycles":17031552,"thread_start_us":569230,"threads_started":1,"wal-append.queue_time_us":999850}
I20260709 13:01:58.630666  3702 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:58.628185  3709 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:58.703814  3760 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:58.822582  1643 raft_consensus.cc:2410] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:01:58.949544  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 305229us
I20260709 13:01:58.809783  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:59.205097  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 364854us
W20260709 13:01:59.044832  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 413690us
W20260709 13:01:58.815651  1534 leader_election.cc:336] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.637s (SENT)
W20260709 13:01:59.207866  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.712821  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:01:59.209246  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.711982  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:58.916965  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 120881us
I20260709 13:01:58.642206  1711 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
W20260709 13:01:58.835168  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 137482us
I20260709 13:01:59.224402  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 707) took 1722 ms. Trace:
I20260709 13:01:59.224994  2302 rpcz_store.cc:276] 0709 13:01:57.496014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.630274 (+1134260us) service_pool.cc:224] Handling call
0709 13:01:59.217993 (+587719us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:58.711328  1636 raft_consensus.cc:2393] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:01:59.239682  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 600135us
I20260709 13:01:59.242362  1681 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:01:59.242662  3728 consensus_meta.cc:287] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.541s	user 0.008s	sys 0.000s
I20260709 13:01:59.254431  3878 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:59.255677  1681 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:59.256855  3760 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:59.257722  3760 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:58.980716  3762 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:59.265005  1714 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:58.873970  3822 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:59.244122  1534 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:58.955726  3853 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:59.261705  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 695) took 1775 ms. Trace:
I20260709 13:01:59.307379  2303 rpcz_store.cc:276] 0709 13:01:57.486442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.712447 (+1226005us) service_pool.cc:224] Handling call
0709 13:01:59.261595 (+549148us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:59.266371  3765 consensus_meta.cc:287] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.667s	user 0.011s	sys 0.000s
I20260709 13:01:59.272277  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 436) took 1517 ms. Trace:
I20260709 13:01:59.310657  1711 rpcz_store.cc:276] 0709 13:01:57.754572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.602479 (+847907us) service_pool.cc:224] Handling call
0709 13:01:58.613694 (+ 11215us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.224006 (+610312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.266668 (+ 42662us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:59.310823  3765 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:59.284291  2302 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:59.285434  3702 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:59.290938  3709 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:58.700109  3773 consensus_queue.cc:1059] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.565s
W20260709 13:01:59.294467  3764 consensus_meta.cc:287] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.761s	user 0.009s	sys 0.000s
I20260709 13:01:59.323375  3764 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:01:59.295680  3855 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:59.325034  3702 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:59.299715  3805 consensus_meta.cc:287] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.786s	user 0.012s	sys 0.000s
I20260709 13:01:59.325734  3764 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:01:59.270555  3802 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:01:59.301633  1560 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:01:58.835541  3882 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.448s
W20260709 13:01:59.320477  2269 leader_election.cc:336] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.598s (SENT)
I20260709 13:01:59.323706  3765 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:59.325143  3855 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:59.302309  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:59.327250  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 747) took 2658 ms. Trace:
W20260709 13:01:59.320081  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2101) took 2436 ms (client timeout 3038 ms). Trace:
I20260709 13:01:59.332100  2302 rpcz_store.cc:276] 0709 13:01:56.654369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.244983 (+2590614us) service_pool.cc:224] Handling call
0709 13:01:59.283929 (+ 38946us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.312294 (+ 28365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.312529 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:59.323406  1710 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:01:59.328150  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2118) took 2791 ms. Trace:
I20260709 13:01:59.328308  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2121) took 2777 ms. Trace:
I20260709 13:01:59.326074  3805 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:59.273111  3728 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:59.335850  1715 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:59.333848  3855 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:01:59.341930  3867 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:59.342339  2302 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:59.353474  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 751) took 2682 ms. Trace:
I20260709 13:01:59.353855  2302 rpcz_store.cc:276] 0709 13:01:56.670702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.333911 (+2663209us) service_pool.cc:224] Handling call
0709 13:01:59.342109 (+  8198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.353165 (+ 11056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.353414 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:59.340448  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 748) took 2684 ms. Trace:
I20260709 13:01:59.356123  2303 rpcz_store.cc:276] 0709 13:01:56.656155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.315041 (+2658886us) service_pool.cc:224] Handling call
0709 13:01:59.339489 (+ 24448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.339873 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.339909 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:59.339986 (+    77us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:59.340062 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.340199 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.340348 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:59.343451  1565 rpcz_store.cc:276] 0709 13:01:56.538035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.557041 (+2019006us) service_pool.cc:224] Handling call
0709 13:01:58.557452 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:58.572389 (+ 14937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.572438 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:58.642104 (+ 69666us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.250742 (+608638us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:59.250804 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.250906 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.315125 (+ 64219us) raft_consensus.cc:1685] finished
0709 13:01:59.315153 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.315366 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4473,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":37276,"thread_start_us":279141,"threads_started":1,"wal-append.queue_time_us":494052}
I20260709 13:01:59.343816  1563 rpcz_store.cc:276] 0709 13:01:56.537089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.950028 (+1412939us) service_pool.cc:224] Handling call
0709 13:01:57.950549 (+   521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.951000 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:57.951579 (+   579us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.322301 (+1370722us) spinlock_profiling.cc:243] Waited 4.7 ms on lock 0x7b180034c1e0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a
0709 13:01:59.322528 (+   227us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:59.322580 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:59.322637 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.327808 (+  5171us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.328094 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":89007,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":4879,"spinlock_wait_cycles":10811136,"thread_start_us":129494,"threads_started":1,"wal-append.queue_time_us":242979}
I20260709 13:01:59.334236  2269 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:01:59.334837  3691 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:59.342783  3636 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.335616  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2124) took 2796 ms. Trace:
W20260709 13:01:59.332307  1644 rpcz_store.cc:269] 0709 13:01:56.883809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.650419 (+766610us) service_pool.cc:224] Handling call
0709 13:01:59.302894 (+1652475us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54001e8a54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:59.320025 (+ 17131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473244416}
I20260709 13:01:59.398288  1643 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:01:59.365654  3867 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:59.365700  3471 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:59.414079  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 407) took 2181 ms. Trace:
I20260709 13:01:59.307297  3853 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:59.395478  3820 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:59.414557  1643 rpcz_store.cc:276] 0709 13:01:57.225479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.268065 (+2042586us) service_pool.cc:224] Handling call
0709 13:01:59.352138 (+ 84073us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.406037 (+ 53899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.406333 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:59.419507  3820 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:01:59.421509  1643 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:59.422227  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1618) took 1735 ms. Trace:
I20260709 13:01:59.423297  3471 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:59.424213  3788 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 2.795s	user 0.006s	sys 0.006s
I20260709 13:01:59.422520  1643 rpcz_store.cc:276] 0709 13:01:57.687336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.420982 (+1733646us) service_pool.cc:224] Handling call
0709 13:01:59.421351 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.421952 (+   601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.422151 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:59.403929  3691 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:59.439746  1715 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:01:59.426831  3853 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.411525  3636 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:01:59.459750  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 509240us
I20260709 13:01:59.405658  1560 rpcz_store.cc:276] 0709 13:01:56.539263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.026359 (+2487096us) service_pool.cc:224] Handling call
0709 13:01:59.271340 (+244981us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.330981 (+ 59641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.335489 (+  4508us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:59.470712  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 222017us
I20260709 13:01:59.472568  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:01:59.469751  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 195344us
W20260709 13:01:59.476240  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 220737us
I20260709 13:01:59.475826  1609 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:01:59.484812  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 669427us
I20260709 13:01:59.481070  1717 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:59.493549  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 400) took 3498 ms. Trace:
I20260709 13:01:59.493923  1681 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:59.487015  3865 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:59.502514  3898 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:59.502887  3636 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:59.503288  1609 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:59.468500  3855 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:59.512506  3855 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.495739  1639 rpcz_store.cc:276] 0709 13:01:55.994879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.608365 (+2613486us) service_pool.cc:224] Handling call
0709 13:01:58.608731 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.609051 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:58.644878 (+ 35827us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.476550 (+831672us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.489287 (+ 12737us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:59.489343 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:59.493286 (+  3943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.493363 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.493495 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47347,"prepare.queue_time_us":698862,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":66,"raft.queue_time_us":148156,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":166,"spinlock_wait_cycles":954752,"thread_start_us":759114,"threads_started":3,"wal-append.queue_time_us":600632}
I20260709 13:01:59.521464  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:59.522301  1639 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:59.523494  1639 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:59.524048  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 409) took 1530 ms. Trace:
I20260709 13:01:59.524319  1639 rpcz_store.cc:276] 0709 13:01:57.994023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.523148 (+1529125us) service_pool.cc:224] Handling call
0709 13:01:59.523380 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.523857 (+   477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.523978 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:59.494570  3882 raft_consensus.cc:2764] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:01:59.492316  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2094) took 3708 ms. Trace:
I20260709 13:01:59.530627  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:59.533149  3691 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:59.539644  1609 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:59.531150  3665 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:59.532446  3805 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:59.488107  3867 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:59.563499  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 402) took 3533 ms. Trace:
I20260709 13:01:59.530649  3774 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:59.539283  3848 consensus_queue.cc:1059] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:01:59.612062  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1641) took 1641 ms. Trace:
I20260709 13:01:59.552335  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:01:59.612803  1643 rpcz_store.cc:276] 0709 13:01:57.970551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.429232 (+1458681us) service_pool.cc:224] Handling call
0709 13:01:59.459582 (+ 30350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.478448 (+ 18866us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.478484 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.482884 (+  4400us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.605800 (+122916us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:59.605848 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.605924 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.611504 (+  5580us) raft_consensus.cc:1685] finished
0709 13:01:59.611527 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.611968 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2902,"raft.queue_time_us":16227,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":8141,"thread_start_us":103401,"threads_started":1,"wal-append.queue_time_us":114975}
I20260709 13:01:59.573244  3855 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:01:59.619496  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 92846.7us
W20260709 13:01:59.623000  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 83442.7us
I20260709 13:01:59.625499  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:01:59.620738  3872 log.cc:927] Time spent T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 1.143s	user 0.000s	sys 0.013s
I20260709 13:01:59.627324  1607 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:59.572697  3471 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:59.622362  3867 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.635332  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 410) took 1473 ms. Trace:
I20260709 13:01:59.631280  2269 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:01:59.645183  3805 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:59.645685  3805 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.645764  3471 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:59.646179  3805 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.646588  3471 consensus_queue.cc:1059] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:59.647616  3471 raft_consensus.cc:3055] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:59.648037  3471 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:59.648213  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:01:59.623922  3774 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:59.640887  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 408) took 1647 ms. Trace:
I20260709 13:01:59.629711  3804 raft_consensus.cc:2764] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:01:59.636722  1643 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:59.649236  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 164984us
W20260709 13:01:59.674808  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.651787  1644 rpcz_store.cc:276] 0709 13:01:57.993677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.436667 (+1442990us) service_pool.cc:224] Handling call
0709 13:01:59.460657 (+ 23990us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.465680 (+  5023us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:59.476437 (+ 10757us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.640474 (+164037us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:59.640536 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:59.640585 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.640657 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.640821 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2131,"prepare.queue_time_us":10040,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":1492,"raft.queue_time_us":151,"thread_start_us":151665,"threads_started":1,"wal-append.queue_time_us":159383}
W20260709 13:01:59.654564  3851 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 1.838s	user 0.009s	sys 0.013s
W20260709 13:01:59.656656  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.663290  3640 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.682322  3640 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:59.683977  1607 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:59.684981  3894 raft_consensus.cc:2764] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:01:59.664700  1609 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:59.686152  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:59.649492  3855 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:59.542578  1635 rpcz_store.cc:276] 0709 13:01:55.780300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.554668 (+2774368us) service_pool.cc:224] Handling call
0709 13:01:58.555102 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.555388 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:01:58.959015 (+403627us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.470658 (+511643us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:59.475184 (+  4526us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:59.476867 (+  1683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.481877 (+  5010us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.487780 (+  5903us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14300,"prepare.queue_time_us":705093,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":4910,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":4904,"thread_start_us":380447,"threads_started":2,"wal-append.queue_time_us":649700}
W20260709 13:01:59.687791  1636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1642) took 1716 ms (client timeout 1780 ms). Trace:
I20260709 13:01:59.693979  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1570) took 2197 ms. Trace:
I20260709 13:01:59.694473  1714 rpcz_store.cc:276] 0709 13:01:57.496816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.410356 (+913540us) service_pool.cc:224] Handling call
0709 13:01:58.431911 (+ 21555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.265662 (+833751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.693697 (+428035us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b54001c1954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:59.693878 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1915372800}
I20260709 13:01:59.695748  3640 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:01:59.696550  3607 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:59.696934  3640 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:59.697934  3640 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:59.651255  3774 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:59.699246  3774 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.667290  3471 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:59.669734  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 739) took 4785 ms. Trace:
I20260709 13:01:59.688639  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:59.689275  3891 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:01:59.649819  2296 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEARNER]: Advancing to term 5
I20260709 13:01:59.667424  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 754) took 2891 ms. Trace:
I20260709 13:01:59.713106  1643 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:01:59.710188  3882 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:59.713472  2302 rpcz_store.cc:276] 0709 13:01:56.776215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.381406 (+2605191us) service_pool.cc:224] Handling call
0709 13:01:59.431831 (+ 50425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.476971 (+ 45140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.477005 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.487005 (+ 10000us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.620668 (+133663us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:59.620725 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.620802 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.667170 (+ 46368us) raft_consensus.cc:1685] finished
0709 13:01:59.667192 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.667336 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7220,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":6168,"thread_start_us":73013,"threads_started":1,"wal-append.queue_time_us":131146}
I20260709 13:01:59.715306  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:59.716949  2302 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:01:59.717304  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 368) took 1720 ms. Trace:
I20260709 13:01:59.717585  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 690) took 2232 ms. Trace:
W20260709 13:01:59.691967  1636 rpcz_store.cc:269] 0709 13:01:57.970985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.210677 (+239692us) service_pool.cc:224] Handling call
0709 13:01:59.687544 (+1476867us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b54000d0054. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:59.687720 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009576832}
I20260709 13:01:59.700325  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 755) took 2893 ms. Trace:
I20260709 13:01:59.703814  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:59.702983  2300 rpcz_store.cc:276] 0709 13:01:54.884646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.889374 (+  4728us) service_pool.cc:224] Handling call
0709 13:01:55.056693 (+167319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.972298 (+915605us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:56.759005 (+786707us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.238665 (+479660us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:57.243964 (+  5299us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:57.246232 (+  2268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.669190 (+2422958us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b54005dc0d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:59.669263 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.669522 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":552022,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":31,"raft.queue_time_us":118055,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":51842,"spinlock_wait_cycles":2032229632,"thread_start_us":1180738,"threads_started":2,"wal-append.queue_time_us":664036,"wal-append.run_cpu_time_us":2426,"wal-append.run_wall_time_us":1539162}
W20260709 13:01:59.708640  1534 leader_election.cc:336] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 2.213s (SENT)
I20260709 13:01:59.722793  1534 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:01:59.723295  3891 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:59.701574  3607 raft_consensus.cc:1064] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:01:59.723819  3891 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:59.724251  3891 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.724525  3820 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:59.724936  3820 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:59.725490  3615 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:59.725852  3820 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:59.726243  3607 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:59.718067  2302 rpcz_store.cc:276] 0709 13:01:57.485241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.715919 (+2230678us) service_pool.cc:224] Handling call
0709 13:01:59.716326 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.717332 (+  1006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.717512 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:59.690521  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2125) took 3077 ms. Trace:
I20260709 13:01:59.729511  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:59.728122  1565 rpcz_store.cc:276] 0709 13:01:56.613122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.415459 (+2802337us) service_pool.cc:224] Handling call
0709 13:01:59.464918 (+ 49459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.470216 (+  5298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.470255 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.480640 (+ 10385us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.532269 (+ 51629us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:01:59.532312 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.532380 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.690146 (+157766us) raft_consensus.cc:1685] finished
0709 13:01:59.690179 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.690412 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4196,"raft.queue_time_us":6051,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":5592,"thread_start_us":33507,"threads_started":1,"wal-append.queue_time_us":146556}
I20260709 13:01:59.730114  1643 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.719529  2303 rpcz_store.cc:276] 0709 13:01:56.806969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.382997 (+2576028us) service_pool.cc:224] Handling call
0709 13:01:59.446499 (+ 63502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.479136 (+ 32637us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.479166 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.498736 (+ 19570us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.642571 (+143835us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:59.642607 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.642663 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.699999 (+ 57336us) raft_consensus.cc:1685] finished
0709 13:01:59.700031 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.700230 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":103664,"threads_started":1,"wal-append.queue_time_us":146078}
I20260709 13:01:59.726854  3820 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:59.745177  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:59.712590  3640 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:59.703877  3889 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:59.742300  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:01:59.755549  3889 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:01:59.755985  3889 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.757273  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 424) took 2125 ms. Trace:
I20260709 13:01:59.757688  1710 rpcz_store.cc:276] 0709 13:01:57.631797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.463628 (+831831us) service_pool.cc:224] Handling call
0709 13:01:58.544995 (+ 81367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.334712 (+789717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.757009 (+422297us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b5400135bd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:59.757208 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1789691136}
W20260709 13:01:59.751502  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.762223  3471 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.770568  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:59.764772  1644 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:01:59.779841  1565 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:59.776575  1533 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 2.389s (SENT) [suppressed 43 similar messages]
I20260709 13:01:59.790433  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:59.717733  1560 rpcz_store.cc:276] 0709 13:01:57.996484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.470312 (+1473828us) service_pool.cc:224] Handling call
0709 13:01:59.492472 (+ 22160us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.502737 (+ 10265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.502776 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.527230 (+ 24454us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.630616 (+103386us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:59.630712 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.639074 (+  8362us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.701799 (+ 62725us) raft_consensus.cc:1685] finished
0709 13:01:59.702439 (+   640us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.717220 (+ 14781us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5057,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":21374,"thread_start_us":77303,"threads_started":1,"wal-append.queue_time_us":142750}
I20260709 13:01:59.795446  1643 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:59.796454  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:59.797744  1643 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:01:59.798456  1644 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:59.778481  3824 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.793583  3848 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:59.817150  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.818852  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:59.822338  1712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 1819) took 3011 ms (client timeout 1925 ms). Trace:
I20260709 13:01:59.823367  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2085) took 4840 ms. Trace:
I20260709 13:01:59.824026  1644 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
W20260709 13:01:59.826479  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.833727  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:59.834683  1636 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:01:59.836912  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:59.837776  1636 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:59.830696  1644 pending_rounds.cc:85] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 4
I20260709 13:01:59.799270  3805 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:59.830400  1712 rpcz_store.cc:269] 0709 13:01:56.811514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.811659 (+   145us) service_pool.cc:224] Handling call
0709 13:01:59.822084 (+3010425us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54001487d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:59.822260 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2783173376}
I20260709 13:01:59.843585  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:59.847844  1567 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
W20260709 13:01:59.850694  3911 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:01:59.852689  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 371) took 1841 ms. Trace:
I20260709 13:01:59.853109  1563 rpcz_store.cc:276] 0709 13:01:58.011574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.659910 (+1648336us) service_pool.cc:224] Handling call
0709 13:01:59.660302 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.660770 (+   468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.660801 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.661354 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.712795 (+ 51441us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:59.770260 (+ 57465us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.770554 (+   294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.770654 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.852439 (+ 81785us) raft_consensus.cc:1685] finished
0709 13:01:59.852465 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.852631 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":74586,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":6864,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":2653,"thread_start_us":82802,"threads_started":2,"wal-append.queue_time_us":98852}
I20260709 13:01:59.868269  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:59.840924  1644 pending_rounds.cc:107] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:01:59.879349  1644 raft_consensus.cc:2889] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:59.879745  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:59.830466  1640 rpcz_store.cc:276] 0709 13:01:54.968463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.563295 (+1594832us) service_pool.cc:224] Handling call
0709 13:01:56.579373 (+ 16078us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:57.228442 (+649069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.228488 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:57.229293 (+   805us) log.cc:844] Serialized 52 byte log entry
0709 13:01:57.435072 (+205779us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:57.485363 (+ 50291us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.579195 (+ 93832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.579309 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.653681 (+2074372us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:01:59.794196 (+140515us) raft_consensus.cc:1685] finished
0709 13:01:59.799966 (+  5770us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.808258 (+  8292us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":51563,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":5203,"raft.queue_time_us":360134,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":7809,"spinlock_wait_cycles":5869824,"thread_start_us":947273,"threads_started":3,"wal-append.queue_time_us":586438}
I20260709 13:01:59.882022  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:59.884007  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:01:59.885056  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:59.882017  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:59.887516  1710 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:01:59.889110  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:01:59.891122  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.843488  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:01:59.896214  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 87808.9us
W20260709 13:01:59.903551  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:59.912083  3913 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:01:59.913862  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:59.915321  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:59.915967  1567 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.919401  1643 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 3: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:01:59.920118  3665 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:01:59.920570  3665 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:01:59.920971  3665 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:59.922585  3873 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 1.427s	user 0.004s	sys 0.004s
I20260709 13:01:59.923921  1682 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:59.643482  1639 rpcz_store.cc:276] 0709 13:01:58.157485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.524914 (+1367429us) service_pool.cc:224] Handling call
0709 13:01:59.525249 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.525483 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:59.525820 (+   337us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.630892 (+105072us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:59.630975 (+    83us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:59.631033 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.631096 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.631254 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1232,"prepare.queue_time_us":187,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":2980,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":8909,"thread_start_us":101647,"threads_started":1,"wal-append.queue_time_us":117773}
I20260709 13:01:59.925436  3863 consensus_queue.cc:1059] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:59.926429  3863 raft_consensus.cc:3055] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:59.926791  3863 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:59.927990  1639 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:01:59.928401  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1580) took 1319 ms. Trace:
I20260709 13:01:59.927918  3863 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:59.928755  1717 rpcz_store.cc:276] 0709 13:01:58.608780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.303214 (+694434us) service_pool.cc:224] Handling call
0709 13:01:59.303630 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.495463 (+191833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.928194 (+432731us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400239954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:01:59.928347 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":393076096}
I20260709 13:01:59.929296  3863 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:59.920714  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:59.931315  1643 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:59.931645  3847 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:59.932454  3843 consensus_queue.cc:1059] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:59.933408  3843 consensus_queue.cc:1243] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:01:59.934896  3805 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:01:59.932526  2298 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 701) took 2442 ms (client timeout 2136 ms). Trace:
I20260709 13:01:59.936257  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:01:59.937204  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1613) took 2251 ms. Trace:
I20260709 13:01:59.937566  1637 rpcz_store.cc:276] 0709 13:01:57.685721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.554612 (+868891us) service_pool.cc:224] Handling call
0709 13:01:59.936152 (+1381540us) spinlock_profiling.cc:243] Waited 7.18 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ceee 0000000000375939 00007fd7a6bf01e0 00007fd7a6cb90c4 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:59.937132 (+   980us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16522112}
I20260709 13:01:59.939265  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:59.939999  1711 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:01:59.941072  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:59.943908  3865 raft_consensus.cc:3055] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:59.944401  3865 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:01:59.945519  3865 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:59.882743  3848 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:01:59.948158  1641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 406) took 2752 ms (client timeout 1598 ms). Trace:
W20260709 13:01:59.948606  1641 rpcz_store.cc:269] 0709 13:01:57.195628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.382328 (+186700us) service_pool.cc:224] Handling call
0709 13:01:58.017757 (+635429us) spinlock_profiling.cc:243] Waited 7.64 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:59.947899 (+1930142us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b54001c7354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:01:59.948062 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229716736}
I20260709 13:01:59.949044  3848 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:01:59.949875  1641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:60370
I20260709 13:01:59.951189  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:59.952530  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:59.953575  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:59.957780  3855 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:01:59.958369  3855 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:59.960111  3758 raft_consensus.cc:3055] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:01:59.960587  3758 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:01:59.960673  2296 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEARNER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
W20260709 13:01:59.961418  2296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 691) took 2475 ms (client timeout 2213 ms). Trace:
W20260709 13:01:59.961905  2296 rpcz_store.cc:269] 0709 13:01:57.485469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.551821 (+2066352us) service_pool.cc:224] Handling call
0709 13:01:59.961304 (+409483us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:59.961759  3758 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:01:59.963532  3758 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.968778  1643 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:01:59.954602  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:59.845721  3824 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:01:59.950699  1641 raft_consensus.cc:606] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Received request to transfer leadership
W20260709 13:01:59.887036  1562 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 369) took 1890 ms (client timeout 2026 ms). Trace:
W20260709 13:01:59.970294  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 46533.1us
I20260709 13:01:59.895749  3615 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:59.979095  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:01:59.980566  1565 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:01:59.987216  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:01:59.972364  1567 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:01:59.974196  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2117) took 3437 ms. Trace:
I20260709 13:01:59.990583  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:59.987088  1637 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:59.946851  3865 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:01:59.935992  2298 rpcz_store.cc:269] 0709 13:01:57.490162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.932297 (+2442135us) service_pool.cc:211] Skipping call since client already timed out
0709 13:01:59.932437 (+   140us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:01:59.994989  1682 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:01:59.967892  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:01:59.998934  1607 connection.cc:256] RPC call timeout handler was delayed by -2.1867255530000307s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
I20260709 13:02:00.000018  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:60370 (request call id 81) took 2020 ms. Trace:
I20260709 13:02:00.001837  1641 rpcz_store.cc:276] 0709 13:01:57.972798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.949647 (+1976849us) service_pool.cc:224] Handling call
0709 13:01:59.993440 (+ 43793us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:00.002838  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 694) took 2516 ms. Trace:
I20260709 13:02:00.002933  3893 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:00.003535  3893 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:00.003961  3893 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:01:59.978646  1562 rpcz_store.cc:269] 0709 13:01:57.996824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.444996 (+448172us) service_pool.cc:224] Handling call
0709 13:01:59.818677 (+1373681us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:01:59.883592 (+ 64915us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6389632}
I20260709 13:01:59.992867  3567 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:00.005638  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:00.006863  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:59.978631  3774 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.007481  1563 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:00.007705  1715 raft_consensus.cc:2410] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:00.008698  3900 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:00.009114  3774 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:00.009380  3900 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.011068  3538 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:00.011701  3538 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:00.013578  3875 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 1.517s	user 0.004s	sys 0.007s
I20260709 13:02:00.014434  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:01:59.987434  1636 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:00.015237  1640 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:00.016132  3876 raft_consensus.cc:3055] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:00.016536  3876 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:00.016889  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:00.017437  3876 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.018626  3876 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:01:59.893321  3853 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.019232  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:00.020021  3893 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:00.020334  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.022568  1641 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:00.023854  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:00.001261  1607 connection.cc:256] RPC call timeout handler was delayed by -2.1642255529999943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1925s for a response
I20260709 13:01:59.991055  1568 rpcz_store.cc:276] 0709 13:01:56.536661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.945389 (+1408728us) service_pool.cc:224] Handling call
0709 13:01:57.945976 (+   587us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:57.946482 (+   506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.946506 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.717392 (+1770886us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b2c00000658. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:01:59.717847 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.768664 (+ 50817us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:59.973271 (+204607us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.973637 (+   366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.973756 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.973813 (+    57us) raft_consensus.cc:1685] finished
0709 13:01:59.973943 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.974109 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18575,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":123155,"raft.run_cpu_time_us":595,"raft.run_wall_time_us":59631,"spinlock_wait_cycles":234776320,"thread_start_us":236950,"threads_started":2,"wal-append.queue_time_us":75463}
I20260709 13:02:00.034250  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:00.015142  1560 raft_consensus.cc:2410] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:02:00.038540  1607 connection.cc:256] RPC call timeout handler was delayed by -2.1552255530000366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1935s for a response
W20260709 13:02:00.040315  1607 connection.cc:256] RPC call timeout handler was delayed by -1.983325552999986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2126s for a response
W20260709 13:02:00.040866  1607 connection.cc:256] RPC call timeout handler was delayed by -1.9788255530000356s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
W20260709 13:02:00.041531  1607 connection.cc:256] RPC call timeout handler was delayed by -1.950025552999989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2163s for a response
I20260709 13:02:00.039469  1640 raft_consensus.cc:2376] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4. Re-sending same reply.
I20260709 13:01:59.997936  3615 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:00.032476  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:00.046633  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.060740  1637 raft_consensus.cc:2410] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:00.060961  3853 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:00.061188  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:00.061641  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.034789  3891 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:00.062547  1637 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:02:00.011076  3508 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:01:59.990300  3889 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:00.044777  1635 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:00.063444  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:01:59.989351  3665 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:00.064368  1710 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:00.064663  3889 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:00.065270  1714 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:00.065732  1714 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:00.065824  3665 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:00.051043  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2115) took 3551 ms. Trace:
I20260709 13:02:00.066591  1714 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.066501  1569 rpcz_store.cc:276] 0709 13:01:56.476172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.944027 (+1467855us) service_pool.cc:224] Handling call
0709 13:01:57.944946 (+   919us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:57.945533 (+   587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.945598 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:58.375213 (+429615us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b2c00000658. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:01:58.375795 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:01:58.444115 (+ 68320us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:58.444150 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.447645 (+  3495us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.966679 (+1519034us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:00.026674 (+ 59995us) raft_consensus.cc:1685] finished
0709 13:02:00.026705 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.026890 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270120576,"thread_start_us":67889,"threads_started":1,"wal-append.queue_time_us":119857}
I20260709 13:02:00.067734  1714 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:00.068207  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:00.069131  1718 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:00.069957  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:00.052842  3824 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:00.053491  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.053803  3615 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:00.071300  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:00.071893  1713 raft_consensus.cc:2376] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4. Re-sending same reply.
I20260709 13:02:00.072845  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:00.055509  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.074051  3855 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.074249  3896 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:00.075431  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.075628  1639 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:00.053843  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:00.076901  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:00.077425  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 85855.5us
I20260709 13:02:00.078053  3824 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:00.078503  3824 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:00.078948  3824 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:00.079675  1532 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:00.074226  1713 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:00.080606  3863 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:00.081382  3863 raft_consensus.cc:697] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:00.083348  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.084122  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:00.084638  1637 raft_consensus.cc:2393] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:02:00.084980  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1632) took 2117 ms (client timeout 1806 ms). Trace:
W20260709 13:02:00.085379  1642 rpcz_store.cc:269] 0709 13:01:57.967338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.339721 (+372383us) service_pool.cc:224] Handling call
0709 13:02:00.084018 (+1744297us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ceee 0000000000375939 00007fd7a6bf01e0 00007fd7a6cb90c4 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:02:00.084913 (+   895us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38535040}
I20260709 13:02:00.035401  1711 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
W20260709 13:02:00.087811  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 366) took 2092 ms (client timeout 2021 ms). Trace:
W20260709 13:02:00.088246  1566 rpcz_store.cc:269] 0709 13:01:57.995883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.353027 (+357144us) service_pool.cc:224] Handling call
0709 13:02:00.087637 (+1734610us) spinlock_profiling.cc:243] Waited 76 ms on lock 0x7b54002a9c54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:00.087740 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174741888}
I20260709 13:02:00.089066  1711 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:00.051326  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:00.090703  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.091717  3891 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:00.091924  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:00.092090  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.093287  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:00.093413  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.094357  3824 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.070022  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:00.059695  3900 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:00.063534  3508 raft_consensus.cc:3055] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:00.095837  3508 raft_consensus.cc:740] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:00.096076  1715 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:02:00.096819  3508 consensus_queue.cc:260] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.098217  3508 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:00.098459  2268 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:00.047312  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 49133.6us
I20260709 13:02:00.100698  1681 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:00.101019  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1569) took 2605 ms (client timeout 2824 ms). Trace:
W20260709 13:02:00.101475  1716 rpcz_store.cc:269] 0709 13:01:57.495412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.495555 (+   143us) service_pool.cc:224] Handling call
0709 13:02:00.100786 (+2605231us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b5400474354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:00.100960 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1232046848}
I20260709 13:02:00.102856  3900 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:00.084172  1636 raft_consensus.cc:2410] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:00.057192  3893 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:00.103524  3900 raft_consensus.cc:697] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:00.104182  3893 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:00.095439  1718 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:00.104830  3900 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.105641  1532 leader_election.cc:400] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3.
I20260709 13:02:00.106055  1532 leader_election.cc:403] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:02:00.107108  3820 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:00.086692  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:00.107797  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 439) took 2329 ms (client timeout 2151 ms). Trace:
W20260709 13:02:00.108366  2301 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 692) took 2622 ms (client timeout 2415 ms). Trace:
W20260709 13:02:00.108779  2301 rpcz_store.cc:269] 0709 13:01:57.485701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.108161 (+2622460us) service_pool.cc:211] Skipping call since client already timed out
0709 13:02:00.108304 (+   143us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:02:00.109728  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:00.110442  2301 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:00.108230  1709 rpcz_store.cc:269] 0709 13:01:57.778439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.107575 (+2329136us) service_pool.cc:211] Skipping call since client already timed out
0709 13:02:00.107718 (+   143us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:02:00.065204  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:00.085547  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.077545  2305 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:00.113445  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:00.076406  1643 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:00.115813  1718 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:00.120913  2301 raft_consensus.cc:2410] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:00.094652  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:00.129057  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:00.146270  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:00.151024  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.158802  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:00.160879  1642 raft_consensus.cc:2393] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:00.158397  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 83579.2us
I20260709 13:02:00.186302  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:00.194597  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:00.197222  1568 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:00.198664  1568 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:02:00.199630  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.199376  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:00.200764  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:00.201602  1566 raft_consensus.cc:2410] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:00.194089  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.194249  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:00.203644  1533 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:00.204708  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:00.205621  1569 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:00.207962  3824 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:00.208021  3816 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:00.208680  3802 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:00.209137  3802 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:00.209501  3802 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:00.159734  3896 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:00.214453  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 56128.4us
I20260709 13:02:00.222675  1714 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:02:00.226529  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 112729us
W20260709 13:02:00.227080  3637 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.716s	user 0.018s	sys 0.041s
I20260709 13:02:00.236788  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.237958  1642 raft_consensus.cc:2393] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:00.211037  1711 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:00.243299  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:00.082492  3863 consensus_queue.cc:237] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:00.217310  3567 raft_consensus.cc:697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:00.245651  3637 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.759s	user 0.019s	sys 0.041s
I20260709 13:02:00.263168  2268 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:00.230417  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.272454  3640 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:00.275126  1563 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:00.248181  3615 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:00.276899  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2119) took 3739 ms. Trace:
I20260709 13:02:00.023134  2300 rpcz_store.cc:276] 0709 13:01:57.486177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.721565 (+2235388us) service_pool.cc:224] Handling call
0709 13:01:59.722088 (+   523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.743640 (+ 21552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.746896 (+  3256us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.747510 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.782489 (+ 34979us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:59.892929 (+110440us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.002466 (+109537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.002572 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.002616 (+    44us) raft_consensus.cc:1685] finished
0709 13:02:00.002630 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.002778 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":204131,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":120500,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":5223,"thread_start_us":194371,"threads_started":3,"wal-append.queue_time_us":47644}
I20260709 13:02:00.279172  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 380) took 2218 ms. Trace:
W20260709 13:02:00.307240  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.279064  3802 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.239394  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:00.299672  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:00.305249  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:00.308305  2305 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:00.313220  1563 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:00.307570  1567 rpcz_store.cc:276] 0709 13:01:58.060957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.991945 (+1930988us) service_pool.cc:224] Handling call
0709 13:01:59.992306 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.009182 (+ 16876us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.009217 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.009777 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.190880 (+181103us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:00.190963 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.191064 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.278864 (+ 87800us) raft_consensus.cc:1685] finished
0709 13:02:00.278891 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.279096 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14212,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":108,"thread_start_us":180160,"threads_started":1,"wal-append.queue_time_us":266982}
I20260709 13:02:00.227730  3896 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:00.317726  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 73154.1us
I20260709 13:02:00.290628  1564 rpcz_store.cc:276] 0709 13:01:56.537450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.983082 (+1445632us) service_pool.cc:224] Handling call
0709 13:01:57.983725 (+   643us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:57.984434 (+   709us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.984467 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.011189 (+2026722us) spinlock_profiling.cc:243] Waited 62 ms on lock 0x7b2c00000658. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:02:00.011685 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.115294 (+103609us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:00.184156 (+ 68862us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.185122 (+   966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.185194 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.276475 (+ 91281us) raft_consensus.cc:1685] finished
0709 13:02:00.276588 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.276716 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23077,"prepare.queue_time_us":47450,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":1710,"raft.queue_time_us":98847,"raft.run_cpu_time_us":432,"raft.run_wall_time_us":4366,"spinlock_wait_cycles":142506880,"thread_start_us":103182,"threads_started":1,"wal-append.queue_time_us":262152}
I20260709 13:02:00.324369  3921 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:00.324527  3567 consensus_queue.cc:237] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:00.325901  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:00.326282  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 75642.4us
I20260709 13:02:00.328154  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:00.329085  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:00.332185  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 379) took 2271 ms. Trace:
I20260709 13:02:00.332166  3895 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:02:00.332868  3802 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:00.332769  1565 rpcz_store.cc:276] 0709 13:01:58.060616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.991626 (+1931010us) service_pool.cc:224] Handling call
0709 13:01:59.991966 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.038485 (+ 46519us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:00.054035 (+ 15550us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.192007 (+137972us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.202025 (+ 10018us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.202163 (+   138us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:00.202410 (+   247us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.220882 (+ 18472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.221000 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.331920 (+110920us) raft_consensus.cc:1685] finished
0709 13:02:00.331939 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.332124 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32027,"prepare.queue_time_us":16476,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":4660,"raft.queue_time_us":28930,"raft.run_cpu_time_us":498,"raft.run_wall_time_us":49704,"spinlock_wait_cycles":46256896,"thread_start_us":137478,"threads_started":1,"wal-append.queue_time_us":196147}
W20260709 13:02:00.336417  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.336894  1563 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:00.337872  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:00.342198  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 696) took 2855 ms. Trace:
I20260709 13:02:00.357082  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:00.359292  1609 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:00.359597  1533 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:00.342800  2303 rpcz_store.cc:276] 0709 13:01:57.488858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.731699 (+2242841us) service_pool.cc:224] Handling call
0709 13:01:59.771258 (+ 39559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:59.849172 (+ 77914us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.849219 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.849836 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.119704 (+269868us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:00.147929 (+ 28225us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.148227 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.148333 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.341848 (+193515us) raft_consensus.cc:1685] finished
0709 13:02:00.341880 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.342060 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27902,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":4521,"raft.queue_time_us":169380,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":21345,"spinlock_wait_cycles":32309632,"thread_start_us":252236,"threads_started":2,"wal-append.queue_time_us":331876}
I20260709 13:02:00.364980  1636 raft_consensus.cc:2410] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:00.364817  1711 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:00.368180  1564 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:00.369941  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:00.370414  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:00.370903  2300 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:00.371097  2302 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:00.371876  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:00.372519  2300 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:02:00.375099  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 46090.5us
W20260709 13:02:00.378931  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.379519  3802 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:00.380722  3855 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:00.325783  3923 log.cc:927] Time spent T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:02:00.383845  3926 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:00.384413  3926 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:00.384827  3926 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:00.386282  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:00.378708  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:00.388002  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 70452.5us
I20260709 13:02:00.394953  1681 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:00.398311  2268 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:00.400959  2302 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:00.401260  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:00.403817  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:00.411237  1681 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:00.413779  2303 raft_consensus.cc:2393] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 6: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:00.399945  3508 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:02:00.417896  3691 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:00.355986  3933 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:00.415140  3665 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:00.413709  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:00.423133  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 63853.6us
I20260709 13:02:00.387956  3743 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:00.400578  3912 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.555s	user 0.003s	sys 0.000s
W20260709 13:02:00.503878  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 105679us
I20260709 13:02:00.429574  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:00.418691  3778 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:00.400066  3820 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3.
I20260709 13:02:00.413049  3802 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:00.396799  3853 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:00.516147  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:00.523900  1533 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:00.516955  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:00.519366  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:00.525871  1644 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:00.519627  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.400770  3855 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:00.520143  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:00.528872  2302 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:00.528540  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:00.532266  1643 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:00.522830  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:01:59.603159  1638 rpcz_store.cc:276] 0709 13:01:56.029964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.701274 (+2671310us) service_pool.cc:224] Handling call
0709 13:01:58.951534 (+250260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.022494 (+ 70960us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:59.462351 (+439857us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.562745 (+100394us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:59.562824 (+    79us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:59.562890 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.563235 (+   345us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.563418 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14326,"prepare.queue_time_us":279230,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":35950,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":31329,"thread_start_us":416829,"threads_started":2,"wal-append.queue_time_us":81434}
W20260709 13:02:00.555749  2304 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 699) took 3066 ms (client timeout 2223 ms). Trace:
W20260709 13:02:00.556354  2304 rpcz_store.cc:269] 0709 13:01:57.489678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.528416 (+3038738us) service_pool.cc:211] Skipping call since client already timed out
0709 13:02:00.555660 (+ 27244us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:02:00.557384  1567 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:00.559413  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:02:00.561421  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.563036  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:00.563776  1637 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:00.564173  3850 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:00.566766  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.565058  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 61728.2us
I20260709 13:02:00.595491  1533 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:00.596422  1635 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:00.597513  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:00.597581  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:00.598124  1641 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:00.566022  3853 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:00.527077  3778 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:00.599581  1642 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:00.566987  3684 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:00.615742  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:00.542150  3896 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:00.570539  2298 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:02:00.555835  3802 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:02:00.572153  2296 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.000s
I20260709 13:02:00.575034  3508 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:00.575771  2302 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:00.576472  3701 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.576790  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:00.579061  3824 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:00.583959  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.673424  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.673794  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.607084  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.587194  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:00.519408  1636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:60370
I20260709 13:02:00.599301  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:00.599627  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 82807.9us
I20260709 13:02:00.600281  3778 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.620287  1643 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:00.653497  1644 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:00.663723  3926 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.665094  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:00.668289  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 69572.3us
I20260709 13:02:00.669227  1638 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:02:00.585611  2299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 713) took 2928 ms (client timeout 2213 ms). Trace:
I20260709 13:02:00.682137  1642 raft_consensus.cc:2368] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5.
I20260709 13:02:00.686107  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:00.689649  1533 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:00.696709  1566 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:00.666929  2296 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:00.700404  1643 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:02:00.692075  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:00.703459  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.676419  3508 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:00.690232  3900 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.008s	sys 0.036s
I20260709 13:02:00.711253  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:00.687713  3824 raft_consensus.cc:697] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:00.703202  2304 raft_consensus.cc:2376] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4. Re-sending same reply.
I20260709 13:02:00.692330  1636 raft_consensus.cc:606] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Received request to transfer leadership
I20260709 13:02:00.599620  3853 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.705821  1644 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:02:00.716972  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.672565  3684 consensus_queue.cc:1059] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:02:00.732249  1607 leader_election.cc:400] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4.
I20260709 13:02:00.732717  1607 leader_election.cc:403] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:00.689321  1635 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:02:00.734458  1638 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:00.721724  1641 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:00.728331  1533 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:00.724457  1609 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:00.712291  1637 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 4: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:00.712273  3896 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:00.691375  2299 rpcz_store.cc:269] 0709 13:01:57.630875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.519999 (+2889124us) service_pool.cc:211] Skipping call since client already timed out
0709 13:02:00.559405 (+ 39406us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:02:00.680348  3701 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:00.722668  3773 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:00.762141  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 49685.7us
W20260709 13:02:00.725857  3900 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.008s	sys 0.037s
I20260709 13:02:00.750777  2302 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:00.766469  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:00.743295  3889 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:00.764088  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 188114us
I20260709 13:02:00.773012  2269 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:00.769464  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:00.762995  3891 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4.
I20260709 13:02:00.778086  1717 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:00.786412  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:00.785667  3896 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.767886  3615 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:00.763402  2300 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:00.812638  2302 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:02:00.787575  3743 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:00.741850  3802 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.782436  2298 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:02:00.820948  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 256447us
I20260709 13:02:00.803851  2296 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:00.775647  3773 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:00.836593  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2106) took 4625 ms. Trace:
I20260709 13:02:00.836930  3900 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:00.837697  3900 raft_consensus.cc:697] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:00.837149  1561 rpcz_store.cc:276] 0709 13:01:56.211603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.986030 (+1774427us) service_pool.cc:224] Handling call
0709 13:01:57.986662 (+   632us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:57.987274 (+   612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.987307 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:59.665532 (+1678225us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b48000030d8. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a6bcaaa1 00007fd7a6bc1b37 00007fd7a6bc19ca 00007fd7a6cb926f 00007fd7a6cb8f37 00007fd7a6ccd55c 00007fd7a6ccd508 00007fd7a6ccd488 00007fd7a6ccd443 00007fd7a6ccc6ec 00007fd7a5bf6a96
0709 13:02:00.429358 (+763826us) spinlock_profiling.cc:243] Waited 52.4 ms on lock 0x7b2c00000658. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:02:00.541534 (+112176us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.701041 (+159507us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:00.707391 (+  6350us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.722522 (+ 15131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.722800 (+   278us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.836137 (+113337us) raft_consensus.cc:1685] finished
0709 13:02:00.836165 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.836387 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14807,"prepare.queue_time_us":8154,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":5236,"raft.queue_time_us":116711,"raft.run_cpu_time_us":20133,"raft.run_wall_time_us":1592853,"spinlock_wait_cycles":201667968,"thread_start_us":159088,"threads_started":1,"wal-append.queue_time_us":292188}
I20260709 13:02:00.841481  1561 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:00.843488  1561 pending_rounds.cc:85] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 6
I20260709 13:02:00.843850  1561 pending_rounds.cc:107] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:02:00.844122  1561 raft_consensus.cc:2889] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:00.814335  3855 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:00.880095  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:00.824496  2299 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:00.821733  3867 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:00.781287  3508 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:00.779304  3824 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:00.812880  3896 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:00.909246  1566 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:00.886623  2297 raft_consensus.cc:2410] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:00.920773  3701 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:00.916445  1566 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:02:00.940922  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.948822  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 149306us
W20260709 13:02:00.927273  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 154353us
W20260709 13:02:00.937116  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 90658.6us
I20260709 13:02:00.825990  3891 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:02:00.910508  3926 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:00.940686  3684 consensus_queue.cc:1243] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:02:00.928495  3855 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:01.047362  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.910427  3900 consensus_queue.cc:237] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.067569  1560 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:02:01.064934  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:00.982616  3743 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:01.072786  1566 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:00.986756  3933 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:01.076704  1637 pending_rounds.cc:85] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 3
W20260709 13:02:01.068964  3942 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 13:02:01.091048  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 328152us
I20260709 13:02:00.986307  3944 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:01.070091  3891 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.112738  1637 pending_rounds.cc:107] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:02:01.097788  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:01.098789  3889 raft_consensus.cc:697] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:01.123360  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 78076.5us
I20260709 13:02:01.117575  3944 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:01.152189  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:01.121588  1637 raft_consensus.cc:2889] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:01.157359  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:01.161141  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 102573us
I20260709 13:02:01.118821  3743 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.167413  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:01.146600  1636 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:01.173205  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:01.173472  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.175987  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:01.177032  3947 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.150642  3944 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.158463  3945 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:02:01.191299  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.126320  3949 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
W20260709 13:02:01.198817  1712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 1833) took 1339 ms (client timeout 1590 ms). Trace:
W20260709 13:02:01.200351  1712 rpcz_store.cc:269] 0709 13:01:59.859208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.894235 (+ 35027us) service_pool.cc:224] Handling call
0709 13:02:01.175809 (+1281574us) spinlock_profiling.cc:243] Waited 26.8 ms on lock 0x7b58001e4598. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:02:01.198595 (+ 22786us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61619072}
I20260709 13:02:01.123281  3891 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.177078  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2131) took 1031 ms. Trace:
I20260709 13:02:01.207036  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:01.206678  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:01.201442  3948 log.cc:927] Time spent T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:02:01.212533  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:01.203590  3945 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.202775  3896 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:01.204667  3926 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:02:01.261157  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 54544.1us
I20260709 13:02:01.204931  3954 raft_consensus.cc:2764] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:02:01.276252  1452 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 171929us
W20260709 13:02:01.277828  3950 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:02:01.280532  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:01.281353  1568 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.283438  1636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:60370
I20260709 13:02:01.284307  1636 raft_consensus.cc:606] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Received request to transfer leadership
I20260709 13:02:01.285923  3702 raft_consensus.cc:993] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: : Instructing follower 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:02:01.286475  3702 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.287734  3951 raft_consensus.cc:1081] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Signalling peer 28bc59d37f664ce0a6f5784f2619f0b5 to start an election
I20260709 13:02:01.216470  3802 raft_consensus.cc:3055] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:01.291783  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:01.294688  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 127386us
I20260709 13:02:01.217876  3958 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.267292  3850 consensus_meta.cc:287] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.671s	user 0.012s	sys 0.000s
I20260709 13:02:01.209499  3855 consensus_queue.cc:1059] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 13:02:01.302592  1643 raft_consensus.cc:3055] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:01.307464  1643 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:01.260489  3945 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.258876  3820 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:01.289459  3508 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.321224  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:01.203835  3947 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.290033  3953 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:01.292866  3802 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:01.309784  3850 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:01.337970  3947 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:01.323982  1643 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.294637  3702 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.340458  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:01.333565  3508 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.341379  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 566) took 1291 ms. Trace:
W20260709 13:02:01.342619  2297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 749) took 4682 ms (client timeout 3154 ms). Trace:
I20260709 13:02:01.345659  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:01.217069  3889 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.348186  1635 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:01.298003  3946 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.316505  3820 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.365446  1638 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:01.369043  1568 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
W20260709 13:02:01.371204  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.372846  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:01.379611  1561 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:01.374893  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 64681.2us
I20260709 13:02:01.376269  1641 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
W20260709 13:02:01.376518  1534 leader_election.cc:336] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.556s (SENT)
I20260709 13:02:01.381376  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:01.382289  3891 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:01.382699  3802 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.384348  3802 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:01.316118  3958 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.386487  3702 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:01.387845  3850 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:01.388125  3508 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:01.388442  3850 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.388906  3850 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.208928  1560 rpcz_store.cc:276] 0709 13:02:00.145703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.184295 (+ 38592us) service_pool.cc:224] Handling call
0709 13:02:01.176834 (+992539us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54000f5d54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:01.176980 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":586517760}
I20260709 13:02:01.339490  3743 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:01.367014  2296 rpcz_store.cc:276] 0709 13:02:00.050142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.729797 (+679655us) service_pool.cc:224] Handling call
0709 13:02:00.779778 (+ 49981us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.842995 (+ 63217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.341077 (+498082us) spinlock_profiling.cc:243] Waited 50.2 ms on lock 0x7b54005175a0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c18dc0 00007fd7a1ca4a02 00007fd7a1ca0dd6 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:02:01.341315 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115457280}
I20260709 13:02:01.403066  3971 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:01.387209  3820 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:01.366772  2297 rpcz_store.cc:269] 0709 13:01:56.660628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.741852 (+3081224us) service_pool.cc:224] Handling call
0709 13:02:01.342386 (+1600534us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400661554. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:01.342540 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2411709312}
I20260709 13:02:01.408044  3877 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:01.410281  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:01.400137  3946 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:01.416970  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:01.419227  1568 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:01.420666  1568 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:02:01.422089  1568 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:02:01.418056  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 126361us
I20260709 13:02:01.423332  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:01.427086  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:01.419001  3877 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:01.433394  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:01.427428  1637 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:01.428833  3855 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:01.438807  3953 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:01.436290  3971 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.436779  3973 raft_consensus.cc:2764] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:02:01.446844  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:01.448886  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:01.446756  3701 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.449637  3971 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.451506  3701 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.454248  3701 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:01.456327  3508 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.456390  1534 leader_election.cc:336] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.636s (SENT)
I20260709 13:02:01.475455  1534 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:01.466807  3945 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.469755  1560 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:01.471042  3665 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.477927  1561 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:01.479504  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:01.490968  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:01.485109  3889 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.492687  3889 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.458441  3947 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:01.495245  3889 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:01.485648  3508 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.485945  3953 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:01.499698  1561 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:01.500572  3508 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:01.488385  3607 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.501394  3607 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.490685  1568 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:01.503595  3607 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:01.503499  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:01.504456  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 1837) took 1603 ms (client timeout 1797 ms). Trace:
I20260709 13:02:01.504597  1640 raft_consensus.cc:2368] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3.
W20260709 13:02:01.504914  1717 rpcz_store.cc:269] 0709 13:01:59.901065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.929589 (+ 28524us) service_pool.cc:224] Handling call
0709 13:02:00.691885 (+762296us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b58001e4598. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:02:01.504314 (+812429us) spinlock_profiling.cc:243] Waited 73.1 ms on lock 0x7b5400268ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:01.504405 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198862336}
I20260709 13:02:01.505640  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2147) took 1111 ms. Trace:
I20260709 13:02:01.508365  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1669) took 1313 ms. Trace:
I20260709 13:02:01.508738  1640 rpcz_store.cc:276] 0709 13:02:00.194572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.203215 (+  8643us) service_pool.cc:224] Handling call
0709 13:02:01.503360 (+1300145us) spinlock_profiling.cc:243] Waited 6.07 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:02:01.508320 (+  4960us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13966720}
W20260709 13:02:01.509969  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 65147.8us
W20260709 13:02:01.511495  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 55171.1us
I20260709 13:02:01.507320  3508 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.489698  3743 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:01.516600  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:01.517489  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:01.512409  3889 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:01.518838  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:01.521435  3946 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:01.522140  3975 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:01.524658  3773 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:01.516956  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:01.449967  3877 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.515986  3958 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:01.506469  3665 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.528717  3947 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:01.518217  3508 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.529322  3877 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.534595  3665 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:01.534518  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1799) took 1151 ms. Trace:
I20260709 13:02:01.535276  1567 rpcz_store.cc:276] 0709 13:02:00.382894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.425157 (+ 42263us) service_pool.cc:224] Handling call
0709 13:02:01.534326 (+1109169us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b54002c5e54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:01.534455 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76680576}
I20260709 13:02:01.532809  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:01.541180  3691 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:01.541657  3691 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:01.543411  3975 raft_consensus.cc:1064] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
W20260709 13:02:01.543916  3975 raft_consensus.cc:1068] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Unable to promote non-voter 6208c2fa49f643b5a50ab8f6d292fab2: Service unavailable: leader transfer in progress
W20260709 13:02:01.545609  3974 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.130s	user 0.002s	sys 0.000s
I20260709 13:02:01.546002  3953 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:01.547156  1637 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:01.530720  3945 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:01.548337  3945 consensus_queue.cc:1059] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:01.549418  3945 raft_consensus.cc:3055] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:01.542553  3691 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.550706  3691 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.556917  3973 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:01.557626  3973 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.559072  3963 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.559005  3850 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.559913  3963 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.560667  1638 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 6: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:01.561887  3850 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:01.562448  3963 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:01.558076  3508 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:01.538429  3944 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.549793  3945 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:02:01.579797  1534 leader_election.cc:336] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.758s (SENT)
I20260709 13:02:01.580587  1534 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:01.582069  3640 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:02:01.568745  3947 consensus_queue.cc:1175] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" responder_term: 6 status { last_received { term: 4 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:02:01.584978  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.585521  3944 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:01.591035  1534 leader_election.cc:336] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.770s (SENT)
W20260709 13:02:01.597600  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 64865.1us
I20260709 13:02:01.598305  3953 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:02:01.604965  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.612583  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:01.615229  3958 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:01.615435  2301 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 559) took 1695 ms (client timeout 2197 ms). Trace:
W20260709 13:02:01.615895  2301 rpcz_store.cc:269] 0709 13:01:59.919563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.113171 (+193608us) service_pool.cc:224] Handling call
0709 13:02:01.614208 (+1501037us) spinlock_profiling.cc:243] Waited 948 us on lock 0x7b54005ce254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:01.615356 (+  1148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2181376}
I20260709 13:02:01.616636  3945 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:01.617858  1607 leader_election.cc:336] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.590s (SENT)
I20260709 13:02:01.618310  3945 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:01.619941  3953 consensus_queue.cc:1059] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:02:01.623625  3973 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:01.624161  1464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60118 (request call id 228) took 12899 ms (client timeout 15000 ms). Trace:
I20260709 13:02:01.625461  3508 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.632555  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:01.635092  1534 leader_election.cc:336] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.780s (SENT)
I20260709 13:02:01.637581  1534 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:01.637270  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 84924.6us
I20260709 13:02:01.610730  3820 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.645134  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:01.645417  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:01.645893  1715 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:01.646524  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:01.645155  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:01.647055  1713 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:01.647336  1709 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:01.647737  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:01.648257  1718 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:01.648610  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.652983  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:01.653883  1638 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:01.654063  3538 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:01.655315  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:01.655774  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:01.655953  1642 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.656352  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.656987  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:01.657371  2297 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:02:01.657764  2297 pending_rounds.cc:85] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 7
I20260709 13:02:01.657920  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.658118  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:01.658727  1565 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.659479  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.660306  2296 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:01.660216  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.660904  2304 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:02:01.661830  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:01.655350  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.663542  1561 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:01.663232  3508 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.665778  3508 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:01.666563  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:01.666626  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.667073  1565 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:01.667461  1561 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:02:01.667675  2296 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:01.657996  2297 pending_rounds.cc:107] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:02:01.668604  2297 raft_consensus.cc:2889] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:01.668598  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:01.669008  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.669615  1644 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:01.674077  1533 leader_election.cc:400] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3.
I20260709 13:02:01.692382  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:01.692688  1569 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:01.693176  1562 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.693316  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2139) took 1481 ms. Trace:
I20260709 13:02:01.693761  1569 rpcz_store.cc:276] 0709 13:02:00.211871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.212050 (+   179us) service_pool.cc:224] Handling call
0709 13:02:01.692093 (+1480043us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:02:01.692327 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.693044 (+   717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.693252 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7676672}
I20260709 13:02:01.692699  3958 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:02:01.692698  1638 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:01.671880  3640 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:01.670560  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.698669  3878 raft_consensus.cc:3055] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:01.699172  3878 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:01.700109  3971 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:01.700429  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:01.700799  3816 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.701349  3816 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.702304  1562 raft_consensus.cc:2410] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:02:01.702315  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:01.702924  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2136) took 1518 ms. Trace:
I20260709 13:02:01.703480  1636 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:01.703495  1562 rpcz_store.cc:276] 0709 13:02:00.184643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.193015 (+  8372us) service_pool.cc:224] Handling call
0709 13:02:01.692238 (+1499223us) spinlock_profiling.cc:243] Waited 7.81 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:02:01.702862 (+ 10624us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17959168}
I20260709 13:02:01.706687  3891 raft_consensus.cc:3055] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:01.707243  3891 raft_consensus.cc:740] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:01.708369  3891 consensus_queue.cc:260] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.708915  1636 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:01.700387  3878 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.709972  3891 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:01.710361  3878 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.670563  3953 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.713649  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.713519  3953 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.715615  1532 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:01.716039  3953 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:01.712877  1534 leader_election.cc:336] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.852s (SENT)
I20260709 13:02:01.716789  1534 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:01.717074  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:01.717320  3538 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.717638  3863 consensus_queue.cc:1059] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:01.717932  3538 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.720407  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:01.721226  1569 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
W20260709 13:02:01.626662  1464 rpcz_store.cc:269] 0709 13:01:48.724576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.724783 (+   207us) service_pool.cc:224] Handling call
0709 13:02:01.623884 (+12899101us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:51.244478 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:51.245330 (+   852us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:51.245379 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:01:51.245419 (+    40us) tablet.cc:662] Decoding operations
0709 13:01:51.246396 (+   977us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:51.246478 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:51.246523 (+    45us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:01:51.251608 (+  5085us) tablet.cc:701] Row locks acquired
0709 13:01:51.251638 (+    30us) write_op.cc:260] PREPARE: finished
0709 13:01:51.251852 (+   214us) write_op.cc:270] Start()
0709 13:01:51.252021 (+   169us) write_op.cc:276] Timestamp: P: 1783602111251783 usec, L: 0
0709 13:01:51.252053 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:01:51.364762 (+112709us) log.cc:844] Serialized 2069 byte log entry
0709 13:01:53.625598 (+2260836us) op_driver.cc:464] REPLICATION: finished
0709 13:01:54.507991 (+882393us) write_op.cc:301] APPLY: starting
0709 13:01:54.508116 (+   125us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:54.508352 (+   236us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:54.508534 (+   182us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:54.509549 (+  1015us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:54.509855 (+   306us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:01:54.509924 (+    69us) write_op.cc:312] APPLY: finished
0709 13:01:54.510250 (+   326us) log.cc:844] Serialized 47 byte log entry
0709 13:01:54.971243 (+460993us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:54.994890 (+ 23647us) write_op.cc:454] Released schema lock
0709 13:01:55.047277 (+ 52387us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:55.127402 (+ 80125us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:55.142374 (+ 14972us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1160504,"reported_tablets":8,"spinlock_wait_cycles":192768,"child_traces":[["op",{"apply.queue_time_us":882020,"apply.run_cpu_time_us":46491,"apply.run_wall_time_us":590819,"num_ops":5,"prepare.queue_time_us":568358,"prepare.run_cpu_time_us":63074,"prepare.run_wall_time_us":1761530,"raft.queue_time_us":138517,"raft.run_cpu_time_us":38593,"raft.run_wall_time_us":1110723,"replication_time_us":2337362,"thread_start_us":3315661,"threads_started":5,"wal-append.queue_time_us":2311890,"wal-append.run_cpu_time_us":56513,"wal-append.run_wall_time_us":3791428}]]}
I20260709 13:02:01.723317  3816 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:01.663925  2301 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:01.725317  3538 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:01.726279  1638 raft_consensus.cc:2410] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:02:01.726773  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.730850  3850 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.731669  3850 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.732584  1642 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
W20260709 13:02:01.733089  3952 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.525s	user 0.000s	sys 0.006s
I20260709 13:02:01.734485  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.735518  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.735795  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:01.737181  1681 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:02:01.738772  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.739099  3593 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:01.739601  3593 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.740015  3593 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.740568  3863 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:01.740976  3863 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.741317  3863 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.743108  3773 raft_consensus.cc:3055] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:01.743299  1607 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:01.743228  3665 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:01.744153  3971 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:01.745273  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.746341  1607 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:01.743618  3773 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:01.747282  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.747929  2305 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:01.747682  3773 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:01.748281  3876 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:01.748878  3876 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:01.749068  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.749650  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.750536  2297 raft_consensus.cc:2393] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:01.749958  3876 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.749809  2304 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:01.745630  1681 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:01.754376  3977 consensus_queue.cc:1059] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:01.752813  3877 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:02:01.749096  3773 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.757370  3951 raft_consensus.cc:3055] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:01.757784  3951 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:01.750808  3863 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.760931  1718 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:01.761294  1709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:36078
I20260709 13:02:01.761965  1709 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:01.762327  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:01.762775  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:01.763330  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:01.764274  3889 raft_consensus.cc:1064] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
W20260709 13:02:01.764685  3889 raft_consensus.cc:1068] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Unable to promote non-voter 6208c2fa49f643b5a50ab8f6d292fab2: Service unavailable: leader transfer in progress
I20260709 13:02:01.764906  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:01.752513  1532 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:01.766115  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:01.762362  1709 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:01.767484  3896 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:01.767576  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:01.768030  3896 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.768312  1714 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:01.768473  3896 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:01.769357  3855 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:01.770371  3855 consensus_queue.cc:1243] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:02:01.771025  3977 raft_consensus.cc:3055] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:01.769387  3951 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.771432  3977 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:01.772027  3951 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.772392  3977 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:01.671622  1515 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
I20260709 13:02:01.754693  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2146) took 1361 ms. Trace:
W20260709 13:02:01.775368  1515 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for fb294ac919b44e56960745f79e83dfc4 already in progress
W20260709 13:02:01.775717  1515 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for fb294ac919b44e56960745f79e83dfc4 already in progress
I20260709 13:02:01.754894  3743 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.776276  3977 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.776631  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:01.777300  1562 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:01.781081  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 765) took 1636 ms. Trace:
I20260709 13:02:01.783600  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:01.781582  2302 rpcz_store.cc:276] 0709 13:02:00.144994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.813484 (+668490us) service_pool.cc:224] Handling call
0709 13:02:00.823955 (+ 10471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.883110 (+ 59155us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:01.120712 (+237602us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.778607 (+657895us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b180036eaa0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a
0709 13:02:01.779383 (+   776us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.780619 (+  1236us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:01.780681 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:01.780735 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.780828 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.781019 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":304083,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":1382,"raft.queue_time_us":1802,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":137,"spinlock_wait_cycles":96357120,"thread_start_us":265004,"threads_started":2,"wal-append.queue_time_us":228149}
I20260709 13:02:01.785472  3946 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:01.785923  3946 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.786356  3946 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:01.786592  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.786953  1566 rpcz_store.cc:276] 0709 13:02:00.393432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.107194 (+713762us) service_pool.cc:224] Handling call
0709 13:02:01.125249 (+ 18055us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.196023 (+ 70774us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:01.208562 (+ 12539us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.545043 (+336481us) spinlock_profiling.cc:243] Waited 1.32 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:01.726620 (+181577us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b180034bf40. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a
0709 13:02:01.753155 (+ 26535us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:01.753217 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:01.753273 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.754456 (+  1183us) spinlock_profiling.cc:243] Waited 70.5 ms on lock 0x7b54001798d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:01.754502 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.754634 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26205,"prepare.queue_time_us":12138,"prepare.run_cpu_time_us":8937,"prepare.run_wall_time_us":38973,"raft.queue_time_us":30297,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"spinlock_wait_cycles":169556480,"thread_start_us":65449,"threads_started":1,"wal-append.queue_time_us":206721}
I20260709 13:02:01.791572  3778 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:01.793006  1462 catalog_manager.cc:5697] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:01.796573  1682 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:01.784330  3508 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:01.798370  3701 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:01.798794  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:01.784201  1609 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:01.796643  3607 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:01.796342  1462 catalog_manager.cc:5697] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 3, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:01.667106  1640 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:01.673714  2269 leader_election.cc:400] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5.
I20260709 13:02:01.810987  2269 leader_election.cc:403] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:01.506045  1564 rpcz_store.cc:276] 0709 13:02:00.393777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.196133 (+802356us) service_pool.cc:224] Handling call
0709 13:02:01.208630 (+ 12497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.208834 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:01.441946 (+233112us) log.cc:844] Serialized 14 byte log entry
0709 13:02:01.505338 (+ 63392us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:01.505390 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:01.505430 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.505487 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.505590 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":81953,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":40,"raft.queue_time_us":16057,"thread_start_us":230767,"threads_started":2}
W20260709 13:02:01.803563  1534 leader_election.cc:336] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.919s (SENT)
I20260709 13:02:01.804090  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:01.811787  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:01.804342  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 58082.9us
I20260709 13:02:01.803321  1709 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:01.812856  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:01.814759  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:01.812004  1716 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:01.816643  1710 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 7. Ops: []
I20260709 13:02:01.818542  1709 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:01.820611  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 147048us
I20260709 13:02:01.821580  2269 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:01.736862  2296 raft_consensus.cc:2410] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:01.823537  1718 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:01.824393  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.795307  3946 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.829336  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:01.830752  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:01.827220  1462 catalog_manager.cc:5697] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 3, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:01.832038  2301 raft_consensus.cc:2393] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:01.825861  1609 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:01.834358  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:01.835150  1716 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.675868  3973 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.153s
I20260709 13:02:01.798138  3508 raft_consensus.cc:697] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:01.846571  2269 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:01.825174  3538 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:01.840281  1566 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:01.818396  3855 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:01.832979  3863 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:01.851967  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 562) took 1820 ms. Trace:
I20260709 13:02:01.894718  2303 rpcz_store.cc:276] 0709 13:02:00.031866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.700582 (+668716us) service_pool.cc:224] Handling call
0709 13:02:00.710501 (+  9919us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.747611 (+ 37110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.747657 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:00.769110 (+ 21453us) log.cc:844] Serialized 26 byte log entry
0709 13:02:00.928457 (+159347us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:00.928493 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.928562 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:01.732418 (+803856us) spinlock_profiling.cc:243] Waited 4.13 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:01.845170 (+112752us) raft_consensus.cc:1685] finished
0709 13:02:01.845202 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.851901 (+  6699us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9493760,"thread_start_us":125076,"threads_started":1,"wal-append.queue_time_us":375285}
I20260709 13:02:01.796655  3977 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.825261  3947 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5.
I20260709 13:02:01.843741  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:01.860077  2269 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:01.839139  3816 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:01.829340  3702 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:01.847345  1462 catalog_manager.cc:5697] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 3 to 4, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: HEALTHY } } }
I20260709 13:02:01.842651  3865 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:02:01.842106  3900 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:01.864621  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:01.826638  3773 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:01.841168  3743 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:01.902109  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 277364us
W20260709 13:02:01.849198  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 45687.6us
I20260709 13:02:01.891964  3944 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:01.916692  3853 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:01.917313  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:01.917488  3593 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.917943  3816 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:01.918251  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 58313.7us
I20260709 13:02:01.918779  3691 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:01.951499  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:01.849187  3878 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:01.951647  3816 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.952533  3878 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:01.953444  3878 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.923575  2301 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:02:01.924965  1534 leader_election.cc:336] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.930s (SENT)
I20260709 13:02:01.955670  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:01.848702  3889 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:01.841487  3953 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 13:02:01.902696  3850 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:01.864009  3945 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:02:01.971937  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.936786  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:01.936909  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.938210  1462 catalog_manager.cc:5697] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 3, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:01.929783  3944 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.930521  3853 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.979434  1566 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:01.979775  3853 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:01.980329  3896 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:01.981245  2300 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 LEARNER]: Advancing to term 4
I20260709 13:02:01.982585  3896 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:01.940791  1715 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:01.970829  3684 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:01.986274  3684 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.986753  3684 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:01.989560  3853 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.942019  3593 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:01.949862  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.992100  3853 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:01.959961  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:01.992841  1710 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:02:01.950323  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.994133  2298 raft_consensus.cc:2410] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:01.961634  2269 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:01.995675  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:01.929201  3607 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:01.964318  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.937142  3702 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:01.975314  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.914770  3977 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:01.975231  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:01.976260  2305 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:01.976533  2303 raft_consensus.cc:2410] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:01.979066  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:01.969529  3867 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.929970  3820 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:02.044574  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:01.960748  3900 raft_consensus.cc:697] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:01.979471  1462 catalog_manager.cc:5697] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 4, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 4 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:01.979524  3944 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:01.983969  3946 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:01.968008  3889 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:01.985673  3951 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.991956  2299 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:01.892534  3877 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:02:01.997444  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:02.069777  1710 raft_consensus.cc:2368] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5.
I20260709 13:02:02.001865  2301 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:02.006765  1709 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:02.034991  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:02.071213  1718 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:02.072402  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:02.073208  1718 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:01.923918  3778 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:02:02.035038  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:02.002430  3691 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:02.004496  3865 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:02.034629  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:02.046778  3702 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:02.035704  3607 raft_consensus.cc:697] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:02.035859  3926 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.983207  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:02.047485  1716 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:02.047369  1462 catalog_manager.cc:5697] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 3, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:02.111989  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:02.112908  1714 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:02.060660  1568 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:02.063082  1565 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:02:02.063915  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:02.066589  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:02.075439  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:02.004323  3824 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.077s
W20260709 13:02:02.074543  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 230879us
I20260709 13:02:02.119813  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:02.120769  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:02.122109  1607 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:02.081247  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:02.123293  1718 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:02.123826  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 327344us
I20260709 13:02:02.042893  2297 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:02:02.085201  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:02.089941  1643 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:02.081903  3691 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.067368  3889 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:02.004659  3947 consensus_queue.cc:1059] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:02:02.076891  1563 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:02:01.971602  3508 consensus_queue.cc:237] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:02.090431  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:02:02.174675  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.136283  1462 catalog_manager.cc:5697] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 3, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:02.125919  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:02.155623  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:02.183678  1568 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
W20260709 13:02:02.173511  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.184753  1718 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
W20260709 13:02:02.185273  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.127928  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:02.187142  1709 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:02.124574  1682 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:02.189977  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 364192us
I20260709 13:02:02.077448  3951 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:02.115775  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 308835us
I20260709 13:02:02.146651  3891 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:02.224088  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:02.233605  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.205982  1462 catalog_manager.cc:5697] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 2, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:02.242175  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 117689us
I20260709 13:02:02.242674  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:02.243685  1642 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:02.191591  3691 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:02.253793  1643 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:02:02.254362  1643 pending_rounds.cc:85] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 3
I20260709 13:02:02.254846  1643 pending_rounds.cc:107] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:02:02.254778  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:02.248884  1682 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:02.255532  1642 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:02.256853  1533 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:02.257249  1462 catalog_manager.cc:5697] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 2, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:02.226511  3891 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:02.264521  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 304575us
I20260709 13:02:02.255663  1643 raft_consensus.cc:2889] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:02.232136  3947 raft_consensus.cc:3055] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:02.271703  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:02.281189  1639 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:02.280678  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 75394.6us
I20260709 13:02:02.236363  3816 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:02.266701  1462 catalog_manager.cc:5697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 3, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 3 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } }
I20260709 13:02:02.287441  1462 catalog_manager.cc:5697] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 4, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:02.122182  3926 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:02.284224  1681 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:02.205570  3701 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 13:02:02.254302  3853 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:02.270709  3891 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:02.301095  1462 catalog_manager.cc:5697] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 3, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:02.288453  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 250266us
I20260709 13:02:02.255327  3991 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:02.205183  3992 consensus_queue.cc:1059] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.123s
I20260709 13:02:02.310290  1462 catalog_manager.cc:5697] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:02.321026  1639 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:02.284732  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:02.287815  1560 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:02.327801  1566 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:02.280267  3947 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:02.349537  3947 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:02.351258  3947 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:02.315471  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:02.302083  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:02.337957  1642 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:02.331935  1462 catalog_manager.cc:5697] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:02.358692  2268 leader_election.cc:400] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5.
I20260709 13:02:02.361186  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:02.362210  3996 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:02.363600  3701 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:02.364255  3701 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.364677  3701 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:02.364818  3802 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:02.365995  3993 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:02:02.365633  3802 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:02.369616  1462 catalog_manager.cc:5697] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 3 to 5, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:02.371342  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:02.374181  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.267283  3900 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:02.367480  3988 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:02:02.323468  3963 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.372908  1462 catalog_manager.cc:5697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:02.399019  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:02.368590  3989 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:02:02.355912  3593 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:02.368583  3986 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:02:02.328508  3865 raft_consensus.cc:697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:02.303591  3816 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:02.356263  2299 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:02.284047  3702 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:02.417515  2301 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:02.315984  3684 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:02.450668  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 49748.3us
I20260709 13:02:02.374328  3973 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:02.419945  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:02.539832  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 252838us
I20260709 13:02:02.546979  1639 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:02.550693  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:02:02.421319  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 266857us
I20260709 13:02:02.370294  3977 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:02.374658  1637 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:02.400638  2297 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:59006
I20260709 13:02:02.305064  3607 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:02:02.556131  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:02.560858  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 276726us
I20260709 13:02:02.525442  3593 raft_consensus.cc:697] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:02.401903  3945 consensus_queue.cc:1059] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.135s
I20260709 13:02:02.419397  3963 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:02.582122  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.578374  3973 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:02.536782  3824 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:02:02.548066  3975 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:02:02.583422  3802 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:02.579352  2297 raft_consensus.cc:606] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:02:02.582696  3855 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:02.601738  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:02.607906  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 261181us
I20260709 13:02:02.612443  2301 raft_consensus.cc:2410] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:02.591755  3945 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:02.604439  3951 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:02.611579  2297 raft_consensus.cc:614] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:02.617525  3802 raft_consensus.cc:3055] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:02.622835  3997 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:02:02.627795  3926 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:02.649919  3684 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:02.655334  4002 log.cc:927] Time spent T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 13:02:02.656167  3999 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:02:02.620596  3855 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:02.609867  3865 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:02.635164  3802 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:02.676355  3963 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:02.629359  3593 consensus_queue.cc:237] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:02.680711  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 276783us
W20260709 13:02:02.682643  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 107126us
I20260709 13:02:02.691356  3816 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:02.681258  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 106818us
W20260709 13:02:02.693133  2296 consensus_meta.cc:287] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.728s	user 0.003s	sys 0.007s
I20260709 13:02:02.634583  4013 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:02:02.693504  3963 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:02.642699  4019 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:02:02.693534  2296 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:02.695063  4011 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:02:02.696523  3802 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:02.697086  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:02.698030  1640 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:02.698944  4014 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:02.700671  1640 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:02.702939  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:02.704424  3684 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:02.667526  4015 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:02.677641  4003 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:02:02.708408  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:02.709331  3855 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:02.690632  3944 consensus_meta.cc:287] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.609s	user 0.007s	sys 0.000s
I20260709 13:02:02.712990  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:02.715267  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:02.717621  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:02.718443  1636 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:02.719669  3701 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:02.720338  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:02.720997  2299 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:02.721149  1636 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:02.722038  3701 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:02.722147  3855 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:02.739426  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 482712us
I20260709 13:02:02.742316  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:02.743188  3820 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:02.743748  3743 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:02.744364  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 428996us
I20260709 13:02:02.744984  3841 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:02.745639  3640 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:02.749001  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:02.749886  3773 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:02.750854  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:02.752323  1638 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
W20260709 13:02:02.755405  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.795501  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:02.755910  1564 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:02:02.756981  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.792588  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:02.797396  1715 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:02:02.799242  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:02.799866  1710 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:02.757171  1569 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:02.757313  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:02.757800  3863 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.758643  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:02.804751  1644 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:02:02.759325  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 205792us
I20260709 13:02:02.763221  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:02.722777  3855 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:02.767676  2300 consensus_meta.cc:287] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.786s	user 0.011s	sys 0.001s
W20260709 13:02:02.694748  3867 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.640s	user 0.004s	sys 0.003s
I20260709 13:02:02.722350  3944 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:02.706312  3889 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:02:02.772342  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 79055.1us
I20260709 13:02:02.714619  3684 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.778163  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:02.781585  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:02.784723  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 154697us
I20260709 13:02:02.726251  3802 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:02.785174  3538 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:02.792328  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:02.792665  4024 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:02.767834  3665 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:02.810045  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:02.805972  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:02.778599  3820 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.782269  4015 raft_consensus.cc:697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:02.810348  1636 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:02.782501  4014 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:02.834975  1568 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:02.841328  1644 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:02.841434  1682 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:02.843163  2268 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:02.771404  3691 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:02.859449  1641 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:02.833931  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:02.810771  3877 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:02.854212  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:02.864378  4025 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:02.865624  3944 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:02.866096  4025 consensus_queue.cc:1059] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:02.821470  3867 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:02:02.870873  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.794328  3841 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:02.884783  1682 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:02.844894  3665 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:02.867020  4021 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:02.811560  3743 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:02.860905  2300 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:02.895223  1682 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:02.896416  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 90602.6us
I20260709 13:02:02.858608  4014 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:02.867292  3865 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:02.890488  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 76892.3us
I20260709 13:02:02.962224  4027 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:02.963290  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:02.964463  3987 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.738s	user 0.003s	sys 0.000s
I20260709 13:02:02.964625  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:02.967131  1567 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:02.876556  1709 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
W20260709 13:02:02.907709  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 129654us
I20260709 13:02:02.857520  3820 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:02.817474  3863 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.002213  3863 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:02.886147  3891 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.000s
I20260709 13:02:02.898365  1715 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:03.003134  3691 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:02.913741  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:03.004451  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 90789.2us
I20260709 13:02:03.004882  3855 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:03.005858  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:03.006116  3841 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:02.889549  3896 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.007220  3896 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:02.915139  4024 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:03.008664  4024 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.009050  4024 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:03.009493  3896 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:03.009783  3900 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:03.010092  3896 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:03.010560  3900 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:03.018253  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 129797us
I20260709 13:02:02.848464  3684 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:02.893288  4021 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:02.878263  3877 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:02.915580  1637 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:02.982726  3665 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:03.054904  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 141099us
I20260709 13:02:02.996017  1711 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:02.966928  3593 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:02.996366  4014 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:02.998376  3743 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:02.999971  3889 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:03.000371  4027 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:02.902984  3640 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.003201  3891 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.007145  4015 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:03.022621  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:03.022163  1534 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:02.856832  1636 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:03.010758  3896 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
W20260709 13:02:03.024794  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.058477  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 736) took 1107 ms. Trace:
I20260709 13:02:03.034941  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:03.078852  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 44101.3us
W20260709 13:02:03.038053  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 142970us
I20260709 13:02:03.079537  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:03.079663  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:03.040563  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:03.055706  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:02.995639  3867 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:03.002297  3820 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.002847  3863 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:02.967564  3773 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:03.065202  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:03.038934  1635 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:03.034729  3538 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:03.072670  1566 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:03.074467  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:03.090499  1563 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:02.886931  3946 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:03.071810  4040 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:03.060559  3593 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:03.073963  3640 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:03.049242  3900 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:03.093310  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 71229.2us
I20260709 13:02:03.088708  3665 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:03.064157  4033 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:02:03.107842  1563 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:03.112403  1534 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:03.121238  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:03.075531  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:03.083660  3865 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:03.087821  4014 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:03.115016  2268 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:03.090586  3691 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:03.124567  1563 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:03.103129  3946 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:03.109700  3867 raft_consensus.cc:697] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:03.113065  3593 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:03.132438  1681 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:03.137758  1566 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:03.139609  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:03.111457  3863 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.110385  4027 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:03.082159  2300 rpcz_store.cc:276] 0709 13:02:01.948963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.959335 (+ 10372us) service_pool.cc:224] Handling call
0709 13:02:01.981106 (+ 21771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.913138 (+932032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.056738 (+143600us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:03.142825  1563 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:03.147701  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:03.149461  1681 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:03.126503  3865 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.116753  3944 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:03.130052  3945 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:03.156147  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:03.140774  4041 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:03.156716  1563 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:02:03.128219  4043 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:03.157809  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:03.205435  1563 raft_consensus.cc:606] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:03.234519  1563 raft_consensus.cc:614] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:03.235051  4023 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:03.235379  3889 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:03.236004  3889 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.236588  3889 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:03.161937  4025 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:03.150894  4019 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:03.159081  4027 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.219092  3684 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:03.106050  4021 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.223335  3867 consensus_queue.cc:237] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:03.224282  3640 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:03.225085  4041 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:03.145864  3773 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:03.227084  4048 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:03.227293  3896 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:03.227696  3946 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:03.153784  4047 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 13:02:03.231514  3877 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:03.158469  3538 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:03.321836  4024 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.302691  4021 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:03.294303  4025 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:03.327879  4025 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:03.162168  1560 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 6: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:03.330567  1560 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:03.164973  3944 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.332386  4043 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.332996  4043 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:03.333773  3944 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:03.216195  4012 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:03.334836  4012 raft_consensus.cc:697] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:03.336057  4012 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:03.248695  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 208228us
I20260709 13:02:03.153518  3865 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:03.142493  3607 consensus_queue.cc:1059] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.097s
W20260709 13:02:03.294564  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:03.295780  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 61782.8us
I20260709 13:02:03.348197  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:03.235047  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 77355.3us
I20260709 13:02:03.314949  4047 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:03.316869  4027 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:03.299747  3896 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:03.302178  3640 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.340600  1683 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:03.288401  3945 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:03.137123  1636 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:03.352290  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:03.154202  3863 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:03.349084  1565 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:02:03.344656  3876 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.352849  4047 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:03.376044  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 270297us
W20260709 13:02:03.381117  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.413278  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:03.350723  3944 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.351146  3802 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.161872  3824 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.189s
W20260709 13:02:03.344049  4051 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.132s	user 0.002s	sys 0.000s
I20260709 13:02:03.303002  3820 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:03.357226  3896 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:03.161717  4029 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.190s
I20260709 13:02:03.352757  1560 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "408b4ec829c54266b41334aac8245405"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:02:03.417111  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:03.510993  1565 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
W20260709 13:02:03.418061  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 77586.4us
I20260709 13:02:03.512991  1683 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:03.359611  3891 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:03.373224  3946 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:03.358560  3816 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:02:03.363874  3640 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:03.526307  1683 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:03.372548  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:03.454584  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:03.372936  3953 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:02:03.424271  3963 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:03.421465  3876 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.423987  3944 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.517977  1717 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:02:03.519431  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 163537us
I20260709 13:02:03.424401  3802 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.482517  3853 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:03.520994  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:03.538093  1639 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:03.524740  1560 raft_consensus.cc:606] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:03.528493  3665 raft_consensus.cc:2764] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:02:03.553414  3640 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:03.541642  4058 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:03.555136  3944 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:03.556915  4054 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:02:03.557137  4041 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:03.559612  3876 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:03.557404  4024 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:03.559350  1643 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:03.550835  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:03.535991  4015 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:03.537499  3891 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:03.577360  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 56472.4us
I20260709 13:02:03.555003  4044 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:03.500871  3945 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:03.582329  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 434704us
I20260709 13:02:03.547519  3963 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:03.590754  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.578889  1560 raft_consensus.cc:614] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:03.580492  3853 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:03.580983  1637 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:03.582858  3824 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:03.524099  4047 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.645125  1563 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:03.602830  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:03.649468  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:03.650141  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 99413.5us
I20260709 13:02:03.656137  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:03.602173  3876 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.660058  1683 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:03.658205  3824 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:03.676957  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:03.677202  3824 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:03.677420  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:03.678103  1714 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:03.678423  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:03.679862  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:03.680418  4068 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.680950  4068 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.687297  4022 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:03.687556  4057 log.cc:927] Time spent T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:02:03.687980  4022 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.688689  4047 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:03.690515  4022 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:03.656551  3947 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.692978  4056 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:03.693645  3947 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.695111  3876 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.695704  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:03.698228  3876 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:03.646337  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:03.697257  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:03.650003  4052 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.240s	user 0.002s	sys 0.000s
I20260709 13:02:03.603607  4024 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:03.707695  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 104985us
I20260709 13:02:03.714424  1639 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:03.714564  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:03.661132  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:03.715224  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:03.695916  3947 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:03.650604  3953 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:03.717891  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:03.717821  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:03.726150  1569 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:03.718876  1713 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:03.726306  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:03.649391  3944 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.651077  3963 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:03.727206  1709 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:02:03.718822  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 58854.6us
W20260709 13:02:03.728544  4030 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.720s	user 0.000s	sys 0.004s
I20260709 13:02:03.723263  3538 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:03.708092  3640 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.708185  4041 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:03.748381  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:03.765658  1639 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:03.740585  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:03.768536  1713 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:03.733654  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:03.733109  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:03.771102  1715 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:03.604588  3891 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.707978  3863 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:03.774340  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 57907us
I20260709 13:02:03.787366  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:03.792156  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:03.724110  3996 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:03.803841  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:03.805500  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:03.806887  2301 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:03.808256  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:03.808808  2301 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEARNER]: Advancing to term 7
I20260709 13:02:03.774252  4024 raft_consensus.cc:697] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:03.810043  4024 consensus_queue.cc:237] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.764420  3538 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:03.814688  3538 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:03.763736  3802 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:03.816861  3891 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:03.755528  3953 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:03.773734  3640 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:03.820147  3953 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:03.821733  3640 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:03.823536  4061 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:02:03.824836  4074 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:03.825500  4074 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:03.827302  3691 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:03.828058  3991 raft_consensus.cc:3055] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:03.827942  3691 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.828583  3991 raft_consensus.cc:740] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:03.829170  4065 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.829739  4065 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.830463  3691 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:03.603626  4058 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.831580  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:03.832873  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:03.833168  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:03.833592  1642 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:03.828100  4074 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:03.835467  4064 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:02:03.830133  3991 consensus_queue.cc:260] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.836865  3991 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:03.834858  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:03.721760  3876 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.779688  4075 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:03.811331  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:03.811677  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:03.852674  2305 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:03.812057  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:03.833395  4058 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:03.784214  4025 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
W20260709 13:02:03.828601  4049 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.309s	user 0.002s	sys 0.000s
W20260709 13:02:03.848752  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 192720us
I20260709 13:02:03.874431  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:03.719218  4068 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:03.857196  2304 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:03.805544  3889 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.568s	user 0.009s	sys 0.002s
W20260709 13:02:03.826784  4000 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 1.463s	user 0.003s	sys 0.000s
I20260709 13:02:03.861454  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:03.864377  4041 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.856856  3640 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.862871  4059 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:02:03.857105  3743 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.863811  3863 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.864372  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:03.864812  2301 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEARNER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:03.934638  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:03.935485  2301 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:03.865942  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:03.883315  4074 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:03.778374  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:03.871538  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 118606us
I20260709 13:02:03.876340  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:03.845228  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:03.938958  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:03.939391  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.940129  1681 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:03.862249  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:03.899151  1683 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:03.943369  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 67126.4us
I20260709 13:02:03.944939  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:03.951215  4077 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:03.951438  4075 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:03.952135  4075 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:03.952349  4070 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:03.894474  2304 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:03.891384  1711 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:03.952767  4075 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:03.956388  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 57377.4us
I20260709 13:02:03.896384  3889 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.957648  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:03.958168  1644 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:03.892305  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:03.858283  3876 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.986326  1607 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:03.893810  2269 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:03.776207  3944 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:03.992723  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 99010.8us
I20260709 13:02:03.871145  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:03.884032  3593 consensus_meta.cc:287] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.724s	user 0.011s	sys 0.000s
I20260709 13:02:03.871755  2296 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
W20260709 13:02:03.942622  4069 log.cc:927] Time spent T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:02:03.994256  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:03.933106  4041 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:03.988324  3997 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:03.941041  3743 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:03.936697  3640 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:03.953773  3863 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:04.055878  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 64850.5us
I20260709 13:02:03.987123  2299 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:04.057255  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:03.995003  4077 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:04.014101  2298 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
W20260709 13:02:04.061724  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 67983.2us
I20260709 13:02:04.009363  3665 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.064631  2269 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:03.982683  3538 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:04.016309  4060 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:04.067328  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2146) took 1251 ms. Trace:
I20260709 13:02:04.076413  1638 rpcz_store.cc:276] 0709 13:02:02.816182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.819080 (+  2898us) service_pool.cc:224] Handling call
0709 13:02:02.819571 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.846738 (+ 27167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.846774 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:02.862067 (+ 15293us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.108494 (+246427us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:03.112477 (+  3983us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.112806 (+   329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.112894 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:04.067037 (+954143us) raft_consensus.cc:1685] finished
0709 13:02:04.067065 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.067241 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":169,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":21840,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":25457,"thread_start_us":196146,"threads_started":1,"wal-append.queue_time_us":425507}
I20260709 13:02:04.020435  3889 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:04.079649  1643 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:04.081892  1638 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:02:04.103736  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.009892  3593 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:04.075748  3665 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:04.107842  3593 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:04.108752  3665 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:04.109910  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:04.110746  1635 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:04.112852  3876 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:04.126044  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:04.127615  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2166) took 1066 ms. Trace:
I20260709 13:02:04.128208  1561 rpcz_store.cc:276] 0709 13:02:03.061168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.061321 (+   153us) service_pool.cc:224] Handling call
0709 13:02:03.074617 (+ 13296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.074822 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:03.084045 (+  9223us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.114519 (+1030474us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b18001c0540. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a
0709 13:02:04.114822 (+   303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.114886 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:04.127263 (+ 12377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.127383 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.127541 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34303,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":38,"raft.queue_time_us":2807,"spinlock_wait_cycles":5937920,"thread_start_us":25450,"threads_started":1,"wal-append.queue_time_us":265775}
I20260709 13:02:04.064468  4074 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:04.138161  1565 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 7. Ops: [5.5-5.5]
I20260709 13:02:04.062500  4077 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:04.153369  4045 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:04.118269  4051 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:02:04.154982  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 90440.8us
I20260709 13:02:04.172327  4090 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:04.174196  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.175204  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:04.136847  1560 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:04.099171  3995 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:04.153956  2300 raft_consensus.cc:2410] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:04.177203  1609 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:04.218396  2269 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:04.227101  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:04.227849  2269 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:04.231447  3640 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:04.124812  4065 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:04.227679  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 50606us
I20260709 13:02:04.241340  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:04.141404  4059 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:04.247798  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2172) took 1091 ms. Trace:
I20260709 13:02:04.248256  1563 rpcz_store.cc:276] 0709 13:02:03.157036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.235429 (+ 78393us) service_pool.cc:224] Handling call
0709 13:02:03.235719 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.659101 (+423382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.247416 (+588315us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b54002a9c54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:04.247599 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941103488}
I20260709 13:02:04.249195  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2145) took 1433 ms. Trace:
I20260709 13:02:04.249646  1641 rpcz_store.cc:276] 0709 13:02:02.815893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.817931 (+  2038us) service_pool.cc:224] Handling call
0709 13:02:02.848034 (+ 30103us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.859861 (+ 11827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.248783 (+1388922us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b54000d16d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:04.249120 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25416448}
I20260709 13:02:04.178926  3855 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.215946  2296 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:04.178467  4013 raft_consensus.cc:3055] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:04.177780  4068 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.167299  4077 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:04.226230  3995 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.245193  3975 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:04.246793  3640 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:04.388134  4087 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:04.388845  4087 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.389258  4087 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:04.253489  4040 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.128s	user 0.008s	sys 0.005s
I20260709 13:02:04.254148  4059 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:04.391791  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:04.392266  4073 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:04.392470  1641 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:04.392472  4040 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:04.393090  4073 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:04.392541  4059 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:04.233536  3945 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:04.435408  4078 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.393s	user 0.001s	sys 0.000s
I20260709 13:02:04.440821  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2149) took 1026 ms. Trace:
I20260709 13:02:04.441473  1639 rpcz_store.cc:276] 0709 13:02:03.414381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.766406 (+352025us) service_pool.cc:224] Handling call
0709 13:02:03.766782 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.774086 (+  7304us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:02:03.982067 (+207981us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.126219 (+144152us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:04.145157 (+ 18938us) log.cc:844] Serialized 26 byte log entry
0709 13:02:04.155240 (+ 10083us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:04.155282 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.156003 (+   721us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:04.440515 (+284512us) raft_consensus.cc:1685] finished
0709 13:02:04.440545 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.440738 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6837,"prepare.queue_time_us":326341,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":41,"raft.queue_time_us":76902,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":24035,"spinlock_wait_cycles":85376384,"thread_start_us":246113,"threads_started":2,"wal-append.queue_time_us":442346}
I20260709 13:02:04.452909  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1815) took 2298 ms. Trace:
I20260709 13:02:04.453577  1567 rpcz_store.cc:276] 0709 13:02:02.154802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.155140 (+   338us) service_pool.cc:224] Handling call
0709 13:02:02.156637 (+  1497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.967525 (+810888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.452539 (+1485014us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b54001cdc54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:04.452820 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1863280000}
I20260709 13:02:04.450572  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:04.253543  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:04.282593  3855 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:04.216187  4045 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:04.364298  4013 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:04.545766  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 406) took 1196 ms. Trace:
I20260709 13:02:04.555855  1566 rpcz_store.cc:276] 0709 13:02:03.349630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.733572 (+383942us) service_pool.cc:224] Handling call
0709 13:02:03.733915 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.778234 (+ 44319us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:03.800273 (+ 22039us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.855048 (+ 54775us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.855096 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:03.855137 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.545357 (+690220us) spinlock_profiling.cc:243] Waited 44.1 ms on lock 0x7b5400217354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:04.545443 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.545671 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63057,"prepare.queue_time_us":21830,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":54901,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":16136,"spinlock_wait_cycles":101335808}
W20260709 13:02:04.559882  1569 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.808s	user 0.011s	sys 0.006s
I20260709 13:02:04.560401  1569 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
W20260709 13:02:04.566664  1718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.039s	sys 0.154s
W20260709 13:02:04.569510  1718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.040s	sys 0.154s
W20260709 13:02:04.523284  3824 consensus_meta.cc:287] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.845s	user 0.008s	sys 0.005s
I20260709 13:02:04.399381  3816 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:04.425064  3975 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:04.408386  3995 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:04.574385  1638 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:04.522418  3691 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:04.586112  1641 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:04.590143  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:04.381400  4068 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:04.591378  1641 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:02:04.587224  4041 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:04.559468  3945 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:02:04.615469  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.574128  4070 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:04.616780  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:04.576833  4022 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.578286  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:04.624030  1639 raft_consensus.cc:2410] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:04.588042  3944 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:04.616573  3865 consensus_meta.cc:287] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.264s	user 0.008s	sys 0.009s
I20260709 13:02:04.599624  4091 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.647578  3702 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.591454  3816 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:04.657721  3702 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:04.657424  4089 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
I20260709 13:02:04.582803  3824 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:04.621865  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:04.622952  3743 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:04.647136  4088 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 13:02:04.637256  3863 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:04.648226  3538 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:04.653802  2296 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 1.
I20260709 13:02:04.687906  2303 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:04.690585  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:04.658891  3865 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:04.694171  3946 consensus_meta.cc:287] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.166s	user 0.010s	sys 0.006s
I20260709 13:02:04.712877  1683 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:04.671567  4013 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:04.705819  1534 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:04.718236  4083 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:02:04.719653  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1612) took 1938 ms. Trace:
W20260709 13:02:04.720034  1637 consensus_meta.cc:287] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.006s	user 0.009s	sys 0.001s
I20260709 13:02:04.720048  1716 rpcz_store.cc:276] 0709 13:02:02.781564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.789974 (+  8410us) service_pool.cc:224] Handling call
0709 13:02:02.790359 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.815335 (+ 24976us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.815364 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.836784 (+1021420us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b2c0010a498. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:02:03.837336 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.959270 (+121934us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:03.959311 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.959384 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:04.719336 (+759952us) raft_consensus.cc:1685] finished
0709 13:02:04.719356 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.719576 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47324,"raft.queue_time_us":25074,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":20009,"spinlock_wait_cycles":2369280,"thread_start_us":121510,"threads_started":1,"wal-append.queue_time_us":529075}
I20260709 13:02:04.726070  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:04.726879  2302 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:04.720482  3992 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:02:04.727871  3538 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:04.728322  3538 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:04.728691  3538 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:04.729252  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 402) took 1624 ms. Trace:
I20260709 13:02:04.729660  1562 rpcz_store.cc:276] 0709 13:02:03.105522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.225365 (+119843us) service_pool.cc:224] Handling call
0709 13:02:03.225749 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.236190 (+ 10441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.236222 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.236726 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.531186 (+294460us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:03.535087 (+  3901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.728854 (+1193767us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b5400297cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:04.728902 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:04.728952 (+    50us) raft_consensus.cc:1685] finished
0709 13:02:04.728969 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.729190 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28193,"raft.queue_time_us":53987,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":25194,"spinlock_wait_cycles":23160064,"thread_start_us":271406,"threads_started":1,"wal-append.queue_time_us":339463}
I20260709 13:02:04.726194  3900 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:04.738489  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:04.739660  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:04.741040  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:04.741273  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:04.741845  1569 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:04.741900  1566 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:04.657444  3975 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:04.662271  4022 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:04.743525  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:04.746268  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:04.755889  1567 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:04.741246  1637 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:04.752509  1569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:02:04.759497  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:04.759960  1569 raft_consensus.cc:606] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:04.760244  1567 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:04.760365  1569 raft_consensus.cc:614] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:04.753670  3853 consensus_meta.cc:287] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.052s	user 0.009s	sys 0.001s
W20260709 13:02:04.751096  3991 consensus_meta.cc:287] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.913s	user 0.005s	sys 0.007s
I20260709 13:02:04.756703  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:04.720824  4044 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:04.671908  4091 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:04.710341  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:04.781520  1569 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:02:04.765805  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 52994.6us
I20260709 13:02:04.760768  2302 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:04.769191  3853 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:04.759210  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:04.740383  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:04.689255  3945 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:04.786149  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:04.755759  4065 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:04.790972  3853 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:04.759263  3855 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:04.771152  3701 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:04.766901  4013 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:04.762162  4075 consensus_meta.cc:287] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.806s	user 0.000s	sys 0.024s
I20260709 13:02:04.745332  3743 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:02:04.781140  1637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 439) took 1421 ms (client timeout 1649 ms). Trace:
I20260709 13:02:04.799549  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:04.799713  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:04.811762  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 584084us
I20260709 13:02:04.800753  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:04.806391  3855 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:04.818883  3896 consensus_meta.cc:287] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.359s	user 0.010s	sys 0.004s
I20260709 13:02:04.819110  2296 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:04.819458  3896 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:04.807404  4075 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:04.805120  2299 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:04.803071  4068 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:04.813647  2269 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:04.802963  3701 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:04.842972  4099 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:04.843313  2269 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:04.817444  4073 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:04.849433  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:04.850315  1566 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:02:04.854336  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:04.855142  2302 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:04.806643  1637 rpcz_store.cc:269] 0709 13:02:03.347414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.350599 (+  3185us) service_pool.cc:224] Handling call
0709 13:02:04.768923 (+1418324us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:04.830487  4075 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:04.833884  3701 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:04.829434  3816 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:04.826597  3896 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:04.812275  3743 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:04.843405  4094 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
W20260709 13:02:04.858945  3867 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.020s	sys 0.040s
I20260709 13:02:04.898145  4099 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:04.931305  3867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.021s	sys 0.040s
I20260709 13:02:04.931495  4099 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.932173  1566 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:04.932694  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:04.910989  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:04.936450  1639 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:04.939168  1639 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:04.940788  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.917s	user 0.019s	sys 0.031s
W20260709 13:02:04.941778  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.020s	sys 0.031s
I20260709 13:02:04.943634  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:04.944536  1644 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
W20260709 13:02:04.935344  1609 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.015s	sys 0.033s
W20260709 13:02:04.946550  1609 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.015s	sys 0.034s
I20260709 13:02:04.926824  1635 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:04.926780  4041 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.949767  1639 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:02:04.871299  4047 consensus_meta.cc:287] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.182s	user 0.012s	sys 0.004s
W20260709 13:02:04.959591  1681 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.021s	sys 0.021s
W20260709 13:02:04.960868  1681 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.021s	sys 0.021s
I20260709 13:02:04.943148  4068 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.943404  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:04.926129  4073 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:04.926326  4102 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.913255  3992 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:04.944504  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:04.984769  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.913255  3863 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:04.855944  3824 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:04.841102  3944 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.106s
I20260709 13:02:04.845049  4090 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 13:02:04.932838  4099 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:04.955024  4094 consensus_queue.cc:1243] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:04.989015  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:04.989462  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:05.080274  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 78859.6us
I20260709 13:02:05.080706  1639 raft_consensus.cc:2410] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:02:04.994036  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 254451us
I20260709 13:02:04.995291  2302 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:04.926512  4045 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:04.952965  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 546475us
I20260709 13:02:05.083402  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:05.084491  1639 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:04.982777  4096 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:02:05.084462  1533 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:04.964998  4041 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:04.987046  4044 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.965257  3865 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:05.086287  1609 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.718s (SENT) [suppressed 11 similar messages]
I20260709 13:02:05.097931  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:05.084596  1607 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:05.105739  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 410) took 1446 ms. Trace:
I20260709 13:02:05.105469  4044 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.123822  3816 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:05.127379  4041 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:05.097544  3855 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:02:04.998896  1562 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:05.126389  4044 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:05.130061  2304 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:05.078403  3992 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.119460  1560 rpcz_store.cc:276] 0709 13:02:03.659422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.712235 (+ 52813us) service_pool.cc:224] Handling call
0709 13:02:05.105496 (+1393261us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54001ff754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:05.105667 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":412206464}
I20260709 13:02:05.132558  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:05.132740  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:05.133414  3992 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:05.133643  2298 raft_consensus.cc:2393] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 5: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:05.134187  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 586) took 1041 ms. Trace:
I20260709 13:02:05.090701  3607 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:02:05.123167  3853 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:05.130825  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 784) took 1262 ms. Trace:
I20260709 13:02:05.079255  3863 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.136324  2304 rpcz_store.cc:276] 0709 13:02:03.869159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.048858 (+179699us) service_pool.cc:224] Handling call
0709 13:02:05.129630 (+1080772us) spinlock_profiling.cc:243] Waited 7.37 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:02:05.129852 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.130548 (+   696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.130768 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16940288}
I20260709 13:02:05.138247  3997 raft_consensus.cc:3055] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:05.138409  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:05.138624  3863 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:05.138741  3997 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:05.131251  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:05.139567  1709 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:05.089659  3743 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:05.134577  2298 rpcz_store.cc:276] 0709 13:02:04.092957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.105532 (+ 12575us) service_pool.cc:224] Handling call
0709 13:02:05.132579 (+1027047us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:02:05.134123 (+  1544us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45521792}
I20260709 13:02:05.140084  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1619) took 1372 ms. Trace:
I20260709 13:02:05.078711  3946 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:05.089764   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:02:05.118119  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:05.136121  3853 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:05.157259  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:05.136183  4095 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.292s	user 0.003s	sys 0.000s
I20260709 13:02:05.158644  3853 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:05.149488  1709 rpcz_store.cc:276] 0709 13:02:03.768035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.768175 (+   140us) service_pool.cc:224] Handling call
0709 13:02:05.131101 (+1362926us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b58001e4598. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:02:05.140021 (+  8920us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8790912}
I20260709 13:02:05.160006  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 472) took 1368 ms. Trace:
I20260709 13:02:05.160394  1718 rpcz_store.cc:276] 0709 13:02:03.792070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.820730 (+ 28660us) service_pool.cc:224] Handling call
0709 13:02:03.821144 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.849505 (+ 28361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.849554 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.850189 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 13:02:04.571708 (+721519us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:04.571763 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.571846 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.135587 (+563741us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:05.159593 (+ 24006us) raft_consensus.cc:1685] finished
0709 13:02:05.159618 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.159938 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80183,"raft.queue_time_us":28038,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":15255,"spinlock_wait_cycles":663849344,"thread_start_us":706343,"threads_started":1,"wal-append.queue_time_us":922308}
I20260709 13:02:05.162192  4090 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.997629  4022 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:05.139895  3997 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.171864  3997 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:05.130502  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:04.999434  4073 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:05.151803  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.177367  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 746) took 1380 ms. Trace:
I20260709 13:02:05.110584  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:05.179136  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:05.152581  3946 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:05.153177  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:05.181479  2303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 779) took 1388 ms (client timeout 1831 ms). Trace:
I20260709 13:02:05.111059  4091 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:05.150497  3865 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.181519  3946 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:05.155428  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:05.156311   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:05.178043  2301 rpcz_store.cc:276] 0709 13:02:03.796900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.942154 (+145254us) service_pool.cc:224] Handling call
0709 13:02:04.009987 (+ 67833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.010233 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.010270 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.010324 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:04.010372 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.010443 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.177119 (+1166676us) spinlock_profiling.cc:243] Waited 67.4 ms on lock 0x7b540049a550. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:05.177287 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155110272}
I20260709 13:02:05.184646  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 592) took 1065 ms. Trace:
I20260709 13:02:05.085515  4104 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.078851  4102 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.185730  4104 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.185267  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:05.185082  2302 rpcz_store.cc:276] 0709 13:02:04.119371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.142226 (+1022855us) service_pool.cc:224] Handling call
0709 13:02:05.184572 (+ 42346us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:05.164938  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 46944.3us
I20260709 13:02:05.188088  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:05.112388  4065 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:05.113274  3824 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:05.199123  3824 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:05.199252  3855 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:05.199712  3824 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:05.199870  3855 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.200484  3855 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:05.165194  4090 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.099963  4075 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:02:04.984490  4068 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:05.168774  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:05.181885  2304 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:05.215108  2305 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:05.215690  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:05.183126  3865 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:05.139978  4087 consensus_meta.cc:287] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.750s	user 0.011s	sys 0.002s
W20260709 13:02:05.182142  2303 rpcz_store.cc:269] 0709 13:02:03.792652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.805225 (+ 12573us) service_pool.cc:224] Handling call
0709 13:02:05.181217 (+1375992us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b54001112d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:05.181402 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2027207552}
I20260709 13:02:05.193840  1636 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:02:05.194621  4102 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:05.226076  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:05.227834  3889 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:05.230333  3889 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.230386  4104 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:05.230832  3889 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:05.231029  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:05.229786  4068 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:05.233232  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:05.229890  4077 consensus_meta.cc:287] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.843s	user 0.007s	sys 0.004s
I20260709 13:02:05.208630  3816 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.202694  4065 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:05.162747  3995 consensus_meta.cc:287] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.567s	user 0.008s	sys 0.000s
I20260709 13:02:05.177024  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:05.158290  4059 consensus_meta.cc:287] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.763s	user 0.013s	sys 0.000s
I20260709 13:02:05.222412  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:05.270756  2302 raft_consensus.cc:2410] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:05.178738  4073 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:05.226181  4087 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.234647  4077 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.136420  3538 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.235755  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:05.240383  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 156008us
I20260709 13:02:05.268716  2304 raft_consensus.cc:2393] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 5: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:05.272994  2305 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:05.204520  4090 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:05.224483  3963 consensus_meta.cc:287] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.431s	user 0.016s	sys 0.002s
I20260709 13:02:05.279407  1533 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:05.279807  2305 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 782) took 1428 ms (client timeout 1867 ms). Trace:
I20260709 13:02:05.254276  3896 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:05.283749  2305 rpcz_store.cc:269] 0709 13:02:03.851212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.868571 (+ 17359us) service_pool.cc:224] Handling call
0709 13:02:05.168545 (+1299974us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:02:05.279726 (+111181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8102784}
I20260709 13:02:05.280421  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:05.240864  4041 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:05.292800  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:05.254191  3877 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:05.296801  1641 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:05.297354  2301 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:05.288587  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:05.293066  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:05.298754  2302 raft_consensus.cc:2376] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5. Re-sending same reply.
I20260709 13:02:05.305436  2303 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:05.306376  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 788) took 1408 ms. Trace:
I20260709 13:02:05.306850  2303 rpcz_store.cc:276] 0709 13:02:03.898056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.304553 (+1406497us) service_pool.cc:224] Handling call
0709 13:02:05.304979 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.306051 (+  1072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.306303 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:05.279134  4086 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.001s	sys 0.063s
I20260709 13:02:05.280685  1562 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:05.311316  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:05.311971  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:05.313061  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 380471us
I20260709 13:02:05.318823  2303 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:05.255101  3816 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:05.321600  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:05.328243  2302 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:05.318181  3926 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.322291  4106 consensus_queue.cc:1059] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:02:05.277622  4059 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.313156  3877 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.291996  4045 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:05.309249  4041 raft_consensus.cc:697] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:05.328981  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 583) took 1237 ms. Trace:
I20260709 13:02:05.355808  2302 rpcz_store.cc:276] 0709 13:02:04.091975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.299727 (+1207752us) service_pool.cc:224] Handling call
0709 13:02:05.320596 (+ 20869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.328655 (+  8059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.328918 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:05.286502  3963 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.297650  3896 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.338567  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:05.369369  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:05.331672  3926 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.341573  4045 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:05.337872  4102 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:05.343714  4011 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:05.303113  4043 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:05.351863  3701 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.352470  4106 raft_consensus.cc:3055] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:05.318164  4075 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.396646  4075 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:05.397362  4059 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:05.359632  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:05.399114  1639 raft_consensus.cc:2376] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6. Re-sending same reply.
I20260709 13:02:05.399338  3877 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:05.398762  4075 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:05.399863  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:05.400547  1566 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:05.400797  3853 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.401407  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:05.402213  4013 consensus_meta.cc:287] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.596s	user 0.009s	sys 0.000s
W20260709 13:02:05.317605  4086 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.001s	sys 0.065s
I20260709 13:02:05.319937  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 584) took 1227 ms. Trace:
I20260709 13:02:05.331782  3743 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:05.402417  1566 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:05.405894  2303 rpcz_store.cc:276] 0709 13:02:04.092331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.308263 (+1215932us) service_pool.cc:224] Handling call
0709 13:02:05.308659 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.319602 (+ 10943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.319862 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:05.370493  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:05.284798  3995 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.384421  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:05.407371  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:05.408149  2302 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:05.408361  1566 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
W20260709 13:02:05.364866  3900 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.027s	sys 0.038s
I20260709 13:02:05.388095  2305 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:05.396098  4106 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:05.362619  3945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.021s	sys 0.046s
I20260709 13:02:05.375754  4066 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:05.416739  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 46345.4us
I20260709 13:02:05.376295  4091 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:05.420575  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:05.417646  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:05.346758  3855 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.383749  4045 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:05.399281  3926 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:05.385959  3867 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:05.440765  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:05.402539  3865 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:05.416566  3900 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.027s	sys 0.039s
I20260709 13:02:05.324648  4065 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.415793  3701 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:05.421241  3945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.021s	sys 0.046s
I20260709 13:02:05.441675  4104 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:05.419731  4102 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.419705  4094 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:05.458458  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:05.421752  3743 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:05.441985  3896 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:05.469297  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2164) took 2446 ms (client timeout 2104 ms). Trace:
I20260709 13:02:05.448912  4087 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:05.444186  3538 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:05.472182  1568 rpcz_store.cc:269] 0709 13:02:03.057909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.064944 (+  7035us) service_pool.cc:224] Handling call
0709 13:02:05.469068 (+2404124us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b5400179dd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:05.469227 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070586112}
I20260709 13:02:05.465751  2302 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:05.449661  3867 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.461427  4104 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.440006  4066 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:05.502785  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 421270us
I20260709 13:02:05.510041  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:05.510456  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:05.510857  1718 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:05.511276  1716 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:05.512898  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:05.507933  1567 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.018s	sys 0.011s
I20260709 13:02:05.513631  2302 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:05.513927  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:05.512962  1683 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:05.514508  1718 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
W20260709 13:02:05.515576  1567 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.019s	sys 0.011s
I20260709 13:02:05.515810  1716 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:05.469151  4116 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:05.519837  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:05.517778  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 532056us
I20260709 13:02:05.520529  1709 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:05.502805  3945 raft_consensus.cc:3055] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:05.509565  4104 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:05.539134  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.019s	sys 0.035s
I20260709 13:02:05.512471  4077 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:05.542233  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.019s	sys 0.036s
I20260709 13:02:05.517753  4011 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:05.484078  4102 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.517995  4116 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:05.634177  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:05.662380  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 44572us
W20260709 13:02:05.541152  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 212319us
W20260709 13:02:05.668823  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 480867us
I20260709 13:02:05.487864  4106 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.668008  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:05.533105  3945 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:05.542636  3853 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:05.670883  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:05.689970  3802 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:02:05.553793  4011 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:05.720289  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 49527.8us
I20260709 13:02:05.530397  1560 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:05.532776  3701 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.143s
I20260709 13:02:05.614742  4091 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:05.614702  3900 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:05.468806  4041 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.624809  4094 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.160s
I20260709 13:02:05.660988  3926 consensus_queue.cc:1059] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.138s
W20260709 13:02:05.793489  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 274206us
I20260709 13:02:05.906126  2302 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
W20260709 13:02:05.923074  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 384681us
W20260709 13:02:06.104089  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 383368us
W20260709 13:02:06.128417  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.128540  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:06.140183  1682 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:05.733160  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:06.022131  1534 leader_election.cc:336] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 2.239s (SENT)
W20260709 13:02:06.142086  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 409054us
W20260709 13:02:05.923288  1563 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.010s	sys 0.027s
I20260709 13:02:06.142169  1534 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:06.143630  1563 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.016s	sys 0.032s
I20260709 13:02:06.145265  1683 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:06.149055  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:06.146499  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:06.151839  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 786) took 2273 ms. Trace:
W20260709 13:02:05.774431  4099 consensus_meta.cc:287] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.686s	user 0.009s	sys 0.000s
I20260709 13:02:05.697463  4077 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:06.189621  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.014s	sys 0.031s
W20260709 13:02:06.289103  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.023s	sys 0.031s
I20260709 13:02:06.144447  4087 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.154373  4019 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:06.192715  1681 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:06.207464  2305 rpcz_store.cc:276] 0709 13:02:03.878844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.296448 (+1417604us) service_pool.cc:224] Handling call
0709 13:02:05.311088 (+ 14640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.412358 (+101270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.151551 (+739193us) spinlock_profiling.cc:243] Waited 76.8 ms on lock 0x7b5400406854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:06.151765 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176646016}
W20260709 13:02:06.193010  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.691068  3816 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:05.718696  3896 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:05.662810  4116 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:06.209858  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:06.365691  2302 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:05.782373  3975 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:06.287153  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 459) took 3411 ms. Trace:
W20260709 13:02:05.796041  4108 log.cc:927] Time spent T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.429s	user 0.001s	sys 0.000s
I20260709 13:02:05.689764  4022 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.470s
W20260709 13:02:06.049940  3824 consensus_meta.cc:287] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.849s	user 0.009s	sys 0.004s
I20260709 13:02:05.918005  3855 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:06.288432  4121 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:06.068143  4112 log.cc:927] Time spent T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.663s	user 0.001s	sys 0.001s
W20260709 13:02:06.339260  1644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.020s	sys 0.014s
I20260709 13:02:06.146992  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 416) took 1199 ms. Trace:
I20260709 13:02:06.048571  4014 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.807920  4106 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:05.787248  3889 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.556s	user 0.005s	sys 0.006s
W20260709 13:02:06.423604  1644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.021s	sys 0.014s
I20260709 13:02:06.528438  3889 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:06.150561  3853 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:06.531262  3853 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:06.531883  3853 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:06.366746  1717 rpcz_store.cc:276] 0709 13:02:02.779546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.779840 (+   294us) service_pool.cc:224] Handling call
0709 13:02:02.780302 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.548000 (+767698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.190474 (+2642474us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b54001473d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:06.190747 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1665702272}
I20260709 13:02:05.795665  3963 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:06.535444  4014 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:06.536137  4126 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:02:06.538211  4014 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:06.538414  3877 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.539186  3877 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:06.402458  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:06.407436  3816 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.407203  3824 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:05.691303  3743 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:06.408890  4053 log.cc:927] Time spent T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 2.985s	user 0.016s	sys 0.029s
I20260709 13:02:06.410214  4124 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:06.128100  4011 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:05.771051  4066 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:06.325999  3607 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.528972  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1703) took 1229 ms. Trace:
I20260709 13:02:06.152212  4091 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:06.540951  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 610) took 1019 ms. Trace:
I20260709 13:02:06.406777   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:06.375934  3975 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.144070  3900 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:06.327852  4099 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:06.396306  3867 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:06.541667  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1822) took 2956 ms. Trace:
I20260709 13:02:06.443199  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:06.341276  4077 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:06.354377  4087 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:06.365047  3896 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:06.404978  3926 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:06.479753  4044 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.674505  3926 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.675164  4044 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:06.676019  3996 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:06.676429  3996 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.676784  3996 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:06.676889  4123 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.523s	user 0.001s	sys 0.000s
I20260709 13:02:06.677389  4044 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:06.572664  4114 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 1.060s	user 0.000s	sys 0.002s
I20260709 13:02:06.402384  4128 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:06.679620  2300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 740) took 2893 ms (client timeout 2239 ms). Trace:
I20260709 13:02:06.679775  4099 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:06.680154  2300 rpcz_store.cc:269] 0709 13:02:03.786460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.800508 (+ 14048us) service_pool.cc:224] Handling call
0709 13:02:06.679343 (+2878835us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54002a44d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:06.679518 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234675072}
I20260709 13:02:06.681666  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:06.682451  2300 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:06.683291  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2175) took 1785 ms. Trace:
I20260709 13:02:06.683825  1567 rpcz_store.cc:276] 0709 13:02:04.898395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.898743 (+   348us) service_pool.cc:224] Handling call
0709 13:02:04.911463 (+ 12720us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:04.938776 (+ 27313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.938807 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:04.950060 (+ 11253us) log.cc:844] Serialized 52 byte log entry
0709 13:02:05.517826 (+567766us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:05.937834 (+420008us) log.cc:844] Serialized 14 byte log entry
0709 13:02:06.108738 (+170904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.108823 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.683004 (+574181us) raft_consensus.cc:1685] finished
0709 13:02:06.683032 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.683214 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115154,"prepare.queue_time_us":1011712,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":234004,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":49288,"thread_start_us":916803,"threads_started":2,"wal-append.queue_time_us":1634093}
I20260709 13:02:06.622498  2303 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:06.634871  2302 rpcz_store.cc:276] 0709 13:02:05.521752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.402163 (+880411us) service_pool.cc:224] Handling call
0709 13:02:06.540873 (+138710us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:06.688055  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:06.688417  2303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 754) took 1357 ms (client timeout 1760 ms). Trace:
I20260709 13:02:06.689796  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:06.675166  3926 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:06.690467  2302 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:06.696455  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:06.652245  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 756) took 1314 ms. Trace:
I20260709 13:02:06.698589  1567 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:06.698526  2305 rpcz_store.cc:276] 0709 13:02:05.338223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.403274 (+1065051us) service_pool.cc:224] Handling call
0709 13:02:06.652176 (+248902us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:06.411298  4019 consensus_queue.cc:1059] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.199s
I20260709 13:02:06.651147  1561 rpcz_store.cc:276] 0709 13:02:03.585546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.130777 (+545231us) service_pool.cc:224] Handling call
0709 13:02:04.182485 (+ 51708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.220029 (+ 37544us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:04.342738 (+122709us) log.cc:844] Serialized 14 byte log entry
0709 13:02:05.171165 (+828427us) spinlock_profiling.cc:243] Waited 9.21 ms on lock 0x7b48000030d8. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a6bcaaa1 00007fd7a6bc1b37 00007fd7a6bc19ca 00007fd7a6cb926f 00007fd7a6cb8f37 00007fd7a6ccd55c 00007fd7a6ccd508 00007fd7a6ccd488 00007fd7a6ccd443 00007fd7a6ccc6ec 00007fd7a5bf6a96
0709 13:02:05.323616 (+152451us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b18001c24c0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a 00007fd7a1caba2a
0709 13:02:05.666035 (+342419us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:05.666526 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:02:05.666649 (+   123us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:05.666682 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.666747 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.541402 (+874655us) raft_consensus.cc:1685] finished
0709 13:02:06.541429 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.541602 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106090,"prepare.queue_time_us":121548,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":63289,"raft.run_cpu_time_us":7241,"raft.run_wall_time_us":969036,"spinlock_wait_cycles":56631936,"thread_start_us":310625,"threads_started":1,"wal-append.queue_time_us":1864700}
I20260709 13:02:06.594138  3889 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:06.543630  3855 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.098s
I20260709 13:02:06.609833  4124 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:06.673625  4077 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:06.682341  2297 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:06.631326  1644 rpcz_store.cc:276] 0709 13:02:05.299681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.299942 (+   261us) service_pool.cc:224] Handling call
0709 13:02:05.300435 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.300695 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:05.688894 (+388199us) log.cc:844] Serialized 14 byte log entry
0709 13:02:06.528607 (+839713us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:06.528664 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:06.528713 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.528778 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.528914 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1111013,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"thread_start_us":1226977,"threads_started":2}
W20260709 13:02:06.689337  2303 rpcz_store.cc:269] 0709 13:02:05.330527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.417360 (+ 86833us) service_pool.cc:224] Handling call
0709 13:02:06.688134 (+1270774us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400708fd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:06.688342 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2720796928}
I20260709 13:02:06.652802  4094 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:06.674425  3896 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:06.691813  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2147) took 3858 ms. Trace:
I20260709 13:02:06.725740  1640 rpcz_store.cc:276] 0709 13:02:02.833269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.844700 (+ 11431us) service_pool.cc:224] Handling call
0709 13:02:02.845005 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.857934 (+ 12929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.857965 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:02.873453 (+ 15488us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.108968 (+235515us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:03.109012 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.109085 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.145610 (+3036525us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:06.691447 (+545837us) raft_consensus.cc:1685] finished
0709 13:02:06.691470 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.691744 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3574,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":5397,"spinlock_wait_cycles":13465856,"thread_start_us":52522,"threads_started":1,"wal-append.queue_time_us":436030}
I20260709 13:02:06.649667  3607 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:06.542661  1560 rpcz_store.cc:276] 0709 13:02:04.939797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.132239 (+192442us) service_pool.cc:224] Handling call
0709 13:02:06.138795 (+1006556us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:06.693713  3702 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:06.761013  1607 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.006s	sys 0.036s
I20260709 13:02:06.423147  3701 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:06.695824  4025 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:06.426229  4121 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:06.711323  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 795) took 1334 ms. Trace:
I20260709 13:02:06.711602  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:06.797061  1710 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
W20260709 13:02:06.797802  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 473) took 3003 ms (client timeout 2282 ms). Trace:
W20260709 13:02:06.798434  1710 rpcz_store.cc:269] 0709 13:02:03.794703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.794864 (+   161us) service_pool.cc:224] Handling call
0709 13:02:06.711420 (+2916556us) spinlock_profiling.cc:243] Waited 7 ms on lock 0x7b58001e4598. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:02:06.797706 (+ 86286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16099584}
I20260709 13:02:06.674417  3900 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:06.638280  3975 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:06.718137  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:06.821470  2305 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:06.718266  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:06.722344  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:06.823984  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1710) took 1507 ms (client timeout 1760 ms). Trace:
W20260709 13:02:06.824525  1644 rpcz_store.cc:269] 0709 13:02:05.316942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.722088 (+1405146us) service_pool.cc:224] Handling call
0709 13:02:06.823904 (+101816us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:06.825956  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:06.609227  3538 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.649950  4066 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:06.594528  3816 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:06.722383  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:06.658095  4127 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:06.724822  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 773) took 2935 ms. Trace:
I20260709 13:02:06.730636  2296 raft_consensus.cc:2393] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:06.759125  1534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.014s	sys 0.029s
I20260709 13:02:06.759951  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:06.538851  4014 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:02:06.761802  1607 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.006s	sys 0.036s
I20260709 13:02:06.866623  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:06.866978  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 637611us
I20260709 13:02:06.699491  4102 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:06.821337  2300 rpcz_store.cc:276] 0709 13:02:05.376670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.681343 (+1304673us) service_pool.cc:224] Handling call
0709 13:02:06.711226 (+ 29883us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:06.843222  2297 rpcz_store.cc:276] 0709 13:02:03.789845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.848850 (+ 59005us) service_pool.cc:224] Handling call
0709 13:02:06.681764 (+2832914us) spinlock_profiling.cc:243] Waited 5.23 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:02:06.681961 (+   197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.721583 (+ 39622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.724750 (+  3167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12019584}
I20260709 13:02:06.872061  4099 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.872749  4099 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:06.755551  1567 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:06.874184  4014 raft_consensus.cc:697] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:06.874814  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:06.876506  1567 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:06.877177  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2179) took 1946 ms. Trace:
W20260709 13:02:06.828620  1684 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.017s	sys 0.034s
W20260709 13:02:06.878283  1684 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.018s	sys 0.034s
W20260709 13:02:06.879168  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.881920  1710 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:06.882200  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:06.883064  1642 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
W20260709 13:02:06.883565  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2163) took 1363 ms (client timeout 1749 ms). Trace:
W20260709 13:02:06.883940  1642 rpcz_store.cc:269] 0709 13:02:05.519811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.881885 (+1362074us) service_pool.cc:224] Handling call
0709 13:02:06.883505 (+  1620us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:06.734804  4130 log.cc:927] Time spent T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.312s	user 0.001s	sys 0.001s
I20260709 13:02:06.886454  4120 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.887048  3824 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:06.887220  4024 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.829115  3701 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:06.887771  4024 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:06.889808  4024 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:06.890410  4024 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:06.742527  3997 consensus_meta.cc:287] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.570s	user 0.009s	sys 0.001s
W20260709 13:02:06.837539  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1713) took 1435 ms (client timeout 1712 ms). Trace:
W20260709 13:02:06.840467  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.021s	sys 0.027s
I20260709 13:02:06.738283  3996 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:06.737560  3607 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:06.892272  2304 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 603) took 1694 ms (client timeout 1658 ms). Trace:
I20260709 13:02:06.730727  4124 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:06.855175  1534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.014s	sys 0.029s
I20260709 13:02:06.897843  3996 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:06.892876  1644 rpcz_store.cc:269] 0709 13:02:05.401885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.825597 (+1423712us) service_pool.cc:224] Handling call
0709 13:02:06.837422 (+ 11825us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:06.867628  1561 raft_consensus.cc:2410] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:02:06.857028  1681 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.006s	sys 0.024s
I20260709 13:02:06.867698  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:06.865876  2302 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2.
I20260709 13:02:06.723842  4094 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.751665  4019 raft_consensus.cc:3055] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:06.877501  1567 rpcz_store.cc:276] 0709 13:02:04.930700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.875844 (+1945144us) service_pool.cc:224] Handling call
0709 13:02:06.876327 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.876903 (+   576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.877121 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:06.897594  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:06.827497  2299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 596) took 2551 ms (client timeout 1737 ms). Trace:
W20260709 13:02:06.898689  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 876629us
I20260709 13:02:06.893776  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:06.916090  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2161) took 1400 ms. Trace:
W20260709 13:02:06.903062  1683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.009s	sys 0.027s
I20260709 13:02:06.903501  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:06.916543  1642 rpcz_store.cc:276] 0709 13:02:05.515352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.893338 (+1377986us) service_pool.cc:224] Handling call
0709 13:02:06.915994 (+ 22656us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:06.875559  4014 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:06.897364  2304 rpcz_store.cc:269] 0709 13:02:05.197484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.215409 (+ 17925us) service_pool.cc:224] Handling call
0709 13:02:06.892063 (+1676654us) spinlock_profiling.cc:243] Waited 52 ms on lock 0x7b54004cf754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:06.892200 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119708544}
W20260709 13:02:06.894587  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.022s	sys 0.027s
W20260709 13:02:06.923728  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 774842us
I20260709 13:02:06.923791  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:06.924697  2304 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 5: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:06.900630  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:06.899120  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:06.900091  1681 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.007s	sys 0.024s
I20260709 13:02:06.898797  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:06.891067  3665 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.925721  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:06.927596  1644 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:06.930949  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:06.942875  1709 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
W20260709 13:02:06.919533  1683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.009s	sys 0.027s
I20260709 13:02:06.919693  1561 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:06.821012  4025 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:06.944393  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2180) took 2013 ms. Trace:
I20260709 13:02:06.944790  1561 rpcz_store.cc:276] 0709 13:02:04.930966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.899652 (+1968686us) service_pool.cc:224] Handling call
0709 13:02:06.919443 (+ 19791us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.944096 (+ 24653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.944332 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:06.892696  1562 consensus_meta.cc:287] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.559s	user 0.006s	sys 0.006s
I20260709 13:02:06.946333  1561 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:06.941529  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:06.947721  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2184) took 2015 ms. Trace:
W20260709 13:02:06.927637  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 735087us
I20260709 13:02:06.948424  1561 rpcz_store.cc:276] 0709 13:02:04.931985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.945637 (+2013652us) service_pool.cc:224] Handling call
0709 13:02:06.945998 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.946717 (+   719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.946898 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:06.942230  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1711) took 1624 ms. Trace:
I20260709 13:02:06.949795  1644 rpcz_store.cc:276] 0709 13:02:05.317192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.898817 (+1581625us) service_pool.cc:224] Handling call
0709 13:02:06.941159 (+ 42342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:06.951144  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1704) took 1636 ms. Trace:
I20260709 13:02:06.900741  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 760) took 1207 ms. Trace:
I20260709 13:02:06.951680  2305 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:06.951788  1642 rpcz_store.cc:276] 0709 13:02:05.314720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.920184 (+1605464us) service_pool.cc:224] Handling call
0709 13:02:06.951056 (+ 30872us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:06.951880  2302 rpcz_store.cc:276] 0709 13:02:05.693704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.689354 (+995650us) service_pool.cc:224] Handling call
0709 13:02:06.900674 (+211320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:06.928392  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:06.953931  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 757) took 1262 ms. Trace:
I20260709 13:02:06.951402  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:06.955147  1709 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:06.953464  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:06.956974  2302 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:06.954294  2305 rpcz_store.cc:276] 0709 13:02:05.691523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.699438 (+1007915us) service_pool.cc:224] Handling call
0709 13:02:06.953865 (+254427us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:06.960891  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:06.872120  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:06.904876  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:06.906947  4045 consensus_meta.cc:287] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.453s	user 0.007s	sys 0.005s
I20260709 13:02:06.965663  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:06.843319  4066 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:06.844834  3538 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:06.939311  4121 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:06.851559  4127 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:06.970336  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:06.905949  4094 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:06.966794  4128 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:06.889808  3701 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:06.906080  4019 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:06.962661  1562 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:07.003012  4091 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:06.966496  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 821324us
I20260709 13:02:06.890882  4024 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:06.985713  4104 consensus_meta.cc:287] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.322s	user 0.013s	sys 0.000s
I20260709 13:02:06.905875  4140 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:06.986625  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 593) took 2855 ms. Trace:
I20260709 13:02:06.962495  3665 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:06.988201  1560 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:06.968919  3877 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:06.988758  4045 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:06.991101  4066 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:06.992808  1640 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:06.946650  4124 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.995095  1642 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:06.999253  4027 raft_consensus.cc:3055] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:02:07.000181  2301 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 588) took 2906 ms (client timeout 3112 ms). Trace:
W20260709 13:02:07.177484  2301 rpcz_store.cc:269] 0709 13:02:04.093620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.184920 (+1091300us) service_pool.cc:224] Handling call
0709 13:02:06.999929 (+1815009us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b540075e754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:07.000114 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252505856}
I20260709 13:02:06.854507  3853 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:06.988618  1636 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.721s	user 0.010s	sys 0.002s
I20260709 13:02:07.106513  4140 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:06.953636  4042 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.022s	sys 0.006s
I20260709 13:02:06.982987  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 456) took 1318 ms. Trace:
W20260709 13:02:07.055019  1712 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.276s	user 0.017s	sys 0.035s
W20260709 13:02:07.012389  4103 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 2.184s	user 0.014s	sys 0.034s
I20260709 13:02:07.006383  4128 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.887084  4120 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:07.147298  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2162) took 1631 ms. Trace:
W20260709 13:02:07.152359  1532 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.020s	sys 0.030s
W20260709 13:02:06.906339  2299 rpcz_store.cc:269] 0709 13:02:04.244378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.662904 (+418526us) service_pool.cc:224] Handling call
0709 13:02:06.744191 (+2081287us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b54004998d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:06.795756 (+ 51565us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39692160}
I20260709 13:02:07.111683  4104 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:07.176383  4121 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.176016  4124 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:07.023491  1562 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 417) took 2077 ms (client timeout 2490 ms). Trace:
W20260709 13:02:07.450788  1533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.038s	user 0.003s	sys 0.025s
I20260709 13:02:07.283135  1568 raft_consensus.cc:1240] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:07.195704  2302 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:07.718611  1642 rpcz_store.cc:276] 0709 13:02:05.516283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.965057 (+1448774us) service_pool.cc:224] Handling call
0709 13:02:07.147215 (+182158us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:07.720032  1609 leader_election.cc:336] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.658s (SENT)
W20260709 13:02:07.723093  1533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.310s	user 0.003s	sys 0.033s
I20260709 13:02:07.723474  1609 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:07.724982  1532 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.322s	user 0.021s	sys 0.031s
W20260709 13:02:07.726167  1712 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.992s	user 0.023s	sys 0.038s
I20260709 13:02:07.719060  1644 rpcz_store.cc:276] 0709 13:02:05.664562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.965397 (+1300835us) service_pool.cc:224] Handling call
0709 13:02:06.982898 (+ 17501us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:07.725131  1609 leader_election.cc:336] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.718s (SENT)
I20260709 13:02:07.730037  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:07.731930  1716 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:02:07.743535  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:07.745249  2302 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEARNER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:07.748337  1609 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:07.748692  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 789) took 3183 ms. Trace:
I20260709 13:02:07.749693  2302 rpcz_store.cc:276] 0709 13:02:04.564857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.744667 (+3179810us) service_pool.cc:224] Handling call
0709 13:02:07.745035 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.748404 (+  3369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.748630 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:07.751796  2302 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:07.638410  4065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.903s	user 0.009s	sys 0.006s
I20260709 13:02:07.746119  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:07.724350  3926 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.026s	user 0.015s	sys 0.003s
I20260709 13:02:07.755821  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:07.756270  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:07.773483  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 790) took 3189 ms. Trace:
W20260709 13:02:07.555610  3743 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.817s	user 0.011s	sys 0.003s
W20260709 13:02:07.238713  3802 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.199s	user 0.027s	sys 0.031s
W20260709 13:02:07.279611  3992 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.014s	sys 0.032s
W20260709 13:02:07.414108  4106 consensus_meta.cc:287] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.874s	user 0.006s	sys 0.005s
I20260709 13:02:07.765983  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:07.772380  1533 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:07.773000  2303 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:07.770741  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:07.177251  4027 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:02:07.726938  4042 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.393s	user 0.028s	sys 0.010s
I20260709 13:02:07.784178  2302 rpcz_store.cc:276] 0709 13:02:04.565134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.751056 (+3185922us) service_pool.cc:224] Handling call
0709 13:02:07.751539 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.752542 (+  1003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.754988 (+  2446us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:07.845232  2302 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:07.846529  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 791) took 3159 ms. Trace:
I20260709 13:02:07.847038  2302 rpcz_store.cc:276] 0709 13:02:04.687550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.844558 (+3157008us) service_pool.cc:224] Handling call
0709 13:02:07.844967 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.846151 (+  1184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.846457 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:07.596169  4109 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.103s	user 0.010s	sys 0.015s
I20260709 13:02:07.451435  4147 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.284166  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2185) took 2290 ms. Trace:
I20260709 13:02:07.881424  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:02:07.791764  1535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.012s	sys 0.047s
W20260709 13:02:07.882308  1535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.579s	user 0.013s	sys 0.048s
I20260709 13:02:07.213433  4019 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:07.792380  1644 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:07.731751  3824 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.186057  3816 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:07.733144  4048 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:07.732316  3701 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:07.840207  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:07.840528  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 68243.6us
I20260709 13:02:07.900420  1709 raft_consensus.cc:2410] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:07.739863  4128 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:07.783337  3896 consensus_meta.cc:287] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.044s	user 0.012s	sys 0.000s
I20260709 13:02:07.722872  1636 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
W20260709 13:02:07.743104  4056 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 4.013s	user 0.023s	sys 0.058s
I20260709 13:02:07.766012  4025 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:07.738204  4140 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:07.773361  4065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.038s	user 0.012s	sys 0.007s
I20260709 13:02:07.742667  4066 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:07.799481  3802 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.007s	user 0.038s	sys 0.042s
W20260709 13:02:07.792735  3743 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.098s	user 0.013s	sys 0.003s
W20260709 13:02:07.839090  3992 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.426s	user 0.016s	sys 0.032s
W20260709 13:02:07.891693  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1714) took 1751 ms (client timeout 1941 ms). Trace:
W20260709 13:02:07.852206  4109 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.443s	user 0.010s	sys 0.018s
I20260709 13:02:07.749466  4121 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:07.906714  4048 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:07.907701  3701 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:08.059736  4116 consensus_meta.cc:287] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.614s	user 0.006s	sys 0.008s
W20260709 13:02:07.851794  4041 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.173s	user 0.037s	sys 0.036s
I20260709 13:02:07.911866  1709 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:02:08.012113  1644 rpcz_store.cc:269] 0709 13:02:06.140040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.755469 (+1615429us) service_pool.cc:224] Handling call
0709 13:02:07.891616 (+136147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:07.895135  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:08.073486  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:07.917996  1533 leader_election.cc:336] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.935s (SENT)
I20260709 13:02:07.107757  2300 rpcz_store.cc:276] 0709 13:02:04.131218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.985618 (+2854400us) service_pool.cc:224] Handling call
0709 13:02:06.986008 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.986240 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:06.986274 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:06.986320 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:06.986358 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.986422 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.986544 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:08.016738  4110 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 2.485s	user 0.021s	sys 0.041s
I20260709 13:02:08.049527  4124 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:08.009720  4103 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:02:08.037925  1642 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:08.024566  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:08.067565  1567 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:08.053352  4060 raft_consensus.cc:3055] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:08.053735  3665 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:08.075531  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 157592us
W20260709 13:02:08.023768  3850 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 7 is higher.
I20260709 13:02:08.008144  4045 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:08.070618  4027 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:08.076282  1644 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
W20260709 13:02:07.727097  1562 rpcz_store.cc:269] 0709 13:02:04.940111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.176698 (+236587us) service_pool.cc:224] Handling call
0709 13:02:07.017223 (+1840525us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:08.052305  4011 consensus_meta.cc:287] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.414s	user 0.009s	sys 0.003s
I20260709 13:02:07.889730  4042 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:07.955794  4127 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:08.072751  4041 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.394s	user 0.040s	sys 0.039s
W20260709 13:02:07.964984  3975 consensus_meta.cc:287] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.120s	user 0.010s	sys 0.000s
I20260709 13:02:08.016858  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 752) took 3274 ms. Trace:
W20260709 13:02:08.084947  1681 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.818s	user 0.009s	sys 0.080s
I20260709 13:02:08.093061  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:08.021373  4056 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.291s	user 0.027s	sys 0.066s
I20260709 13:02:08.053857  4024 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:07.965899  4150 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:07.966661  4148 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
I20260709 13:02:08.075306  3853 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:08.069416  4116 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:08.024008  3896 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:08.138326  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:07.798563  3926 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:08.095526  4060 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:08.071527  4065 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.017938  3816 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:08.071014  1636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2140) took 5285 ms (client timeout 2357 ms). Trace:
W20260709 13:02:08.159077  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2167) took 1574 ms (client timeout 1644 ms). Trace:
I20260709 13:02:08.168457  1716 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:08.242801  2305 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:07.881997  1560 rpcz_store.cc:276] 0709 13:02:04.965494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.987508 (+2022014us) service_pool.cc:224] Handling call
0709 13:02:06.987830 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.118945 (+131115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.256145 (+137200us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:08.157662  2297 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.279s	user 0.127s	sys 0.248s
I20260709 13:02:08.177073  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 409) took 4522 ms. Trace:
I20260709 13:02:08.177997  1566 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:08.178402  4011 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:07.897296  4147 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:08.194832  1681 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.009s	sys 0.080s
I20260709 13:02:08.195396  1709 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:08.214829  2300 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:08.198123  1563 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.885s	user 0.042s	sys 0.103s
W20260709 13:02:08.171945  4094 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.160s	user 0.013s	sys 0.000s
I20260709 13:02:07.910192  3824 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:08.201087  2268 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.248s	user 0.056s	sys 0.089s
I20260709 13:02:08.213357  1717 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
W20260709 13:02:08.214879  1561 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.076s	user 0.006s	sys 0.051s
W20260709 13:02:08.259245  2268 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.306s	user 0.056s	sys 0.090s
W20260709 13:02:08.259537  1561 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.137s	user 0.006s	sys 0.052s
W20260709 13:02:08.259966  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 1.30679e+06us
I20260709 13:02:08.260779  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 415) took 3321 ms. Trace:
W20260709 13:02:08.247285  2297 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.373s	user 0.127s	sys 0.248s
I20260709 13:02:08.261219  1561 rpcz_store.cc:276] 0709 13:02:04.939454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.949393 (+2009939us) service_pool.cc:224] Handling call
0709 13:02:06.978325 (+ 28932us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.978615 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:07.122134 (+143519us) log.cc:844] Serialized 14 byte log entry
0709 13:02:08.260334 (+1138200us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:08.260387 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:08.260439 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.260533 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.260701 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":147935,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":46,"thread_start_us":1137770,"threads_started":1}
I20260709 13:02:08.199740  4027 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:08.222793  1638 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1698) took 4046 ms (client timeout 1821 ms). Trace:
W20260709 13:02:08.267076  1638 rpcz_store.cc:269] 0709 13:02:04.176594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.226818 (+ 50224us) service_pool.cc:224] Handling call
0709 13:02:08.222568 (+3995750us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b5400246b54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:08.222715 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":796906240}
W20260709 13:02:08.241928  1644 rpcz_store.cc:269] 0709 13:02:06.584389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.073188 (+1488799us) service_pool.cc:224] Handling call
0709 13:02:08.158999 (+ 85811us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.158602  3876 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:08.214696  3991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.125s	user 0.037s	sys 0.064s
I20260709 13:02:08.207392  4120 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:08.194883  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1825) took 3450 ms. Trace:
W20260709 13:02:08.254611  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 987896us
I20260709 13:02:08.247467  1564 rpcz_store.cc:276] 0709 13:02:03.654645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.178223 (+523578us) service_pool.cc:224] Handling call
0709 13:02:04.178652 (+   429us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:04.179188 (+   536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.179231 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:04.236392 (+ 57161us) log.cc:844] Serialized 52 byte log entry
0709 13:02:04.606956 (+370564us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:04.618218 (+ 11262us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.618439 (+   221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.618516 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.840776 (+2222260us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:08.169717 (+1328941us) raft_consensus.cc:1685] finished
0709 13:02:08.176760 (+  7043us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.176988 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10849,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":173,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":24772,"spinlock_wait_cycles":700251392,"thread_start_us":233428,"threads_started":1,"wal-append.queue_time_us":537328}
I20260709 13:02:08.213446  4104 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:08.258891  1563 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.946s	user 0.043s	sys 0.103s
I20260709 13:02:08.200173  3975 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:08.247014  2299 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEARNER]: Advancing to term 7
I20260709 13:02:08.296175  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:08.296777  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:08.195089  2298 rpcz_store.cc:276] 0709 13:02:04.742211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.966861 (+3224650us) service_pool.cc:224] Handling call
0709 13:02:08.016180 (+ 49319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.016473 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:08.016503 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:08.016550 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:08.016595 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.016659 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.016796 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.262380  1637 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:02:08.293800  4161 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.144s	user 0.001s	sys 0.001s
I20260709 13:02:08.313992  4011 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:08.283331  3991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.193s	user 0.040s	sys 0.071s
I20260709 13:02:08.284664  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:08.315899  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1707) took 2999 ms (client timeout 2998 ms). Trace:
W20260709 13:02:08.316488  1644 rpcz_store.cc:269] 0709 13:02:05.316033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.284390 (+2968357us) service_pool.cc:224] Handling call
0709 13:02:08.315790 (+ 31400us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.317586  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:08.286361  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:08.251147  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:08.209215  4150 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:08.249645  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:08.264367  4144 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:08.283915  2302 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:08.310770  1565 rpcz_store.cc:276] 0709 13:02:04.744642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.744781 (+   139us) service_pool.cc:224] Handling call
0709 13:02:04.745684 (+   903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.954166 (+208482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.954197 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.954243 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:04.954277 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.194637 (+3240360us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400276854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:08.194688 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.194827 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":479022976}
I20260709 13:02:08.312739  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:08.272096  3802 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.320130  1638 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 6: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:02:08.324759  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 52090us
I20260709 13:02:08.318194  1644 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
W20260709 13:02:08.331368  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1705) took 3016 ms (client timeout 3024 ms). Trace:
W20260709 13:02:08.332616  2268 connection.cc:256] RPC call timeout handler was delayed by -1.3175660789999597s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16590000000000002s for a response
W20260709 13:02:08.332659  1638 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 463) took 2187 ms (client timeout 2142 ms). Trace:
W20260709 13:02:08.341200  1609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.020s	sys 0.029s
W20260709 13:02:08.245404  1636 rpcz_store.cc:269] 0709 13:02:02.748541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.748688 (+   147us) service_pool.cc:224] Handling call
0709 13:02:05.186673 (+2437985us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b58006704d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1bfd0aa 00007fd7a1c7f447 00007fd7a1c97fe7 00007fd7a1c997de 00007fd7a1ca527c 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c
0709 13:02:08.023154 (+2836481us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026932608}
W20260709 13:02:08.341780  2268 connection.cc:256] RPC call timeout handler was delayed by -1.2313480250000453s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24900000000000003s for a response
I20260709 13:02:08.342463  1717 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:02:08.331929  1644 rpcz_store.cc:269] 0709 13:02:05.315443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.317332 (+3001889us) service_pool.cc:224] Handling call
0709 13:02:08.331271 (+ 13939us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.309273  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 601) took 3104 ms. Trace:
W20260709 13:02:08.343616  1609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.020s	sys 0.029s
W20260709 13:02:08.343482  2268 connection.cc:256] RPC call timeout handler was delayed by -1.1162615399999822s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
I20260709 13:02:08.251997  4065 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:08.264257  4094 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:08.352694  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 465810us
W20260709 13:02:08.352411  2268 connection.cc:256] RPC call timeout handler was delayed by -1.1162615399999822s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
I20260709 13:02:08.314031  3975 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:08.352264  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 627191us
W20260709 13:02:08.354154  2268 connection.cc:256] RPC call timeout handler was delayed by -1.0549714770000378s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2174s for a response
W20260709 13:02:08.333626  1638 rpcz_store.cc:269] 0709 13:02:06.152795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.286106 (+2133311us) service_pool.cc:224] Handling call
0709 13:02:08.332562 (+ 46456us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.326769  4121 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:08.360173  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:08.360915  1638 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:02:08.361380  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:08.361425  1638 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1706) took 3045 ms (client timeout 3404 ms). Trace:
I20260709 13:02:08.298190  4139 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:02:08.294587  3876 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:08.370422  2304 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 5: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:08.357951  4156 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:02:08.343927  2298 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:08.330219  1711 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:08.351631  2300 rpcz_store.cc:276] 0709 13:02:05.190537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.147873 (+2957336us) service_pool.cc:224] Handling call
0709 13:02:08.202551 (+ 54678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.264319 (+ 61768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.294395 (+ 30076us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.385643  4150 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:08.346469  3802 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:08.289472  4060 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:08.363570  4087 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:08.366654  4048 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:08.384749  4116 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:08.440506  4094 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:08.441938  4065 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:08.441865  4128 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:08.429999  3896 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:08.463466  2298 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 1. Current term is 2. Ops: []
W20260709 13:02:08.464986  4159 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:02:08.465283  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 797) took 3078 ms. Trace:
I20260709 13:02:08.434605  3926 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:08.467650  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 585) took 4375 ms. Trace:
I20260709 13:02:08.468334  2297 rpcz_store.cc:276] 0709 13:02:04.092626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.872382 (+2779756us) service_pool.cc:224] Handling call
0709 13:02:06.872903 (+   521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:06.873545 (+   642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:06.873592 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:06.874239 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 13:02:08.262517 (+1388278us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:08.266026 (+  3509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.286199 (+ 20173us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:08.466205 (+180006us) raft_consensus.cc:1685] finished
0709 13:02:08.466244 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.467518 (+  1274us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1696,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":108,"thread_start_us":1387665,"threads_started":1,"wal-append.queue_time_us":1457751}
I20260709 13:02:08.458710  4060 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:08.486656  2297 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:08.494127  3802 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:08.491266  4147 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:08.465922  2298 rpcz_store.cc:276] 0709 13:02:05.386706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.408312 (+3021606us) service_pool.cc:224] Handling call
0709 13:02:08.435793 (+ 27481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.464810 (+ 29017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.465194 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.556708  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 748) took 4424 ms. Trace:
W20260709 13:02:08.557320  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.553987  2299 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:08.562458  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 793) took 3241 ms. Trace:
I20260709 13:02:08.563017  2299 rpcz_store.cc:276] 0709 13:02:05.321200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.188449 (+2867249us) service_pool.cc:224] Handling call
0709 13:02:08.227310 (+ 38861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.561967 (+334657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.562358 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.565564  4144 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:08.569113  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:08.570006  4042 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:08.570322  4056 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:02:08.590509  4140 consensus_meta.cc:287] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.523s	user 0.013s	sys 0.000s
I20260709 13:02:08.600479  4140 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:08.603694  4140 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:08.574777  4128 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:08.599547  4056 raft_consensus.cc:697] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:08.609376  4056 consensus_queue.cc:237] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:08.678992  1534 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.908s	user 0.000s	sys 0.019s
W20260709 13:02:08.680146  1534 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.000s	sys 0.020s
W20260709 13:02:08.680930  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 910921us
I20260709 13:02:08.325153  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:08.689651  1560 raft_consensus.cc:2393] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 3: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:08.693081  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:08.695905  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:08.697273  1562 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:08.350651  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:08.700989  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:08.701910  1565 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:08.723214  1608 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.890s	user 0.009s	sys 0.040s
W20260709 13:02:08.723897  1608 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.010s	sys 0.041s
W20260709 13:02:08.362308  1638 rpcz_store.cc:269] 0709 13:02:05.315751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.359129 (+3043378us) service_pool.cc:224] Handling call
0709 13:02:08.361371 (+  2242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.736714  1635 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:08.738265  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:08.738708  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1708) took 3422 ms. Trace:
I20260709 13:02:08.739444  1636 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:08.703640  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:08.739528  1560 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:08.739387  1638 rpcz_store.cc:276] 0709 13:02:05.316320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.736645 (+3420325us) service_pool.cc:224] Handling call
0709 13:02:08.737450 (+   805us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.738144 (+   694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:08.738213 (+    69us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:08.738279 (+    66us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:08.738309 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.738373 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.738490 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:08.740397  1636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 457) took 3075 ms (client timeout 2748 ms). Trace:
I20260709 13:02:08.741380  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:08.570721  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:08.905821  3853 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:08.893810  1636 rpcz_store.cc:269] 0709 13:02:05.664886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.356400 (+2691514us) service_pool.cc:224] Handling call
0709 13:02:08.740262 (+383862us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.765734  1717 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:08.909832  3853 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:08.918282  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 347486us
I20260709 13:02:08.919397  1562 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:02:08.920928  4147 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:08.921093  3853 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:08.921814  4147 raft_consensus.cc:697] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:08.923736  1565 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:08.924562  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2201) took 1029 ms. Trace:
I20260709 13:02:08.924981  1565 rpcz_store.cc:276] 0709 13:02:07.894682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.700315 (+805633us) service_pool.cc:224] Handling call
0709 13:02:08.924496 (+224181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.928790  4060 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.929821  4060 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:08.930373  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:08.931887  1643 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:08.557627  2301 rpcz_store.cc:276] 0709 13:02:04.132064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.738961 (+3606897us) service_pool.cc:224] Handling call
0709 13:02:07.788799 (+ 49838us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:08.004514 (+215715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:08.004555 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:08.005397 (+   842us) log.cc:844] Serialized 52 byte log entry
0709 13:02:08.326077 (+320680us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:08.556007 (+229930us) log.cc:844] Serialized 14 byte log entry
0709 13:02:08.556325 (+   318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.556396 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:08.556439 (+    43us) raft_consensus.cc:1685] finished
0709 13:02:08.556451 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.556635 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":215256,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":119779,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":194,"thread_start_us":467367,"threads_started":2,"wal-append.queue_time_us":334105}
I20260709 13:02:08.926761  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:08.763321  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:08.786237  3701 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:08.762128  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:08.765021  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:08.764246  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:08.799044  4041 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:08.806900  4124 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:08.560726  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 799) took 3060 ms. Trace:
W20260709 13:02:08.942890  1609 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.006s	sys 0.037s
I20260709 13:02:08.949071  1715 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:08.949175  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1824) took 4204 ms. Trace:
I20260709 13:02:08.949750  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:08.950203  1682 connection.cc:256] RPC call timeout handler was delayed by -1.674173498000016s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
I20260709 13:02:08.959229  1636 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 6: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:08.961503  3995 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:08.961783  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:08.957942  1682 connection.cc:256] RPC call timeout handler was delayed by -1.5364734980000208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1902s for a response
I20260709 13:02:08.907339  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2195) took 1010 ms. Trace:
W20260709 13:02:08.963563  1682 connection.cc:256] RPC call timeout handler was delayed by -1.2754734979999967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21920000000000003s for a response
W20260709 13:02:08.959448  1609 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.007s	sys 0.037s
I20260709 13:02:08.958510  1710 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:08.972157  4147 consensus_queue.cc:237] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:08.972867  4093 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 4.116s	user 0.028s	sys 0.043s
I20260709 13:02:08.962736  1636 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:08.978118  1564 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:08.985271  3896 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.966689  1560 rpcz_store.cc:276] 0709 13:02:07.890321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.692209 (+801888us) service_pool.cc:224] Handling call
0709 13:02:08.900874 (+208665us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:08.989715  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:08.990198  1709 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:08.991607  4042 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.996178  4042 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:08.998140  4041 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:09.000103  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:08.999238  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 641030us
I20260709 13:02:09.001159  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:08.993177  3701 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.004890  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:09.005147  3701 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:08.994447  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:09.005224  3896 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.005939  1714 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
W20260709 13:02:08.998874  4151 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.749s	user 0.008s	sys 0.017s
I20260709 13:02:09.010946  1565 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:09.012733  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:09.005934  1609 leader_election.cc:336] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.891s (SENT)
W20260709 13:02:09.012408  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1709) took 3695 ms (client timeout 1764 ms). Trace:
I20260709 13:02:09.010106  1716 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
W20260709 13:02:09.013747  1640 rpcz_store.cc:269] 0709 13:02:05.316622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.728395 (+1411773us) service_pool.cc:224] Handling call
0709 13:02:09.008168 (+2279773us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54001950d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:09.011594 (+  3426us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315757440}
I20260709 13:02:08.956843  1563 rpcz_store.cc:276] 0709 13:02:04.744295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.744429 (+   134us) service_pool.cc:224] Handling call
0709 13:02:05.112255 (+367826us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:05.240778 (+128523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:05.240810 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:05.241517 (+   707us) log.cc:844] Serialized 52 byte log entry
0709 13:02:06.149728 (+908211us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:08.297232 (+2147504us) log.cc:844] Serialized 14 byte log entry
0709 13:02:08.742734 (+445502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.742893 (+   159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:08.743039 (+   146us) raft_consensus.cc:1685] finished
0709 13:02:08.743057 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.948608 (+205551us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b54001ff750. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:08.948882 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127913,"prepare.queue_time_us":379953,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":193901,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":384,"spinlock_wait_cycles":845163264,"thread_start_us":3335238,"threads_started":3,"wal-append.queue_time_us":1876097,"wal-append.run_cpu_time_us":2262,"wal-append.run_wall_time_us":1024963}
I20260709 13:02:09.022256  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:09.025444  4060 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:09.026062  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:09.019883  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:09.027935  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:08.994330  4124 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:09.033321  1609 leader_election.cc:336] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.916s (SENT)
W20260709 13:02:09.033320  4152 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.956s	user 0.005s	sys 0.019s
I20260709 13:02:09.036383  1640 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.038375  3824 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:09.036434  1565 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:09.040223  1609 leader_election.cc:336] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.924s (SENT)
I20260709 13:02:09.040335  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 1870) took 3526 ms. Trace:
I20260709 13:02:09.047700  1636 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:09.049839  2296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 600) took 3961 ms (client timeout 2109 ms). Trace:
W20260709 13:02:09.050505  2296 rpcz_store.cc:269] 0709 13:02:05.087808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.178809 (+ 91001us) service_pool.cc:224] Handling call
0709 13:02:09.049274 (+3870465us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400710f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:09.049580 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3554179456}
I20260709 13:02:09.042078  1712 rpcz_store.cc:276] 0709 13:02:05.513882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.513996 (+   114us) service_pool.cc:224] Handling call
0709 13:02:05.527680 (+ 13684us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:05.527994 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:05.528032 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:05.540826 (+ 12794us) log.cc:844] Serialized 26 byte log entry
0709 13:02:07.726620 (+2185794us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:07.726662 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.726736 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:08.997571 (+1270835us) spinlock_profiling.cc:243] Waited 9.55 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:09.039720 (+ 42149us) raft_consensus.cc:1685] finished
0709 13:02:09.039741 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.040131 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21973376,"thread_start_us":1992808,"threads_started":1,"wal-append.queue_time_us":2482016}
W20260709 13:02:09.053828  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 461) took 3315 ms (client timeout 2946 ms). Trace:
I20260709 13:02:09.055239  3995 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:09.055487  1609 leader_election.cc:336] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.962s (SENT)
W20260709 13:02:09.055665  1569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1830) took 4230 ms (client timeout 1916 ms). Trace:
I20260709 13:02:09.056149  3995 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:09.056715  1709 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.058071  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2181) took 4126 ms. Trace:
W20260709 13:02:09.056187  1569 rpcz_store.cc:269] 0709 13:02:04.824880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.055306 (+4230426us) service_pool.cc:211] Skipping call since client already timed out
0709 13:02:09.055478 (+   172us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:02:09.063377  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.063608  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.064522  1569 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:09.014165  2300 rpcz_store.cc:276] 0709 13:02:05.499481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.443694 (+2944213us) service_pool.cc:224] Handling call
0709 13:02:08.469413 (+ 25719us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:08.469925 (+   512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:08.469976 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:08.470620 (+   644us) log.cc:844] Serialized 26 byte log entry
0709 13:02:08.552001 (+ 81381us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:08.552055 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.552129 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:08.553808 (+  1679us) raft_consensus.cc:1685] finished
0709 13:02:08.553831 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.559525 (+  5694us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":912,"raft.queue_time_us":1966,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":206,"spinlock_wait_cycles":31471360,"thread_start_us":78878,"threads_started":1,"wal-append.queue_time_us":81266}
I20260709 13:02:09.064776  4149 consensus_queue.cc:1059] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:09.064996  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.065447  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:09.065987  1560 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:09.068152  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2178) took 4138 ms. Trace:
I20260709 13:02:09.068760  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.068625  1568 rpcz_store.cc:276] 0709 13:02:04.929840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.492020 (+562180us) service_pool.cc:224] Handling call
0709 13:02:05.492349 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.720306 (+2227957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.067623 (+1347317us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b5400216954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:09.067988 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4118090624}
I20260709 13:02:09.071599  4045 raft_consensus.cc:3055] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:09.071774  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:09.072062  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1702) took 3773 ms (client timeout 1935 ms). Trace:
W20260709 13:02:09.072715  1639 rpcz_store.cc:269] 0709 13:02:05.298468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.311033 (+ 12565us) service_pool.cc:224] Handling call
0709 13:02:09.071569 (+3760536us) spinlock_profiling.cc:243] Waited 80.4 ms on lock 0x7b5400154d54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:09.071892 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184886656}
I20260709 13:02:09.073540  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:09.074309  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:09.028216  4042 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:09.079938  1532 connection.cc:256] RPC call timeout handler was delayed by -1.376746154999978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18910000000000002s for a response
I20260709 13:02:09.079865  4149 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:09.081156  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.066282  1569 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:09.054486  1644 rpcz_store.cc:269] 0709 13:02:05.738590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.053260 (+3314670us) service_pool.cc:211] Skipping call since client already timed out
0709 13:02:09.053507 (+   247us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:02:09.058596  1567 rpcz_store.cc:276] 0709 13:02:04.931240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.902553 (+1971313us) service_pool.cc:224] Handling call
0709 13:02:06.908333 (+  5780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.091002 (+1182669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.057444 (+966442us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400250154. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:09.057850 (+   406us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2665714048}
W20260709 13:02:09.088207  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 99473.4us
I20260709 13:02:09.090644  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:09.033547  4124 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:09.072079  4045 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:02:09.191039  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 50071.6us
I20260709 13:02:09.185272  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:09.095430  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2200) took 1201 ms. Trace:
I20260709 13:02:09.187682  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:09.192478  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2176) took 4281 ms. Trace:
I20260709 13:02:09.093173  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:09.180586  1710 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:09.208099  4045 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.210168  4045 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.099443  4094 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.211052  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:09.211628  4094 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.212466  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:09.214130  4094 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:09.091536  1638 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:09.215757  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:09.186232  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:09.218712  1565 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:09.204190  1714 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:09.092748  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:09.095484  1532 connection.cc:256] RPC call timeout handler was delayed by -1.2031942360000016s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
I20260709 13:02:09.204684  1566 rpcz_store.cc:276] 0709 13:02:04.911053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.409671 (+498618us) service_pool.cc:224] Handling call
0709 13:02:05.410043 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.249108 (+2839065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.191892 (+942784us) spinlock_profiling.cc:243] Waited 2.77 s on lock 0x7b54001ffc54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:09.192258 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6365708032}
W20260709 13:02:09.225602  1532 connection.cc:256] RPC call timeout handler was delayed by -1.1825288019999789s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
I20260709 13:02:09.217760  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:09.203910  1561 rpcz_store.cc:276] 0709 13:02:07.894291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.266805 (+372514us) service_pool.cc:224] Handling call
0709 13:02:09.080752 (+813947us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:02:09.095353 (+ 14601us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28756864}
I20260709 13:02:09.226387  1644 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:09.226502  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:09.229758  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 174392us
W20260709 13:02:09.230268  1532 connection.cc:256] RPC call timeout handler was delayed by -1.13194615499998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2163s for a response
I20260709 13:02:09.237506  3896 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:09.240924  4155 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.983s	user 0.007s	sys 0.018s
I20260709 13:02:09.244146  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.111101  4041 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:09.250967  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2197) took 1360 ms. Trace:
I20260709 13:02:09.251416  1565 rpcz_store.cc:276] 0709 13:02:07.891041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.223872 (+1332831us) service_pool.cc:224] Handling call
0709 13:02:09.250886 (+ 27014us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.254863  4118 raft_consensus.cc:3055] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:09.255434  4118 raft_consensus.cc:740] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:09.255832  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 792) took 4567 ms. Trace:
I20260709 13:02:09.256258  2302 rpcz_store.cc:276] 0709 13:02:04.688394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.848436 (+3160042us) service_pool.cc:224] Handling call
0709 13:02:07.849057 (+   621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.327972 (+478915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.255344 (+927372us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b5400600ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:09.255634 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":999279488}
I20260709 13:02:09.256783  4118 consensus_queue.cc:260] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.261621  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 587) took 5168 ms. Trace:
I20260709 13:02:09.263690  1709 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:09.272619  4099 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:09.273618  1567 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:09.275497  1567 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:09.276113  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2187) took 4310 ms. Trace:
I20260709 13:02:09.225229  3995 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:09.276813  1567 rpcz_store.cc:276] 0709 13:02:04.966055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.274500 (+4308445us) service_pool.cc:224] Handling call
0709 13:02:09.275023 (+   523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.275864 (+   841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.276038 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.227414  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:09.262231  2305 rpcz_store.cc:276] 0709 13:02:04.093268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.972126 (+2878858us) service_pool.cc:224] Handling call
0709 13:02:07.715764 (+743638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.243478 (+527714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.260960 (+1017482us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b54004af454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:09.261342 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1211070336}
I20260709 13:02:09.282480  1640 raft_consensus.cc:2410] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:02:09.287622  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 52126.5us
I20260709 13:02:09.288699  1636 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:09.294016  3865 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:09.294342  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:09.294950  3865 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.295749  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:09.296440  1567 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.297272  3865 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:09.258733  4118 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.300379  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.301779  2300 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:09.301646  4015 consensus_queue.cc:1059] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:09.302000  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.303644  1713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:36054
I20260709 13:02:09.303349  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.281143  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.304591  1713 raft_consensus.cc:606] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Received request to transfer leadership
I20260709 13:02:09.305083  1713 raft_consensus.cc:614] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:09.305169  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.303984  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.306864  2298 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:09.307046  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.305752  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:09.309572  2300 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.312181  2302 raft_consensus.cc:1240] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:02:09.314198  2296 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 7. Ops: []
I20260709 13:02:09.315253  4013 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:09.315722  4013 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:09.316139  4013 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.320521  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.321283  1568 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:09.322260  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:09.323213  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 45638.5us
W20260709 13:02:09.327852  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.324132  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:09.324642  2269 connection.cc:256] RPC call timeout handler was delayed by -1.1357798010000124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27480000000000004s for a response
I20260709 13:02:09.328969  1568 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:09.294113  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.332643  1569 raft_consensus.cc:2410] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:09.305042  1566 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:02:09.328379  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:09.266635  3701 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:09.095152  4149 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:09.338805  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 47221.5us
W20260709 13:02:09.342972  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 50076.5us
I20260709 13:02:09.278271  4094 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:09.337590  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 157653us
I20260709 13:02:09.345434  3508 consensus_queue.cc:1059] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:09.354298  2300 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
W20260709 13:02:09.346939  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 464) took 3157 ms (client timeout 1609 ms). Trace:
W20260709 13:02:09.385773  1642 rpcz_store.cc:269] 0709 13:02:06.189229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.731295 (+1542066us) service_pool.cc:224] Handling call
0709 13:02:09.346206 (+1614911us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54001e9954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:09.346668 (+   462us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":546173312}
I20260709 13:02:09.347528  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:09.348107  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 452) took 4458 ms. Trace:
W20260709 13:02:09.349709  2304 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 614) took 2472 ms (client timeout 1737 ms). Trace:
W20260709 13:02:09.352200  1564 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 427) took 1377 ms (client timeout 1819 ms). Trace:
I20260709 13:02:09.352944  1567 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:09.356617  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.350582  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:09.345475  4015 consensus_queue.cc:1059] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:09.362592  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2186) took 4396 ms. Trace:
I20260709 13:02:09.363214  1639 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:09.369382  3945 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.363250  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:09.372426  3538 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:09.372942  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.397187  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.397387  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.397795  1568 raft_consensus.cc:2410] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:09.397878  2296 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:09.398217  1563 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:09.398738  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.398841  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.399109  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.399230  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:09.399991  2296 raft_consensus.cc:2393] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:09.375375  1644 raft_consensus.cc:2410] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 7 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:02:09.400303  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.379586  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.388862  1641 rpcz_store.cc:276] 0709 13:02:04.890070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.890390 (+   320us) service_pool.cc:224] Handling call
0709 13:02:04.926468 (+ 36078us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.297442 (+370974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.347590 (+4050148us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b5400125a54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:09.347915 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":851084032}
I20260709 13:02:09.401206  1568 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:09.401350  2305 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:09.380013  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2155) took 4391 ms. Trace:
I20260709 13:02:09.402052  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:09.403048  1641 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 6: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:02:09.380977  2303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 609) took 4017 ms (client timeout 2568 ms). Trace:
I20260709 13:02:09.390388  1565 rpcz_store.cc:276] 0709 13:02:04.965789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.280657 (+4314868us) service_pool.cc:224] Handling call
0709 13:02:09.358617 (+ 77960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.362109 (+  3492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.362154 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.362243 (+    89us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:09.362283 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.362347 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.362496 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.403630  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:09.403873  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:09.404196  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.404637  1641 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:09.404810  1565 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:09.405643  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:09.389794  1564 rpcz_store.cc:269] 0709 13:02:07.975206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.295857 (+320651us) service_pool.cc:224] Handling call
0709 13:02:09.351891 (+1056034us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b5400276854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:09.352096 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":914233600}
I20260709 13:02:09.407267  1565 raft_consensus.cc:1240] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:09.376186  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2158) took 4378 ms. Trace:
W20260709 13:02:09.407871  4157 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 1.311s	user 0.005s	sys 0.025s
I20260709 13:02:09.408020  1635 rpcz_store.cc:276] 0709 13:02:04.998060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.078548 (+ 80488us) service_pool.cc:224] Handling call
0709 13:02:08.018393 (+2939845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.737389 (+718996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.374814 (+637425us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b5400123c54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:09.375893 (+  1079us) spinlock_profiling.cc:243] Waited 2.89 s on lock 0x7b5400123c50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:09.376107 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8279636736}
I20260709 13:02:09.402242  1637 rpcz_store.cc:276] 0709 13:02:04.988317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.998292 (+  9975us) service_pool.cc:224] Handling call
0709 13:02:08.241149 (+3242857us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.313256 (+ 72107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.379640 (+1066384us) spinlock_profiling.cc:243] Waited 3.13 s on lock 0x7b5400246b50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:09.379849 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7207473408}
I20260709 13:02:09.408811  4013 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:09.389245  2304 rpcz_store.cc:269] 0709 13:02:06.877181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.925480 (+ 48299us) service_pool.cc:224] Handling call
0709 13:02:09.349287 (+2423807us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54004998d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:09.349511 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3288148224}
I20260709 13:02:09.407899  1564 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:09.412058  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2189) took 4444 ms. Trace:
I20260709 13:02:09.412386  1564 rpcz_store.cc:276] 0709 13:02:04.967436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.407275 (+4439839us) service_pool.cc:224] Handling call
0709 13:02:09.407635 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.411767 (+  4132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.411972 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.414535  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2192) took 3284 ms. Trace:
I20260709 13:02:09.415546  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:09.403388  2303 rpcz_store.cc:269] 0709 13:02:05.363918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.722087 (+1358169us) service_pool.cc:224] Handling call
0709 13:02:09.380588 (+2658501us) spinlock_profiling.cc:243] Waited 938 ms on lock 0x7b54002a44d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:09.380775 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2156694784}
I20260709 13:02:09.400391  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:09.405526  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:09.417193  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 54083.2us
I20260709 13:02:09.416981  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:09.419241  1567 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.417003  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:09.407872  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2188) took 4441 ms. Trace:
I20260709 13:02:09.414899  1566 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:09.420418  1565 rpcz_store.cc:276] 0709 13:02:04.966391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.406673 (+4440282us) service_pool.cc:224] Handling call
0709 13:02:09.407044 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.407637 (+   593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.407822 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.417008  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:09.402310  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:09.417907  1718 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:09.423568  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 478) took 1197 ms. Trace:
I20260709 13:02:09.414875  1564 rpcz_store.cc:276] 0709 13:02:06.129864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.413339 (+3283475us) service_pool.cc:224] Handling call
0709 13:02:09.413743 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.414141 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.414172 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.414213 (+    41us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:09.414247 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.414319 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.414472 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2296,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":132}
I20260709 13:02:09.418018  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 481) took 1057 ms. Trace:
I20260709 13:02:09.401291  2269 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:09.425987  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 65158.9us
I20260709 13:02:09.416970  2298 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:09.431854  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:09.428856  1568 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:09.424072  1718 rpcz_store.cc:276] 0709 13:02:08.225935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.243274 (+ 17339us) service_pool.cc:224] Handling call
0709 13:02:09.350258 (+1106984us) spinlock_profiling.cc:243] Waited 7.84 ms on lock 0x7b58001e4598. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:02:09.423487 (+ 73229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18027776}
W20260709 13:02:09.430466  1609 leader_election.cc:336] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 2.568s (SENT)
I20260709 13:02:09.430397  1682 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:09.431371  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:09.430104  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:09.425886  1717 rpcz_store.cc:276] 0709 13:02:08.360980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.361129 (+   149us) service_pool.cc:224] Handling call
0709 13:02:09.417819 (+1056690us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b5400337cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:09.417936 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":927192320}
I20260709 13:02:09.403183  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.435109  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 421) took 3296 ms. Trace:
I20260709 13:02:09.436038  1641 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 6: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:09.436151  1566 rpcz_store.cc:276] 0709 13:02:06.130809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.414372 (+3283563us) service_pool.cc:224] Handling call
0709 13:02:09.414742 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.427107 (+ 12365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.427369 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:09.439705  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 51192.9us
I20260709 13:02:09.417891  3701 raft_consensus.cc:2764] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:02:09.432058  1560 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:09.456717  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:09.449666  3945 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.427659  4094 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.463917  1563 raft_consensus.cc:2376] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4. Re-sending same reply.
I20260709 13:02:09.447908  3538 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:09.462031  4021 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:09.461787  4015 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:02:09.467921  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.456537  4099 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:09.468394  4021 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:09.471782  4175 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:02:09.471843  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:09.468966  1682 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:09.476464  4021 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.479429  4015 raft_consensus.cc:3055] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:09.479939  4015 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:09.479997  3865 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:09.480608  3865 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.481053  4015 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.481657  4013 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:09.482622  4015 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.485625  1609 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:09.487535  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2164) took 3950 ms. Trace:
I20260709 13:02:09.474448  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:09.495107  1640 rpcz_store.cc:276] 0709 13:02:05.523312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.292277 (+3768965us) service_pool.cc:224] Handling call
0709 13:02:09.355170 (+ 62893us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.467536 (+112366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.467562 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.473674 (+  6112us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:09.473727 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.473800 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.473936 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4994,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":9330}
I20260709 13:02:09.497798  2299 raft_consensus.cc:1240] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:09.498678  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:09.504097  4066 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:02:09.500494  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:09.502108  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:09.481086  3865 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.499873  2269 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:09.504807  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.185010   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:09.510787  4099 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:09.512727  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 422) took 3381 ms. Trace:
I20260709 13:02:09.503358  4149 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:09.511129  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:09.515560  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 54974us
W20260709 13:02:09.513302  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 87377.9us
I20260709 13:02:09.519246  1565 rpcz_store.cc:276] 0709 13:02:06.131312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.442308 (+3310996us) service_pool.cc:224] Handling call
0709 13:02:09.464573 (+ 22265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.489363 (+ 24790us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.489474 (+   111us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.508087 (+ 18613us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:09.512393 (+  4306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.512488 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.512653 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.523495  4099 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.524618  1682 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:09.518441  3824 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:09.528091  3824 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:02:09.529299  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 53457.6us
I20260709 13:02:09.529400  3824 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.531277  3963 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:09.532045  1561 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:02:09.533154  3850 consensus_queue.cc:1059] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:09.533517  4021 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:09.533851  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 424) took 3385 ms. Trace:
I20260709 13:02:09.534789  3855 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:09.535241  3855 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:09.536852  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:09.537391  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:09.534428  1561 rpcz_store.cc:276] 0709 13:02:06.148974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.532821 (+3383847us) service_pool.cc:224] Handling call
0709 13:02:09.533281 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.533524 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.533551 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.533595 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:09.533630 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.533690 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.533795 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.539539  4021 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:09.539873  4189 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:09.540427  4189 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.540530  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:09.540889  4189 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.540861  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:09.541567  3508 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:09.542035  1711 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:09.542548  4066 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:09.542011  3508 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.542992  4066 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.536409  3855 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.543481  4066 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.543174  3508 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:09.543428  1682 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:09.544008  4013 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:09.519763  1562 consensus_meta.cc:287] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.600s	user 0.017s	sys 0.002s
I20260709 13:02:09.551775  1562 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:09.549518  3701 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:02:09.549752  1533 leader_election.cc:336] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.821s (SENT)
I20260709 13:02:09.543152  1637 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:09.539834  1561 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:09.553534  3876 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:09.486297  3995 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:02:09.540429  4021 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:09.558496  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.559413  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:09.546732  3824 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:09.563082  1532 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:09.569723  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2198) took 1672 ms. Trace:
I20260709 13:02:09.570454  1561 rpcz_store.cc:276] 0709 13:02:07.891414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.538645 (+1647231us) service_pool.cc:224] Handling call
0709 13:02:09.539120 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.561986 (+ 22866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.563723 (+  1737us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.556087  4013 raft_consensus.cc:3055] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:09.562544  4124 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:09.578383  1641 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:09.574460  4048 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.579308  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1712) took 4210 ms. Trace:
I20260709 13:02:09.541021  4060 consensus_queue.cc:1059] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:02:09.579576  1563 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:09.581519  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 459) took 3883 ms. Trace:
I20260709 13:02:09.582896  1641 rpcz_store.cc:276] 0709 13:02:05.695951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.539063 (+3843112us) service_pool.cc:224] Handling call
0709 13:02:09.548533 (+  9470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:09.578830 (+ 30297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.579177 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.568332  4021 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:09.586308  2269 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:09.597941  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 48886.9us
I20260709 13:02:09.581013  1638 rpcz_store.cc:276] 0709 13:02:05.365192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.275645 (+3910453us) service_pool.cc:224] Handling call
0709 13:02:09.276069 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.276440 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:09.450366 (+173926us) log.cc:844] Serialized 14 byte log entry
0709 13:02:09.569393 (+119027us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.572778 (+  3385us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:09.574311 (+  1533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.575048 (+   737us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.576010 (+   962us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5800,"prepare.queue_time_us":250846,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"thread_start_us":263834,"threads_started":2}
I20260709 13:02:09.577517  4013 raft_consensus.cc:740] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:09.619010  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 109201us
I20260709 13:02:09.621068  1641 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:09.585134  4021 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:09.603447  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 44192.6us
I20260709 13:02:09.633328  1567 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:09.596537  4192 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:09.604939  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2165) took 4077 ms. Trace:
W20260709 13:02:09.639178  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 63225.2us
I20260709 13:02:09.624102  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1715) took 3483 ms. Trace:
W20260709 13:02:09.637145  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 81550.9us
I20260709 13:02:09.640304  1641 rpcz_store.cc:276] 0709 13:02:06.140898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.583856 (+3442958us) service_pool.cc:224] Handling call
0709 13:02:09.620896 (+ 37040us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.623859 (+  2963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.624040 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.640763  1563 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:09.587303  4048 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.635975  1562 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:09.636617  4192 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.651863  4198 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:09.655107  3963 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:09.656756  3963 raft_consensus.cc:3055] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:09.657202  3963 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:09.658983  4128 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:09.659529  4128 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.659901  4128 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.665911  3855 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.667816  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2153) took 5959 ms. Trace:
I20260709 13:02:09.668247  1643 rpcz_store.cc:276] 0709 13:02:03.708622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.080797 (+372175us) service_pool.cc:224] Handling call
0709 13:02:04.081207 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.932416 (+4851209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.667537 (+735121us) spinlock_profiling.cc:243] Waited 4.85 s on lock 0x7b54002d8554. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:09.667755 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11154689408}
I20260709 13:02:09.669932  1643 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:09.635067  4060 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:02:09.638280  4106 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:02:09.663971  1567 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:09.673338  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 458) took 3983 ms. Trace:
I20260709 13:02:09.642966  4118 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:09.655088  4144 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:09.643922  4094 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:09.683406  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2166) took 3352 ms. Trace:
I20260709 13:02:09.683851  1638 rpcz_store.cc:276] 0709 13:02:06.327548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.608461 (+3280913us) service_pool.cc:224] Handling call
0709 13:02:09.648258 (+ 39797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.678691 (+ 30433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.678732 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.678789 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:09.678834 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.678934 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.679082 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:09.637969  1642 rpcz_store.cc:276] 0709 13:02:05.527182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.386768 (+3859586us) service_pool.cc:224] Handling call
0709 13:02:09.387250 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.387671 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:09.464108 (+ 76437us) log.cc:844] Serialized 14 byte log entry
0709 13:02:09.604585 (+140477us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.604636 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:09.604677 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.604741 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.604886 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":140196,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"thread_start_us":141559,"threads_started":2,"wal-append.queue_time_us":139100}
I20260709 13:02:09.673970  1644 rpcz_store.cc:276] 0709 13:02:05.689878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.538318 (+3848440us) service_pool.cc:224] Handling call
0709 13:02:09.538746 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:09.561402 (+ 22656us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.561448 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:09.567995 (+  6547us) log.cc:844] Serialized 26 byte log entry
0709 13:02:09.625887 (+ 57892us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:09.625977 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.648474 (+ 22497us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:09.673082 (+ 24608us) raft_consensus.cc:1685] finished
0709 13:02:09.673108 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.673270 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18284,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":4259,"thread_start_us":43125,"threads_started":1,"wal-append.queue_time_us":81967}
I20260709 13:02:09.682279  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2160) took 4166 ms. Trace:
I20260709 13:02:09.688380  1636 rpcz_store.cc:276] 0709 13:02:05.514678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.290550 (+3775872us) service_pool.cc:224] Handling call
0709 13:02:09.291173 (+   623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.291616 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:09.560827 (+269211us) log.cc:844] Serialized 14 byte log entry
0709 13:02:09.680998 (+120171us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.681070 (+    72us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:09.681136 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.681242 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.681386 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37260,"prepare.queue_time_us":235993,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"thread_start_us":305802,"threads_started":2}
I20260709 13:02:09.690603  3865 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.658622  3963 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.694629  1639 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
W20260709 13:02:09.701531  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 250085us
I20260709 13:02:09.663515  4192 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.717388  1563 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:09.679494  4118 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.740207  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.741014  2302 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:09.742446  4106 raft_consensus.cc:3055] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:09.742952  4106 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:09.744226  4106 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.745926  4106 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:09.747089  4094 consensus_queue.cc:1059] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:09.747311  4144 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.747586  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:09.748179  4094 raft_consensus.cc:3055] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:09.748376  1642 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.749559  1637 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:02:09.749944  4144 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:09.750339  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 462) took 3641 ms. Trace:
W20260709 13:02:09.752128  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 44374.3us
I20260709 13:02:09.752408  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:09.753365  1640 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:09.672494  3508 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:09.755784  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 132106us
I20260709 13:02:09.686764  4124 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:09.715145  3963 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.687884  4200 raft_consensus.cc:3055] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:02:09.756896  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 107868us
W20260709 13:02:09.760785  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.761471  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.751339  4066 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
W20260709 13:02:09.765851  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 118109us
I20260709 13:02:09.727147  4118 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:09.762701  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 183979us
I20260709 13:02:09.735225  3865 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:09.771977  1532 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:09.775180  3538 raft_consensus.cc:2764] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:02:09.775395  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 51032.2us
I20260709 13:02:09.748623  4094 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:09.776666  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.783551  2303 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:02:09.783089  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 158199us
I20260709 13:02:09.785432  1683 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:09.763677  4200 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:09.766993  1560 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:09.790221  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.793215  2302 raft_consensus.cc:2410] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:09.740280  4205 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
W20260709 13:02:09.802865  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.806751  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:09.808751  1532 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:09.809749  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:09.810424  1532 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:09.750813  1637 rpcz_store.cc:276] 0709 13:02:06.108604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.581248 (+3472644us) service_pool.cc:224] Handling call
0709 13:02:09.749000 (+167752us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ceee 0000000000375939 00007fd7a6bf01e0 00007fd7a6cb90c4 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:02:09.749173 (+   173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.749998 (+   825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.750258 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15972608}
W20260709 13:02:09.812638  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 45921.6us
I20260709 13:02:09.801630  1562 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:09.765236  4013 consensus_queue.cc:260] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.817297  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:09.818564  2303 raft_consensus.cc:2376] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 2. Re-sending same reply.
I20260709 13:02:09.825699  4202 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 13:02:09.826388  3508 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:09.820312  4149 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:09.827255  1532 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:09.828394  4124 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.835525  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:09.838018  1532 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:09.838371  4094 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.838625  4200 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.839602  4045 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:09.839869  4094 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.840157  4045 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:09.840210  4200 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.840731  4045 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.845651  2302 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.847929  4212 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:09.848583  4149 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:09.847033  3508 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:09.849490  3743 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:09.855139  4195 log.cc:927] Time spent T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:02:09.807046  3701 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:02:09.858333  4013 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:09.799585  3538 consensus_queue.cc:1059] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:02:09.859401  3538 raft_consensus.cc:3055] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:09.859767  3538 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:09.860028  3995 raft_consensus.cc:3055] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:09.860466  3995 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:09.860580  4187 raft_consensus.cc:3055] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:09.861322  4187 raft_consensus.cc:740] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:09.861536  3995 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.864763  1532 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:09.867093  3743 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:09.867507  3743 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.867823  3743 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.872972  1642 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
W20260709 13:02:09.875010  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.879160  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 76329.6us
I20260709 13:02:09.883138  3947 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.883785  3947 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.887621  4128 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.890395  4128 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:09.862895  4187 consensus_queue.cc:260] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.891391  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:09.892246  2303 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:09.860674  3538 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:09.893949  3538 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:09.895543  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 460) took 4171 ms. Trace:
I20260709 13:02:09.895730  4212 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:09.896013  1635 rpcz_store.cc:276] 0709 13:02:05.724703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.544079 (+3819376us) service_pool.cc:224] Handling call
0709 13:02:09.894491 (+350412us) spinlock_profiling.cc:243] Waited 966 us on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:02:09.894757 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.895059 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.895096 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:09.895140 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:09.895208 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.895333 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.895480 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2222464}
I20260709 13:02:09.898813  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:09.899619  4147 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:09.901533  4147 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:09.902055  4147 raft_consensus.cc:697] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:09.904453  4021 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:09.846110  4066 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:09.883178  4041 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:02:09.908499  3508 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:09.912678  4147 consensus_queue.cc:237] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.869201  3995 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:09.914666  3947 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:09.916054  4019 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:09.917582  1532 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:09.892300  4187 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:09.918306  4048 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:09.875631  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 55492.5us
W20260709 13:02:09.940706  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.945988  4045 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:09.955968  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:09.958082  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:09.962021  1607 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:09.963532  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:09.964707  4124 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:09.965085  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:09.965222  4124 raft_consensus.cc:697] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:09.966226  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:09.963367  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:09.966259  4124 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:09.824995  4189 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:09.973518  2303 raft_consensus.cc:2410] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:09.973814  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:09.970651  1462 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45164 (request call id 362) took 10150 ms. Trace:
I20260709 13:02:09.990341  4128 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:09.982879  4048 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:09.938090  3508 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.995893  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:09.996393  1711 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:09.990341  4144 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:10.008766  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.009372  4144 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:10.009984  4144 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.010381  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:09.989322  4149 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:10.007222  1639 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:09.969250  3850 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:09.969483  4066 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:10.008026  1637 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 8 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:02:09.997820  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.028537  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.029062  1636 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:10.006795  3508 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.013190  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:10.013622  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:10.034377  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.014406  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:10.015367  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.019534  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.041996  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.043452  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 467) took 1304 ms. Trace:
I20260709 13:02:09.993598  1462 rpcz_store.cc:276] 0709 13:01:59.820048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.820556 (+   508us) service_pool.cc:224] Handling call
0709 13:02:09.970364 (+10149808us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:02.965236 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:02.966016 (+   780us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:02.966068 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:02:02.966090 (+    22us) tablet.cc:662] Decoding operations
0709 13:02:03.228813 (+262723us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:03.228907 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:03.228941 (+    34us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:02:03.230477 (+  1536us) tablet.cc:701] Row locks acquired
0709 13:02:03.230525 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:02:03.230707 (+   182us) write_op.cc:270] Start()
0709 13:02:03.294533 (+ 63826us) write_op.cc:276] Timestamp: P: 1783602123230638 usec, L: 0
0709 13:02:03.294569 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:02:03.504495 (+209926us) log.cc:844] Serialized 6745 byte log entry
0709 13:02:04.630812 (+1126317us) op_driver.cc:464] REPLICATION: finished
0709 13:02:05.459148 (+828336us) write_op.cc:301] APPLY: starting
0709 13:02:05.459250 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:05.459655 (+   405us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:05.459785 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:06.407680 (+947895us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:06.408179 (+   499us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:02:06.408244 (+    65us) write_op.cc:312] APPLY: finished
0709 13:02:06.408736 (+   492us) log.cc:844] Serialized 119 byte log entry
0709 13:02:08.075168 (+1666432us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:08.105891 (+ 30723us) write_op.cc:454] Released schema lock
0709 13:02:08.174821 (+ 68930us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:08.221738 (+ 46917us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:08.239887 (+ 18149us) write_op.cc:454] Released schema lock
0709 13:02:09.032231 (+792344us) spinlock_profiling.cc:243] Waited 8.83 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
Metrics: {"mutex_wait_us":229821,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":789073,"apply.run_cpu_time_us":36598,"apply.run_wall_time_us":2746522,"num_ops":17,"prepare.queue_time_us":194250,"prepare.run_cpu_time_us":18481,"prepare.run_wall_time_us":684128,"raft.queue_time_us":693872,"raft.run_cpu_time_us":67575,"raft.run_wall_time_us":829140,"replication_time_us":1319913,"spinlock_wait_cycles":20297600,"thread_start_us":2642034,"threads_started":5,"wal-append.queue_time_us":1806735,"wal-append.run_cpu_time_us":35709,"wal-append.run_wall_time_us":1242231}]]}
I20260709 13:02:10.022742  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:09.966853  3855 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:10.019523  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:09.968127  1464 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 2 to 6, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:10.034584  2305 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.003862  4198 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:10.045630  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.044128  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.037693  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.047309  1709 raft_consensus.cc:2376] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6. Re-sending same reply.
I20260709 13:02:10.038347  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:10.039276  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:10.039352  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 81385.6us
I20260709 13:02:10.048990  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.049320  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:10.042954  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.050235  1564 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:10.049625  1713 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
W20260709 13:02:10.052958  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 235792us
I20260709 13:02:10.053321  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.054055  1568 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:10.027151  3850 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:10.026966  4066 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:10.055160  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.058667  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.056831  2301 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:10.060911  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.043826  1643 rpcz_store.cc:276] 0709 13:02:08.738720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.672329 (+933609us) service_pool.cc:224] Handling call
0709 13:02:09.683390 (+ 11061us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:09.762250 (+ 78860us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.762315 (+    65us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:09.782245 (+ 19930us) log.cc:844] Serialized 52 byte log entry
0709 13:02:10.003952 (+221707us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:10.003990 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.004051 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:10.043254 (+ 39203us) raft_consensus.cc:1685] finished
0709 13:02:10.043274 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:10.043400 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62730,"raft.queue_time_us":66802,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":7626,"thread_start_us":202806,"threads_started":1,"wal-append.queue_time_us":236208}
I20260709 13:02:10.063076  1710 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:10.062999  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.063848  1714 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:10.071054  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:10.071313  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.072110  2297 raft_consensus.cc:2410] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:10.063040  2296 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:10.052371  4214 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 13:02:10.071988  4189 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:10.074563  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.024106  2298 raft_consensus.cc:2376] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6. Re-sending same reply.
I20260709 13:02:10.045265  3855 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:10.060382  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:10.058969  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:10.079880  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.082655  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.085806  1715 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:10.087618  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.087734  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.088318  1716 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:10.089138  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 279479us
I20260709 13:02:10.057462  3850 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:10.059773  4066 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:10.091305  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.092492  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1839) took 1044 ms. Trace:
I20260709 13:02:10.091358  1636 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 13:02:10.088783  2268 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.819s (SENT) [suppressed 46 similar messages]
I20260709 13:02:10.093569  1564 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:10.093860  1683 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:10.094178  1712 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:10.093493  1565 rpcz_store.cc:276] 0709 13:02:09.042466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.581914 (+539448us) service_pool.cc:224] Handling call
0709 13:02:09.582386 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:09.605873 (+ 23487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.605908 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:09.621036 (+ 15128us) log.cc:844] Serialized 26 byte log entry
0709 13:02:09.836797 (+215761us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:09.836859 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.836960 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:10.086820 (+249860us) raft_consensus.cc:1685] finished
0709 13:02:10.086837 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:10.086983 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9030,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":19050,"spinlock_wait_cycles":9600,"thread_start_us":173157,"threads_started":1,"wal-append.queue_time_us":200483}
I20260709 13:02:10.086534  3876 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:10.083570  1464 catalog_manager.cc:5697] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:10.072992  4198 raft_consensus.cc:697] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:10.097502  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:10.098356  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:10.099148  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 76525.3us
I20260709 13:02:10.102154  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:10.099654  1563 raft_consensus.cc:2393] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:10.100589  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.044494  3743 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:10.085068  3855 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.078009  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:10.104336  1714 raft_consensus.cc:3055] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:10.111477  1714 raft_consensus.cc:740] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:10.110245  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.115617  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.116580  1637 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:10.116641  1715 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:10.117389  3876 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:10.117859  3876 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:10.118251  3876 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:10.119499  4211 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:02:10.120258  4128 raft_consensus.cc:1064] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:02:10.123050  1562 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:10.123216  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.082672  4204 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.289s
I20260709 13:02:10.124378  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.125207  4021 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:10.125447  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.126097  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:10.126485  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.126871  2298 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:10.127293  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.127616  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:10.128307  1635 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:10.127293  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:10.129093  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.129799  2296 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 5: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:10.130102  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.131160  2297 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:10.130724  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.133483  1642 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:10.098544  4144 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.135679  4144 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:10.135470  1714 consensus_queue.cc:260] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.136650  1714 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.136438  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.142858  1717 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:10.108120  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:10.145197  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 84885.1us
I20260709 13:02:10.107856  4045 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:10.148772  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.149530  1637 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:10.151135  4099 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.151538  2305 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:10.152070  4099 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.153426  1638 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:02:10.155314  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 49343.1us
I20260709 13:02:10.157325  1534 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:10.158684  3538 raft_consensus.cc:2764] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:02:10.158788  4045 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:10.159883  3538 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:10.161082  3538 raft_consensus.cc:697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:10.138243  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:10.176918  2302 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
W20260709 13:02:10.177942  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 88137.7us
W20260709 13:02:10.182789  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.183612  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.183995  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.184623  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.186740  1681 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:10.188730  2305 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:10.190059  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.136891  4198 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:10.191277  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 53111.5us
I20260709 13:02:10.199882  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.163023  2297 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:10.083079  4192 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.162863  3538 consensus_queue.cc:237] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.203303  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:10.203474  4192 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:10.205363  4207 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:10.205852  4207 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:10.206281  4207 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.201399  4099 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:10.190553  1609 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:10.213007  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.208076  1716 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
W20260709 13:02:10.208225  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 95473.4us
I20260709 13:02:10.162878  4187 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:10.199566  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.225455  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:02:10.221932  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.227476  1711 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:02:10.228943  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.229514  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.230808  2269 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:10.234717  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.235522  1713 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:10.198575  4115 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.237917  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.168426  2299 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:10.231164  1464 catalog_manager.cc:5697] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: HEALTHY } } }
I20260709 13:02:10.241583  1714 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:02:10.228816  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.242362  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.243155  1717 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.248570  1714 pending_rounds.cc:85] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 4
I20260709 13:02:10.243438  1718 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:10.243818  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.104161  3508 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:10.158812  3855 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.169710  3850 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:10.250890  3850 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:10.221423  4189 raft_consensus.cc:2764] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:02:10.168603  4149 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:02:10.251566  3850 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:10.251863  3945 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:10.251016  3855 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:10.252281  3850 consensus_queue.cc:1243] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:10.252794  3855 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:10.252319  4149 raft_consensus.cc:3055] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:10.253753  1640 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:10.253818  4149 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:10.254846  4149 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:10.245757  4094 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:10.259253  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.261059  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.207310  4118 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.278380  1532 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:10.266471  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.289534  1717 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:10.320169  1715 raft_consensus.cc:2393] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:10.271102  1710 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:10.262442  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:10.275967  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.275995  1683 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:10.248991  1714 pending_rounds.cc:107] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:02:10.122807  4128 consensus_queue.cc:237] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.321188  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.327723  1713 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 5: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:10.266417  1681 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:10.323741  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.263680  4115 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:10.325773  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.265064  4206 consensus_queue.cc:1059] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:02:10.344869  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.272251  3743 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:10.351184  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.260103  4094 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.354105  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.367869  4200 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:10.368218  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.368327  4192 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:10.368975  4192 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:10.369447  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.270243  2300 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:10.370055  4192 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.324823  1714 raft_consensus.cc:2889] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:10.330154  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.248359  3896 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 13:02:10.361022  2304 raft_consensus.cc:2368] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6.
I20260709 13:02:10.346386  4149 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:10.361387  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:10.362087  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 95766.9us
I20260709 13:02:10.265486  2298 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:10.265290  4199 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:10.363449  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.364498  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.268469  3508 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:10.368040  4045 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:10.365484  1713 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:02:10.381304  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 103008us
I20260709 13:02:10.381976  1532 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:10.265064  4217 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:10.370680  1709 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:10.359833  1715 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:10.372027  1717 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
W20260709 13:02:10.373435  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 121890us
W20260709 13:02:10.375913  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.376925  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 128899us
I20260709 13:02:10.260048  4019 consensus_queue.cc:1059] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.116s
I20260709 13:02:10.379652  1710 raft_consensus.cc:2393] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:10.397008  1533 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:10.320874  3865 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:02:10.323463  4041 consensus_queue.cc:1059] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:02:10.264401  1464 catalog_manager.cc:5697] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:10.397578  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.398567  1717 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:10.338492  3824 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:02:10.371731  4200 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.371755  4094 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:10.405300  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.374610  4144 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:10.353776  3945 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 13:02:10.405378  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 129468us
I20260709 13:02:10.380955  4199 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:10.452391  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 70465.5us
I20260709 13:02:10.465849  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:10.466132  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 69186.4us
I20260709 13:02:10.467101  1711 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:10.467870  1640 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:10.468660  1711 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:02:10.468997  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.469559  4124 raft_consensus.cc:3055] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:10.469687  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:10.470091  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.470934  1714 raft_consensus.cc:2410] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:02:10.471483  4147 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:10.470082  4124 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:10.473330  4124 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.474574  4124 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.265185  4208 consensus_queue.cc:1059] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.181s
I20260709 13:02:10.475556  4198 raft_consensus.cc:3055] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:10.475970  4198 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:10.478066  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.479429  1717 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:02:10.475572  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.482589  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:10.486874  3508 raft_consensus.cc:3055] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:10.487582  3508 raft_consensus.cc:740] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:10.453328  4199 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.488641  4198 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.488742  3508 consensus_queue.cc:260] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.490216  4198 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.492602  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:10.488794  1681 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:10.494385  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 89710.1us
I20260709 13:02:10.494648  1709 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:10.408085  4200 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:10.491066  4048 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:10.497907  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 44306.8us
W20260709 13:02:10.504861  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 314389us
I20260709 13:02:10.504901  1532 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:10.491214  3508 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:10.395550  4021 consensus_queue.cc:1059] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.098s
I20260709 13:02:10.502554  4045 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:10.521718  4045 consensus_queue.cc:1059] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:10.522079  4060 raft_consensus.cc:2764] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:02:10.522563  4045 consensus_queue.cc:1243] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:10.523208  4045 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:10.523608  4045 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:10.523767  1681 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:10.524109  4045 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:10.527129  4212 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:10.527902  4212 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:10.528594  4212 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:02:10.530314  4231 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.132s	user 0.000s	sys 0.006s
I20260709 13:02:10.531505  4220 raft_consensus.cc:3055] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:10.531899  4144 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:10.531985  4220 raft_consensus.cc:740] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:10.532524  4144 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.533032  4144 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:10.533558  4013 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:10.533227  4220 consensus_queue.cc:260] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:10.535013  4208 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:02:10.569681  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 87189.8us
I20260709 13:02:10.547534  1464 catalog_manager.cc:5697] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 2 to 4, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:10.506691  4066 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:10.495311  3538 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:10.573099  1644 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:10.586818  1638 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:02:10.587395  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 82945.3us
I20260709 13:02:10.604589  1609 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:10.570601  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:10.581516  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.586568  4066 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:10.602517  4220 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:10.586993  4175 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:10.623241  4175 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:10.624370  4175 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.624677  4144 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.625842  4175 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.580256  4118 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:10.627542  4019 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.628109  4019 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:10.630290  4019 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:10.584040  4226 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 13:02:10.584442  1464 catalog_manager.cc:5697] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 3 to 4, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:10.616869  4228 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:02:10.635870  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.637001  1464 catalog_manager.cc:5697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:10.574239  4115 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:10.645134  1718 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.574862  4048 raft_consensus.cc:3055] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:10.665381  1709 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:10.644155  1464 catalog_manager.cc:5697] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 2 to 3, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:10.648257  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.651293  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.667382  1717 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:10.630896  4019 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.590957  4217 raft_consensus.cc:697] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:10.674861  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 104347us
I20260709 13:02:10.677091  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.677822  1642 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
W20260709 13:02:10.680346  4232 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:02:10.583380  3876 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:10.687842  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 195371us
W20260709 13:02:10.681561  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.664692  4048 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:10.607797  3855 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:02:10.683964  1464 catalog_manager.cc:5697] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 3 to 5, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:10.576443  4207 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:10.589483  4094 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.664197  4204 consensus_queue.cc:1059] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
W20260709 13:02:10.680821  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 76369.6us
W20260709 13:02:10.680794  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 65286.7us
W20260709 13:02:10.689947  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 474378us
I20260709 13:02:10.691787  1682 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:10.632330  4045 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.689740  4013 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:10.707907  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 203144us
I20260709 13:02:10.692675  4124 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:10.711230  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:10.694120  4144 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:10.714102  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.714572  1532 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:10.714960  1644 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:02:10.722054  2304 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:10.722816  2297 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:10.724341  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:10.725090  1644 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:10.692257  4149 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:02:10.700284  4189 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:10.684089  4019 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.730044  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:10.733807  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.738325  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:10.738896  1532 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:10.724007  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:10.744247  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.709987  4208 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:02:10.746156  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.707629  4013 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:10.729605  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:10.788905  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 47644.8us
I20260709 13:02:10.746821  1683 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:10.735038  4115 raft_consensus.cc:3055] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:10.733434  3508 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:10.766996  4013 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:10.740788  3855 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:02:10.769304  4041 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:10.771903  4019 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:10.772420  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.775251  4217 consensus_queue.cc:237] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:10.776731  2300 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:10.776913  4045 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:10.747931  4144 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:02:10.802747  4115 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:10.777496  1532 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:02:10.835095  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 57687.7us
W20260709 13:02:10.835757  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 45833.2us
I20260709 13:02:10.836175  1532 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:10.748143  4237 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:10.780073  4207 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:10.780589  3995 consensus_meta.cc:287] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.814s	user 0.007s	sys 0.008s
I20260709 13:02:10.782884  3743 raft_consensus.cc:3055] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:10.767083  4048 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:10.742043  4099 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:10.786360  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:10.810774  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 64041.6us
I20260709 13:02:10.811578  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:10.856771  1641 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:10.857201  1635 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:10.814860  1644 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:10.859402  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:10.813578  1718 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:10.774717  4094 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:10.744796  4189 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.777398  3876 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:10.807750  3508 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.845606  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.847008  1715 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:10.821498  4204 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:10.846864  3743 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:10.851879  4124 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:10.863257  1534 leader_election.cc:336] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.941s (SENT)
I20260709 13:02:10.864154  1643 raft_consensus.cc:2410] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:10.868584  1714 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:02:10.843017  4041 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:10.843787  4118 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.884990  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:10.847594  4045 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:10.827164  4199 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:10.890529  1709 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:02:10.892906  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 56609.1us
I20260709 13:02:10.900441  1534 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:10.901394  1718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
W20260709 13:02:10.909025  1515 auto_rebalancer.cc:893] Removing replace marker failed: Replica 28bc59d37f664ce0a6f5784f2619f0b5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:02:10.909569  1515 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:02:10.909880  1515 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:02:10.918962  1710 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:10.919642  1714 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:10.933352  1637 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:10.927749  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:10.919621  1718 raft_consensus.cc:606] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:02:10.890663  3508 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:10.934801  1718 raft_consensus.cc:614] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:10.921016  1712 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:10.939043  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:10.938001  4124 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.920408  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:10.919114  4254 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:10.941082  4237 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:10.942212  4249 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
W20260709 13:02:10.935338  4223 log.cc:927] Time spent T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.434s	user 0.005s	sys 0.000s
I20260709 13:02:10.953758  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:10.958943  4189 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:10.959096  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:10.956250  4240 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:02:10.907500  4118 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:10.962795  4200 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:10.965207  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:11.027549  1644 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:10.966612  1635 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:10.973904  4206 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:10.987812  2301 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:02:10.999740  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 79432.4us
I20260709 13:02:10.890540  4048 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:10.907114  4115 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:11.001602  4187 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:11.003273  4149 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:11.032114  1533 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:11.032612  1532 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:11.033046  4060 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.990760  4013 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.024981  4254 raft_consensus.cc:697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:11.023674  2300 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:11.050532  1532 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:11.025213  4220 consensus_queue.cc:1059] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:02:11.055013  1643 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:11.033528  4206 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:11.058887  4254 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.065750  4200 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:11.052831  4060 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.067812  4045 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:11.088961  1717 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:11.087929  1715 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:11.068794  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:11.069952  1644 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:11.057037  4233 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:02:11.084108  4255 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:11.084609  4236 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:11.052366  4041 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:11.075119  4115 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:11.052529  4149 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:11.062587  4118 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:11.051517  4021 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
W20260709 13:02:11.137943  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 210320us
I20260709 13:02:11.145224  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:11.145993  2300 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:11.153218  1638 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:02:11.158526  2301 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:02:11.156818  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 293611us
I20260709 13:02:11.147081  4199 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:11.171602  4149 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.146821  4060 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.153582  4013 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.191460  3538 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:11.206314  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 63033.6us
W20260709 13:02:11.204559  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 120799us
I20260709 13:02:11.207508  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:11.200387  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.156555  4144 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:11.199045  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 92331.8us
I20260709 13:02:11.221309  1715 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:02:11.224417  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.229732  1683 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:11.232043  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:11.257459  1641 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:11.255605  1683 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:11.212448  4099 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:11.151986  4041 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.213327  4060 consensus_queue.cc:1059] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:02:11.225011  4144 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:11.237218  4175 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.611s	user 0.017s	sys 0.000s
I20260709 13:02:11.240980  4189 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:11.344513  4187 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.345069  4187 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:11.347868  4270 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:11.348512  4270 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:11.317040  4182 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.355880  4182 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.356374  4060 raft_consensus.cc:3055] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:11.356959  4060 raft_consensus.cc:740] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:11.357163  4144 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.358289  4144 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.357952  4060 consensus_queue.cc:260] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:11.361864  4060 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:11.251674  4233 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.251737  4259 consensus_queue.cc:1059] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.103s
I20260709 13:02:11.363981  4199 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:11.364681  4199 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.365662  4048 raft_consensus.cc:3055] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:11.300740  4013 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:11.306862  4261 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:11.292443  4236 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.367650  4261 raft_consensus.cc:697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:11.368270  4182 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.368099  4236 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.368620  4261 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.355257  4021 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:11.371217  4021 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:11.371680  4021 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:11.371845  4263 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 13:02:11.363332  4233 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.317453  3896 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:02:11.366962  4013 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:11.293462  4226 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:02:11.384024  1534 leader_election.cc:336] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 2.205s (SENT)
I20260709 13:02:11.412495  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:11.387974  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 576039us
W20260709 13:02:11.413972  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.366055  4048 raft_consensus.cc:740] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:11.414777  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:11.350711  4270 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:11.421207  2296 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:11.423134  1635 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:11.362406  3508 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:11.418156  4237 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:02:11.432893  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.413044  4236 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.412634  4013 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:11.435227  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 179763us
W20260709 13:02:11.439786  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 55806us
I20260709 13:02:11.446758  1640 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:02:11.447242  1640 pending_rounds.cc:85] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 2
I20260709 13:02:11.447494  1640 pending_rounds.cc:107] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:02:11.447734  1640 raft_consensus.cc:2889] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:11.425441  4259 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:11.358829  4144 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.456316  4182 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:11.457026  4187 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.460527  4189 raft_consensus.cc:1064] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:02:11.464722  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:11.465462  2302 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 2.
I20260709 13:02:11.465397  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:11.467171  4192 consensus_queue.cc:1059] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:11.468405  4212 raft_consensus.cc:3055] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:11.468815  4212 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:11.468969  2298 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:11.469754  4212 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:11.471109  4212 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:11.476740  1683 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:11.477887  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:11.480448  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.481382  1640 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:11.485754  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:11.486526  2298 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:02:11.486939  3743 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:11.505831  4187 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.515475  4187 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:11.517159  3743 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:11.530738  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:11.531746  1564 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:11.533921  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:11.534646  1569 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:11.540539  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:11.541448  1565 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:11.452709  4048 consensus_queue.cc:260] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.524767  4245 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.553056  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:11.592018  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:11.566509  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 260855us
I20260709 13:02:11.600353  1566 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:11.525682  4267 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:02:11.601063  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.601430  1564 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
W20260709 13:02:11.611358  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 133555us
I20260709 13:02:11.611539  1563 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:11.613140  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:11.614099  1564 raft_consensus.cc:2410] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:11.615216  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:11.615545  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:11.616789  4245 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.617233  3896 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.623042  4013 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.620822  4282 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:02:11.631881  3896 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.620318  4258 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:11.639513  4258 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.639878  4258 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:11.640193  3896 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.636096  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:11.589005  4128 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.641471  1562 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:11.641620  4128 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:11.634074  4013 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.630853  4245 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.653015  1566 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:11.528483  4280 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.089s
I20260709 13:02:11.655613  1565 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:11.665465  4013 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.666953  4128 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:11.666435  4013 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.669579  4013 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.670806  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:11.678860  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:11.678555  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:11.687235  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:11.688074  1563 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:02:11.688619  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.688774  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:11.689370  1714 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:11.614511  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:11.676438  4279 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:02:11.671829  4268 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.345s	user 0.001s	sys 0.000s
W20260709 13:02:11.694235  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 209477us
I20260709 13:02:11.700821  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:11.701743  1640 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:11.701040  1636 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:11.721501  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:11.687067  4282 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:11.722846  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:11.696806  3508 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:11.695137  4128 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.735229  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:11.704190  4217 raft_consensus.cc:3055] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:11.754690  4217 raft_consensus.cc:740] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:11.755129  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:02:11.755815  1709 raft_consensus.cc:606] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:02:11.712621  4283 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:02:11.756822  4290 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:11.756141  4217 consensus_queue.cc:260] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:11.757414  4290 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.759868  4290 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.733772  4282 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.761762  4258 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.762473  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:11.763026  4282 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.763079  4144 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.736176  1563 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:11.756168  1709 raft_consensus.cc:614] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:11.741089  1560 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:02:11.742614  1567 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:11.753495  1562 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:11.767302  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:11.770818  1565 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:11.770879  1641 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:11.773471  4128 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.773782  4021 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.776427  4128 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.770305  4280 consensus_queue.cc:1059] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:02:11.778097  4245 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:11.778846  4245 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:11.779292  4198 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:11.779876  4199 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:11.781162  4245 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:11.781929  1560 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:02:11.782866  4206 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:11.784246  4206 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:11.785228  3947 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:11.786708  1640 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:11.778151  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:11.826422  4296 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:11.839200  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.842037  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 63969.5us
I20260709 13:02:12.051640  4245 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.450903  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:12.451498  4245 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:12.452867  4128 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.453469  4128 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:12.129370  4021 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:12.399127  2304 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.002s	sys 0.011s
W20260709 13:02:12.455427  2304 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.003s	sys 0.012s
I20260709 13:02:12.027506  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:12.456071  4175 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:12.456650  4175 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:12.047807  2269 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:12.393296  1563 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.005s	sys 0.012s
W20260709 13:02:12.458631  1563 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.005s	sys 0.013s
W20260709 13:02:12.025484  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.951973  4271 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.575s	user 0.004s	sys 0.000s
W20260709 13:02:12.460934  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.963495  4252 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.475476  4252 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:12.477851  4252 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:12.461867  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:12.478885  2302 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEARNER]: Advancing to term 6
I20260709 13:02:12.479557  4124 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:12.453814  4245 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:12.457010  2300 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:12.449512  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 1.49589e+06us
I20260709 13:02:12.510212  2268 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:12.511693  2302 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEARNER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
W20260709 13:02:12.465842  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 418160us
I20260709 13:02:12.466873  4275 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:12.096549  4299 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.072s	user 0.001s	sys 0.001s
I20260709 13:02:12.462568  4258 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:12.471841  4128 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:12.519724  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:12.520804  2300 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:12.522789  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:12.523778  2300 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:12.524175  4275 raft_consensus.cc:697] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:12.525300  4275 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:12.526726  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:12.527455  2297 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:12.517309  4206 consensus_meta.cc:287] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.733s	user 0.014s	sys 0.000s
I20260709 13:02:12.531540  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 829) took 1054 ms. Trace:
I20260709 13:02:12.540925  3963 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:12.540349  2301 rpcz_store.cc:276] 0709 13:02:11.476738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.477260 (+   522us) service_pool.cc:224] Handling call
0709 13:02:11.477573 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.963094 (+485521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.963138 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.963742 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.270168 (+306426us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:12.270227 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.270328 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.531304 (+260976us) raft_consensus.cc:1685] finished
0709 13:02:12.531320 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.531469 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":483595,"raft.queue_time_us":483911,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":305905,"threads_started":1,"wal-append.queue_time_us":558696}
I20260709 13:02:12.534663  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 639) took 1318 ms. Trace:
I20260709 13:02:12.535503  4233 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:12.541574  3963 raft_consensus.cc:697] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:12.536782  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:12.544401  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:12.543152  2269 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:12.538250  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:12.548879  1644 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:02:12.549396  1644 pending_rounds.cc:85] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 4
I20260709 13:02:12.549675  1644 pending_rounds.cc:107] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:02:12.549885  1644 raft_consensus.cc:2889] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:12.538817  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:12.539254  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 637) took 1631 ms. Trace:
I20260709 13:02:12.544503  4220 raft_consensus.cc:3055] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:12.551898  1564 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:12.545341  2301 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:12.598559  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:12.603935  1561 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:12.604943  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2206) took 1060 ms. Trace:
I20260709 13:02:12.605356  1561 rpcz_store.cc:276] 0709 13:02:11.544461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.545571 (+  1110us) service_pool.cc:224] Handling call
0709 13:02:12.603294 (+1057723us) spinlock_profiling.cc:243] Waited 7.91 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:02:12.603564 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.604566 (+  1002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.604832 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18203008}
I20260709 13:02:12.607372  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:12.608310  2303 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:12.599802  2305 raft_consensus.cc:2393] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:12.544401  3963 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:12.643718  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:12.645219  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:12.645872  1564 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:12.646862  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:12.665256  1564 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:12.551602  2304 rpcz_store.cc:276] 0709 13:02:10.908282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.908475 (+   193us) service_pool.cc:224] Handling call
0709 13:02:10.908853 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.785164 (+876311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.785196 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.785749 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.455906 (+670157us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:12.455944 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.457679 (+  1735us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.539040 (+ 81361us) raft_consensus.cc:1685] finished
0709 13:02:12.539063 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.539196 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":862318,"raft.queue_time_us":876109,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":669761,"threads_started":1,"wal-append.queue_time_us":749356}
I20260709 13:02:12.536854  4283 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:02:12.685691  4271 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
W20260709 13:02:12.704479  1637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2192) took 2341 ms (client timeout 1952 ms). Trace:
W20260709 13:02:12.708079  1637 rpcz_store.cc:269] 0709 13:02:10.362969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.363121 (+   152us) service_pool.cc:224] Handling call
0709 13:02:12.704195 (+2341074us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b540032bed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:12.704379 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1272407296}
I20260709 13:02:12.592368  4220 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:12.712085  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 175396us
I20260709 13:02:12.712970  2268 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:12.713696  1564 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:12.717530  2268 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:12.719506  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1669) took 1041 ms. Trace:
I20260709 13:02:12.707053  4283 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:12.719900  1710 rpcz_store.cc:276] 0709 13:02:11.677954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.686983 (+  9029us) service_pool.cc:224] Handling call
0709 13:02:11.687284 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.687668 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.687700 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.704435 (+ 16735us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.790209 (+ 85774us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:11.790271 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.790374 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.719309 (+928935us) raft_consensus.cc:1685] finished
0709 13:02:12.719329 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.719450 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20728,"raft.queue_time_us":3854,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":10829,"thread_start_us":61544,"threads_started":1,"wal-append.queue_time_us":785293}
I20260709 13:02:12.727027  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:12.730549  1564 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:12.731781  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:12.735628  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2212) took 1175 ms. Trace:
I20260709 13:02:12.736090  1569 rpcz_store.cc:276] 0709 13:02:11.560272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.623572 (+ 63300us) service_pool.cc:224] Handling call
0709 13:02:12.731626 (+1108054us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:02:12.735568 (+  3942us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28344192}
I20260709 13:02:12.714398  4275 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:12.739392  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:12.741297  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 290512us
I20260709 13:02:12.743188  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:12.744110  4305 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:02:12.744316  4206 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:12.748935  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:12.750164  1564 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:12.751148  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:12.751636  1564 raft_consensus.cc:2376] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4. Re-sending same reply.
I20260709 13:02:12.752673  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:12.735276  4118 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:12.753558  1564 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:12.754979  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:12.755543  1564 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:12.756587  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:12.756939  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:12.739916  4275 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:12.755103  4296 consensus_meta.cc:287] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.906s	user 0.007s	sys 0.006s
I20260709 13:02:12.763763  1564 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:12.759987  4275 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:12.763830  4233 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:12.766222  4301 log.cc:927] Time spent T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:12.769066  1564 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:02:12.763410  4296 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:12.770501  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:12.766680  4118 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:12.750909  4220 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:12.779208  4220 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:12.780064  2296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 633) took 2074 ms (client timeout 1877 ms). Trace:
W20260709 13:02:12.780490  2296 rpcz_store.cc:269] 0709 13:02:10.705877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.707210 (+  1333us) service_pool.cc:224] Handling call
0709 13:02:12.779824 (+2072614us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b54006414d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:12.779956 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1243120512}
I20260709 13:02:12.785920  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2209) took 1237 ms. Trace:
I20260709 13:02:12.791522  4309 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:12.791859  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:12.792233  4309 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:12.792680  4309 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:12.793715  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:12.786429  1565 rpcz_store.cc:276] 0709 13:02:11.550480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.657030 (+106550us) service_pool.cc:224] Handling call
0709 13:02:11.770603 (+113573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.771507 (+   904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.785674 (+1014167us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54002aab50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:12.785846 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260198784}
I20260709 13:02:12.841455  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:12.843050  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:12.853116  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 310040us
I20260709 13:02:12.860569  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:12.861500  1642 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:12.865264  4317 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.866032  4317 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:12.869202  2269 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:12.872076  4118 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:12.872509  4118 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.872861  4118 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:12.872903  4317 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:12.876425  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:12.877347  1635 raft_consensus.cc:2410] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:12.883421  4118 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:12.886891  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:12.887789  1635 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:12.899310  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 181861us
I20260709 13:02:12.900620  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:12.903910  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 435141us
I20260709 13:02:12.905186  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2215) took 1315 ms. Trace:
I20260709 13:02:12.906689  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:12.910517  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:12.843873  4245 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:12.897879  4233 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:12.916688  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:12.917564  1638 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:12.907442  1563 rpcz_store.cc:276] 0709 13:02:11.590128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.764500 (+174372us) service_pool.cc:224] Handling call
0709 13:02:11.783942 (+ 19442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.827750 (+ 43808us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.827785 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.841653 (+ 13868us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.459137 (+617484us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:12.903462 (+444325us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.904766 (+  1304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.904862 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.904917 (+    55us) raft_consensus.cc:1685] finished
0709 13:02:12.904935 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.905116 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43456,"prepare.queue_time_us":397555,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":51,"raft.queue_time_us":44259,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":8823,"thread_start_us":1045283,"threads_started":2,"wal-append.queue_time_us":673460}
W20260709 13:02:12.921630  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 412948us
I20260709 13:02:12.923594  2268 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:12.921898  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:12.923619  1638 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:12.902395  2297 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:12.542438  2299 rpcz_store.cc:276] 0709 13:02:11.215935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.223995 (+  8060us) service_pool.cc:224] Handling call
0709 13:02:11.311714 (+ 87719us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.827191 (+515477us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:11.827642 (+   451us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.229192 (+401550us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:12.403910 (+174718us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.457119 (+ 53209us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:12.457159 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.457236 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.531274 (+ 74038us) raft_consensus.cc:1685] finished
0709 13:02:12.534428 (+  3154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.534596 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":477154,"prepare.queue_time_us":1331,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":1760,"raft.queue_time_us":557151,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":84686,"spinlock_wait_cycles":12360960,"thread_start_us":356906,"threads_started":1,"wal-append.queue_time_us":683482}
I20260709 13:02:12.910393  4296 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:12.925269  1643 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:12.912442  4314 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:12.928023  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 85104.6us
I20260709 13:02:12.926209  4256 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.948894  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:12.886940  4118 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:12.927634  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2214) took 1163 ms. Trace:
I20260709 13:02:12.909108  4320 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:12.950806  4233 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:12.952032  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:12.952353  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2222) took 1259 ms. Trace:
I20260709 13:02:12.927619  3963 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:12.957540  1635 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:12.958796  1636 rpcz_store.cc:276] 0709 13:02:11.761310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.762131 (+   821us) service_pool.cc:224] Handling call
0709 13:02:11.762482 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.762874 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.762903 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.763401 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.229696 (+466295us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:12.229751 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.229849 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.924364 (+694515us) raft_consensus.cc:1685] finished
0709 13:02:12.924377 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.924474 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3861,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":3664,"thread_start_us":465826,"threads_started":1,"wal-append.queue_time_us":768729}
I20260709 13:02:12.960376  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 1898) took 2238 ms. Trace:
I20260709 13:02:12.942545  4228 raft_consensus.cc:2764] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:02:12.979331  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.983624  2269 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:12.982746  1717 rpcz_store.cc:276] 0709 13:02:10.721617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.721861 (+   244us) service_pool.cc:224] Handling call
0709 13:02:10.722152 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.089437 (+367285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.960174 (+1870737us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b54003db3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:12.960321 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":843139328}
I20260709 13:02:12.986755  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:12.989831  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2224) took 1253 ms. Trace:
I20260709 13:02:12.990346  1569 rpcz_store.cc:276] 0709 13:02:11.736892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.759297 (+1022405us) service_pool.cc:224] Handling call
0709 13:02:12.765169 (+  5872us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.765319 (+   150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.765333 (+    14us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:12.767875 (+  2542us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.961097 (+193222us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:12.961144 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.961220 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.989572 (+ 28352us) raft_consensus.cc:1685] finished
0709 13:02:12.989594 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.989744 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":191713,"threads_started":1,"wal-append.queue_time_us":219207}
I20260709 13:02:12.961642  4320 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:13.005971  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 54035.4us
I20260709 13:02:13.006181  4328 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:13.006994  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:13.008611  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:13.005220  3963 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:13.004647  4320 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:13.037384  2268 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:13.002357  4118 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:13.034464  4314 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:13.040889  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:13.043677  4048 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:13.044512  4206 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:13.045266  4206 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:13.045439  1642 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:02:12.961057  4256 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:13.048303  4256 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:13.051007  3963 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:12.986100  4317 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:02:13.031117  4233 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:13.077394  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:13.078886  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1665) took 1389 ms. Trace:
I20260709 13:02:13.079581  1712 rpcz_store.cc:276] 0709 13:02:11.676757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.679095 (+  2338us) service_pool.cc:224] Handling call
0709 13:02:11.679388 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.679609 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:11.679922 (+   313us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.767548 (+ 87626us) log.cc:844] Serialized 14 byte log entry
0709 13:02:13.064944 (+1297396us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b18001d1e20. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb2ec0a 00007fd7a826e219 00007fd79ebb6a40 00007fd79ebb14e4 00007fd79ebb1455 00007fd7a1c99060 00007fd7a1cab716 00007fd7a1cbc287 00007fd7a1cbc208
0709 13:02:13.065164 (+   220us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.065239 (+    75us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:13.065336 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.065430 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.065599 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35594,"prepare.queue_time_us":3023,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":2755,"raft.queue_time_us":1333710,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":6145,"spinlock_wait_cycles":161557120,"thread_start_us":54500,"threads_started":1,"wal-append.queue_time_us":188814}
I20260709 13:02:13.085516  4252 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:13.086155  4252 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:13.086629  4252 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:13.088145  4175 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:13.089840  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:13.103477  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:13.104714  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2205) took 1562 ms (client timeout 1744 ms). Trace:
W20260709 13:02:13.105048  1568 rpcz_store.cc:269] 0709 13:02:11.542740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.543973 (+  1233us) service_pool.cc:224] Handling call
0709 13:02:13.103259 (+1559286us) spinlock_profiling.cc:243] Waited 8.05 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:02:13.104634 (+  1375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18517376}
I20260709 13:02:13.083621  4317 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:02:13.108076  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.109614  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:13.110749  4094 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:13.112732  4290 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.113251  4290 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:13.114269  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.114735  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.115087  4060 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.115338  4182 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.115599  4060 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.115815  4182 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.117622  4182 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.118124  4182 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.119192  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.119360  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.119606  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.119972  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.120258  1682 connection.cc:256] RPC call timeout handler was delayed by -1.0146322190000205s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1952s for a response
I20260709 13:02:13.120393  4267 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.121156  4267 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:13.122242  4291 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:13.122797  4291 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.124754  4291 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:13.120304  1681 connection.cc:256] RPC call timeout handler was delayed by -1.0793310330000168s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1673s for a response
I20260709 13:02:13.125118  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:13.125492  1681 connection.cc:256] RPC call timeout handler was delayed by -1.073931033000008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
W20260709 13:02:13.126075  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.125955  2297 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:02:13.126523  1681 connection.cc:256] RPC call timeout handler was delayed by -1.0154310330000271s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
W20260709 13:02:13.126875  1681 connection.cc:256] RPC call timeout handler was delayed by -1.0109310330000199s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
I20260709 13:02:13.127403  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:13.112663  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:13.136497  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:13.137466  1561 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
W20260709 13:02:13.142786  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 134234us
I20260709 13:02:13.143671  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:13.143999  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:13.055262  4118 raft_consensus.cc:697] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:13.146512  1532 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:02:13.147373  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 235599us
I20260709 13:02:13.063213  4233 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:13.153828  1609 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:13.155267  1609 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:13.155854  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:13.054817  3963 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:13.118546  4182 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.160470  1609 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:13.164858  2302 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:13.159386  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:13.117640  4060 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.146849  4256 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.146436  4094 raft_consensus.cc:2764] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:02:13.146287  4220 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.153885  3508 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:13.158744  4206 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:13.155333  4292 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.155097  4233 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:13.186062  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.157984  4280 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.200304  4267 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:13.166776  3963 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:13.199712  4330 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:13.229615  4330 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:13.230257  4330 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.213529  4060 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:13.230775  4330 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.232599  4328 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:13.233263  4328 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.233628  4330 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.202035  3508 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:13.183787  4290 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.249099  1681 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:13.169858  4115 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.176898  4314 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:13.179111  4331 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.215461  1683 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:13.217482  3850 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:13.198592  4204 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.200968  4206 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.221500  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:13.221503  4118 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:13.225814  4311 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.470s	user 0.000s	sys 0.001s
I20260709 13:02:13.226514  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:13.259312  1642 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:13.236850  4330 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:13.226797  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:13.189472  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:13.237916  1682 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:13.237576  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:13.242224  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.243508  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.245585  1714 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:13.246325  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:13.247222  1709 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:13.247309  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:13.261722  1566 raft_consensus.cc:2410] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:02:13.171365  4315 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:13.233685  4328 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:13.250250  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 89890.6us
I20260709 13:02:13.214428  4236 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.253221  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:13.254340  3508 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:13.226950  4124 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:13.258467  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:13.258842  1637 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:13.260614  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:13.309564  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 49037.2us
I20260709 13:02:13.312363  3508 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:13.313711  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:13.314280  2302 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:13.214210  4292 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:13.188812  4220 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:13.184041  4256 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.292445  1710 raft_consensus.cc:2393] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:02:13.273653  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 58256.8us
I20260709 13:02:13.326701  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:13.248612  4192 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.294153  1716 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:12.964435  1560 rpcz_store.cc:276] 0709 13:02:11.692684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.782894 (+ 90210us) service_pool.cc:224] Handling call
0709 13:02:11.783335 (+   441us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:11.825076 (+ 41741us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:11.825111 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:11.825798 (+   687us) log.cc:844] Serialized 52 byte log entry
0709 13:02:12.601927 (+776129us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b18001a9f20. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:12.754901 (+152974us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:12.757758 (+  2857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.758046 (+   288us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.951983 (+193937us) raft_consensus.cc:1685] finished
0709 13:02:12.952007 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.952279 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10768,"raft.queue_time_us":43896,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":159,"spinlock_wait_cycles":4489984,"thread_start_us":293242,"threads_started":1,"wal-append.queue_time_us":936383}
W20260709 13:02:13.295586  1534 connection.cc:256] RPC call timeout handler was delayed by -1.3792381559999853s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3149s for a response
I20260709 13:02:13.170673  3538 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:13.255301  3850 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.293008  1636 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
W20260709 13:02:13.295969  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 58157.1us
W20260709 13:02:13.303668  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 147886us
W20260709 13:02:13.264303  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 98010us
I20260709 13:02:13.251991  4245 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:13.305692  1715 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:13.257304  4060 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:13.307165  1713 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:02:13.317183  1609 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:13.258898  4206 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:13.293524  4187 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:13.322141  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:13.324471  1681 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:13.258692  4204 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:13.293907  4330 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:13.325256  4261 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.825s	user 0.022s	sys 0.041s
I20260709 13:02:13.293780  4315 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.380658  4315 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:13.327982  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 138643us
I20260709 13:02:13.383605  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:13.372110  3538 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.386935  4175 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:13.257532  4331 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.387493  4175 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:13.387189  4338 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:13.388263  4338 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.388655  4338 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:13.388475  4175 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.389288  4331 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:13.389858  4331 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:02:13.392338  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 67921.3us
I20260709 13:02:13.393095  1681 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:13.394419  4282 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:13.373363  4187 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:13.395617  4187 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:13.335273  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:13.397187  1635 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 3.
I20260709 13:02:13.335996  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2219) took 1731 ms. Trace:
W20260709 13:02:13.336690  4335 log.cc:927] Time spent T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:02:13.339763  4192 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.326272  4317 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:13.348840  4328 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:13.351332  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2207) took 1805 ms (client timeout 1749 ms). Trace:
I20260709 13:02:13.355634  3508 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:13.307090  4200 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:13.357388  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:13.357817  1682 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:13.423290  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 65553.1us
I20260709 13:02:13.356536  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:13.307755  4182 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.311631  4124 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.367267  1712 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:13.255507  4115 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:13.295862  4290 consensus_queue.cc:1059] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:02:13.334314  4252 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.251076  4280 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:13.378883  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 61783.8us
I20260709 13:02:13.395097  4258 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:02:13.402005  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:13.400496  4282 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:13.363080  3850 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:13.365823  4060 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.419060  1717 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:13.314157  4236 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:13.418331  1567 rpcz_store.cc:269] 0709 13:02:11.545949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.552791 (+  6842us) service_pool.cc:224] Handling call
0709 13:02:13.351150 (+1798359us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54003e8354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:13.351263 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327773440}
I20260709 13:02:13.378371  4340 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:13.387276  3538 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:13.459542  4256 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.466780  1709 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:02:13.467963  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:13.334411  4283 consensus_meta.cc:287] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.601s	user 0.009s	sys 0.000s
I20260709 13:02:13.444169  1715 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:13.449138  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:13.470836  3508 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:13.471223  2302 raft_consensus.cc:2376] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5. Re-sending same reply.
W20260709 13:02:13.402743  4261 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.915s	user 0.022s	sys 0.041s
I20260709 13:02:13.472842  1609 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:13.440755  4124 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:13.450742  4060 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:13.429674  4291 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:13.463531  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 106281us
I20260709 13:02:13.432363  4317 raft_consensus.cc:697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:13.475553  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:13.492220  1710 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:02:13.476711  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:13.481173  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:13.470906  4309 consensus_meta.cc:287] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.678s	user 0.007s	sys 0.004s
I20260709 13:02:13.443691  4200 raft_consensus.cc:697] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:13.486768  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:13.447163  4182 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:13.443202  4339 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:13.443341  4147 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:13.475466  4342 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:13.462134  4328 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:13.496397  1640 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:13.485649  4331 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.497308  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:13.497659  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:13.460673  3963 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.465678  4245 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.511032  1718 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:13.547374  2303 raft_consensus.cc:2368] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5.
I20260709 13:02:13.493434  1715 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:13.512714  4280 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.516474  4314 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:13.517985  2297 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:13.517875  4320 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.520835  1635 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:13.498935  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:13.507833  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:02:13.484115  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 126313us
W20260709 13:02:13.522634  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 49881.7us
W20260709 13:02:13.474602  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 50629.7us
I20260709 13:02:13.533818  4337 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:13.533984  1711 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:13.534631  4220 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:13.535240  3850 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.397951  1562 rpcz_store.cc:276] 0709 13:02:11.604094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.777226 (+173132us) service_pool.cc:224] Handling call
0709 13:02:11.777661 (+   435us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:11.824669 (+ 47008us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.824707 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:12.456269 (+631562us) log.cc:844] Serialized 52 byte log entry
0709 13:02:12.708163 (+251894us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:12.857703 (+149540us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.859220 (+  1517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.859333 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:13.225278 (+365945us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:13.335658 (+110380us) raft_consensus.cc:1685] finished
0709 13:02:13.335773 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.335931 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14435,"prepare.queue_time_us":145778,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":50341,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":244,"spinlock_wait_cycles":7279488,"thread_start_us":344697,"threads_started":2,"wal-append.queue_time_us":299248}
I20260709 13:02:13.495811  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:13.539757  4187 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.539912  4315 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.540294  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:13.509996  4275 consensus_meta.cc:287] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.741s	user 0.011s	sys 0.000s
I20260709 13:02:13.496991  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:13.683346  2305 raft_consensus.cc:2410] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:13.545288  1641 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:13.647069  3963 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:13.645867  2298 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:02:13.603062  1710 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:13.598096  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 241628us
I20260709 13:02:13.712620  1683 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:13.603655  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 117014us
I20260709 13:02:13.626964  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:13.627347  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:13.521838  4347 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.543654  4296 consensus_meta.cc:287] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.584s	user 0.009s	sys 0.002s
I20260709 13:02:13.520645  4309 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:13.726178  1709 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:13.635550  1718 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:02:13.635902  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:13.476019  4330 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:13.548053  4048 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.640676  1711 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:13.543016  4182 raft_consensus.cc:697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:13.600338  3865 consensus_queue.cc:1059] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:02:13.645828  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:13.646181  4346 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.646608  4320 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:13.475646  4340 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:13.541392  4339 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.509851  4233 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:13.793304  4340 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:13.636618  4337 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:13.794533  4337 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:13.796944  4187 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.798166  1643 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:13.495965  4326 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:13.602396  4331 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.679338  2296 raft_consensus.cc:2410] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:02:13.676752  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.799579  3865 raft_consensus.cc:3055] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:13.800000  3865 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:13.626420  4245 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.800912  3865 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.688812  3850 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.689203  4192 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.601593  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:13.665527  4189 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:13.803017  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.695583  4275 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.697597  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:13.678573  4200 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.699301  4206 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.699796  4217 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:13.699622  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:13.826463  1716 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:13.673012  4317 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.703382  4236 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.665184  4115 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:13.844229  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.721217  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 93955.4us
I20260709 13:02:13.846015  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:13.784536  4228 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.723194  2301 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:02:13.724170  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:13.724634  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.721647  1534 connection.cc:256] RPC call timeout handler was delayed by -1.1530998669999803s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
I20260709 13:02:13.724980  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:13.794566  4233 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:13.485965  4283 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.727775  2303 raft_consensus.cc:2410] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:13.729516  1710 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:13.732331  1712 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6711207c14044518a57e7c8914ac0091"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:02:13.551923  4282 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.779304  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:13.605989  4342 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:13.705384  4204 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:13.872349  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:13.826076  4326 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:13.784623  1642 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:13.784073  2297 raft_consensus.cc:2376] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5. Re-sending same reply.
I20260709 13:02:13.783864  4346 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.785431  4048 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:13.792681  4339 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.794857  4276 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:13.802215  3865 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:13.650200  4338 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:13.822778  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 175763us
I20260709 13:02:13.824425  2305 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:13.730381  4021 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:13.731676  4296 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:13.726745  4212 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:13.730813  4291 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:13.783502  4350 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.849102  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.936303  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.936893  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.851588  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:13.852463  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:13.678735  2299 raft_consensus.cc:2410] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:02:13.867218  4124 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:13.860136  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 123896us
W20260709 13:02:13.864039  1534 connection.cc:256] RPC call timeout handler was delayed by -1.1422339370000145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17920000000000003s for a response
I20260709 13:02:13.684455  3538 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:13.810384  4252 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:13.871410  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:13.841605  4330 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:13.943393  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:13.945928  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:13.881950  1716 raft_consensus.cc:2410] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:02:13.889503  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 165446us
I20260709 13:02:13.946375  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:13.912953  1714 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 3: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:13.903869  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:13.901965  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:13.899659  4363 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.889544  4228 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.898281  4355 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.726178  4347 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:13.899223  4182 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.895998  4128 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.901392  4315 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.914471  4206 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.898489  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:13.901259  4275 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:13.912321  4283 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.903159  4342 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:13.911432  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2206) took 2697 ms. Trace:
I20260709 13:02:13.919652  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:13.904410  4333 raft_consensus.cc:2764] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:02:13.862859  4220 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:13.916517  4292 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:13.841964  4340 consensus_queue.cc:1059] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:02:13.936213  1532 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:13.853169  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:13.937675  1636 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
I20260709 13:02:13.853855  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:13.939864  1534 connection.cc:256] RPC call timeout handler was delayed by -1.0520963879999954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
I20260709 13:02:13.904233  4309 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:13.947919  3508 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.949110  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:13.943267  2302 raft_consensus.cc:2368] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5.
I20260709 13:02:13.931016  4021 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.951166  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:13.973308  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:14.079514  1569 raft_consensus.cc:2410] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:13.952340  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:13.952785  1564 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:13.953608  2304 raft_consensus.cc:2393] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 4: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:13.961453  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:14.080186  1565 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:14.080231  1644 rpcz_store.cc:276] 0709 13:02:11.213846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.214423 (+   577us) service_pool.cc:224] Handling call
0709 13:02:11.225911 (+ 11488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.550412 (+1324501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.911145 (+1360733us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400247f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:13.911335 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3041806080}
I20260709 13:02:13.966149  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:13.936337  4060 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:13.942548  4350 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:13.970504  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:13.962174  4256 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:02:14.079022  3508 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:14.078148  1561 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:14.092803  1641 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:14.079867  1635 raft_consensus.cc:2393] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:14.034610  1712 raft_consensus.cc:606] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:14.082074  1637 raft_consensus.cc:2410] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:02:14.093779  1607 connection.cc:256] RPC call timeout handler was delayed by -1.7599583280000388s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
I20260709 13:02:14.094002  1712 raft_consensus.cc:614] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:14.029557  4362 log.cc:927] Time spent T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:02:14.078308  1638 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:14.108197  2303 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:02:13.972869  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:13.942482  4369 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.951668  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:14.116562  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:14.131083  1638 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:14.014771  2305 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
W20260709 13:02:14.087596  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 151472us
I20260709 13:02:14.090126  1710 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
W20260709 13:02:14.084733  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 138876us
I20260709 13:02:14.030077  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:14.097225  2302 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:13.952399  4212 raft_consensus.cc:697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:14.097242  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:13.999428  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:14.134032  2297 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:14.136021  4347 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:14.095104  1562 raft_consensus.cc:2376] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6. Re-sending same reply.
I20260709 13:02:14.099107  4342 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:02:14.100162  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2217) took 2509 ms (client timeout 2325 ms). Trace:
I20260709 13:02:14.100596  4060 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:14.032392  4338 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:14.023303  4048 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:14.030836  4346 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:14.102171  4352 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.104019  4350 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:14.104471  4364 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.150s
I20260709 13:02:14.068859  4355 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:14.109025  2300 raft_consensus.cc:2376] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5. Re-sending same reply.
I20260709 13:02:14.022792  4345 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.116559  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:14.130962  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 179378us
I20260709 13:02:14.086627  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:14.137192  4342 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.101339  4187 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.079142  4220 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.071388  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:14.133224  3508 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.981374  4118 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.197s
I20260709 13:02:14.070652  4124 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:13.959444  4094 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.191s
I20260709 13:02:14.035207  3947 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.145s
W20260709 13:02:14.094254  1607 connection.cc:256] RPC call timeout handler was delayed by -1.7401583280000068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16590000000000002s for a response
I20260709 13:02:14.034282  4228 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:14.116559  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:14.087167  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:13.942167  4291 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:14.137660  1709 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:14.387945  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:14.384521  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.084028  4021 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:14.398149  1716 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:13.960768  4363 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:14.152655  1713 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:14.174113  2297 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:14.021936  4276 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:14.091697  4340 consensus_queue.cc:1243] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:14.402740  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:14.425805  1568 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:14.035774  4326 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:14.137506  1566 rpcz_store.cc:269] 0709 13:02:11.590702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.670522 (+ 79820us) service_pool.cc:224] Handling call
0709 13:02:14.099949 (+2429427us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54000a7dd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:14.100071 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3635451648}
W20260709 13:02:14.397859  1607 connection.cc:256] RPC call timeout handler was delayed by -1.6573583280000435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 13:02:14.429883  1607 connection.cc:256] RPC call timeout handler was delayed by -1.5439583280000306s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1877s for a response
W20260709 13:02:14.182749  4360 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 13:02:14.411643  1569 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:14.303761  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:14.430511  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:14.194433  1718 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:14.427882  1533 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:14.317008  1560 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:14.418900  4342 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:14.441520  4359 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.420745  4187 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:14.426558  2299 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:14.109797  4309 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.251189  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:14.450408  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:14.417760  4314 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:14.471238  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:14.425480  4350 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.371506  4256 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:14.451834  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:14.452471  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:14.477268  1569 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:14.420564  4345 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:14.476531  1561 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:14.401453  4220 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:14.186144  4315 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:14.454741  4363 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:14.455225  4333 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:14.461851  4291 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:14.463956  4048 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:14.464772  4339 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:14.465956  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1672) took 1220 ms. Trace:
I20260709 13:02:14.466102  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:14.442194  4359 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:14.467905  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:14.505687  2297 raft_consensus.cc:2393] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:14.505038  1717 rpcz_store.cc:276] 0709 13:02:13.245659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.246993 (+  1334us) service_pool.cc:224] Handling call
0709 13:02:14.465741 (+1218748us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54001365d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:14.465878 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284696832}
I20260709 13:02:14.469211  4330 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:14.470463  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:14.471164  4314 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:14.472092  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:14.513352  2300 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:14.179168  4369 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:14.438210  1710 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:14.473994  2304 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:14.085409  4128 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:14.473675  4124 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:14.494400  4346 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.437444  4338 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:14.428233  2302 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:14.475734  2298 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:14.487870  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:14.431357  1638 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:14.524660  1563 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:14.518201  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 47811.7us
I20260709 13:02:14.516970  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:14.478503  4350 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:14.474500  4309 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:14.422123  4352 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:14.484261  4256 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:14.522637  1569 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:14.532613  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:14.516109  4269 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:14.480927  2305 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:02:14.524772  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.529397  2302 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:14.475618  4212 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:14.533912  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:14.531113  1561 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:14.551075  1561 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:14.538491  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 524) took 1060 ms. Trace:
I20260709 13:02:14.507746  4339 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:14.535018  4258 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.014s	sys 0.005s
I20260709 13:02:14.527555  4370 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.526330  4124 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:14.529464  4206 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:14.552286  1715 rpcz_store.cc:276] 0709 13:02:13.478285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.600521 (+122236us) service_pool.cc:224] Handling call
0709 13:02:13.600871 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:13.601238 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:13.601278 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:13.601769 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:02:13.730843 (+129074us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:13.730895 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.730996 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:14.182147 (+451151us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:14.530459 (+348312us) raft_consensus.cc:1685] finished
0709 13:02:14.530491 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.538383 (+  7892us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4085,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":6468,"spinlock_wait_cycles":45202688,"thread_start_us":84317,"threads_started":1,"wal-append.queue_time_us":282579}
I20260709 13:02:14.527940  4333 raft_consensus.cc:697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:14.558109  1566 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:14.548713  2302 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:14.540216  4338 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:14.557075  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 140999us
I20260709 13:02:14.618855  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:14.716089  1534 leader_election.cc:336] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.792s (SENT)
I20260709 13:02:14.661556  1563 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:14.604137  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:14.620999  1569 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:02:14.669282  1566 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:14.601984  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:14.716768  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:14.621587  4368 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.589s	user 0.000s	sys 0.001s
I20260709 13:02:14.560514  2302 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:14.553045  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 792) took 1013 ms. Trace:
I20260709 13:02:14.674562  4128 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:14.556725  4339 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:14.755645  1534 leader_election.cc:336] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.999s (SENT)
I20260709 13:02:14.756392  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:14.557685  4124 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:14.767532  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:14.618101  4256 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:14.717449  4369 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:14.772068  2302 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:02:14.555769  4233 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:14.589812  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 1.19566e+06us
I20260709 13:02:14.662802  1568 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
W20260709 13:02:14.554175  4258 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.015s	sys 0.005s
I20260709 13:02:14.681849  4296 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.671136  4355 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:14.601263  4021 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:14.613107  3947 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.127s
I20260709 13:02:14.586723  4094 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:14.544898  4346 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:14.624179  4352 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:14.720985  4350 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:14.665405  4269 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 13:02:14.597673  4187 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:14.596416  4314 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:14.779968  4256 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:14.717653  4220 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:14.613557  3963 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.153s
I20260709 13:02:14.594781  2299 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:14.753504  4261 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:14.775260  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 800) took 1096 ms. Trace:
I20260709 13:02:14.824225  2302 rpcz_store.cc:276] 0709 13:02:13.679274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.402574 (+723300us) service_pool.cc:224] Handling call
0709 13:02:14.410409 (+  7835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.772507 (+362098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.775199 (+  2692us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.766186  4206 raft_consensus.cc:697] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:14.826347  4377 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:02:14.826519  4338 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:14.558341  4370 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:14.622016  4309 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:14.826524  4206 consensus_queue.cc:237] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:14.743883  2298 rpcz_store.cc:276] 0709 13:02:13.532712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.075511 (+542799us) service_pool.cc:224] Handling call
0709 13:02:14.546430 (+470919us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:14.825039  3963 consensus_queue.cc:1243] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:02:14.836773  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.839432  4245 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:14.843422  4345 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:14.845301  4359 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:14.846275  4355 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:14.846716  4355 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:14.847754  4355 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:14.849349  4355 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:14.852200  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:14.853082  1638 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:14.854302  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:14.854887  1638 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:14.855758  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:14.856444  1638 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:02:14.856732  4147 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:02:14.857160  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:14.858604  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:14.858982  4314 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.859272  1562 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:14.859493  4314 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:14.861693  4314 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:14.864022  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:14.864485  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.794457  4094 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:14.805648  1681 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:14.792806  4296 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:14.578627  4349 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.011s	sys 0.016s
I20260709 13:02:14.814383  1534 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:14.820101  4021 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:14.821297  4344 log.cc:927] Time spent T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 1.183s	user 0.001s	sys 0.001s
I20260709 13:02:14.776739  4342 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:14.854399  4331 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:14.863906  4220 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:14.795167  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:14.865648  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 60053.4us
I20260709 13:02:14.953975  1681 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:14.811489  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:14.867079  1533 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:14.607443  4333 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
W20260709 13:02:14.870882  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.967012  1682 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:14.874048  4187 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:14.875346  4094 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:14.877586  4296 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:14.878036  1643 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:14.919275  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 221194us
I20260709 13:02:14.947487  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 498) took 2091 ms. Trace:
I20260709 13:02:14.822224  4233 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:14.883924  4349 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.012s	sys 0.017s
I20260709 13:02:14.971251  1635 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:14.973006  1682 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:14.992390  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:14.993036  1635 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:14.975296  1681 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:14.974431  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 107513us
I20260709 13:02:14.994127  1533 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:14.982363  1639 rpcz_store.cc:276] 0709 13:02:12.856251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.858878 (+  2627us) service_pool.cc:224] Handling call
0709 13:02:12.864927 (+  6049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.926851 (+ 61924us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:12.948809 (+ 21958us) log.cc:844] Serialized 14 byte log entry
0709 13:02:13.029250 (+ 80441us) log.cc:844] Serialized 14 byte log entry
0709 13:02:13.142662 (+113412us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.142716 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:13.142755 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.947101 (+1804346us) spinlock_profiling.cc:243] Waited 61.7 ms on lock 0x7b5400126454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:14.947179 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.947394 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24609,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":128,"raft.queue_time_us":33825,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"spinlock_wait_cycles":141901056,"thread_start_us":192745,"threads_started":2,"wal-append.queue_time_us":221334,"wal-append.run_cpu_time_us":1913,"wal-append.run_wall_time_us":1074657}
I20260709 13:02:14.946733  4147 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:14.952373  4359 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:14.999364  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:15.001633  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.971962  4256 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:15.001578  1635 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:14.966802  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 454) took 1012 ms. Trace:
W20260709 13:02:14.991657  4200 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.084s	user 0.006s	sys 0.013s
I20260709 13:02:14.986349  4258 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:15.004159  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:14.972522  4350 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:14.983151  4370 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:15.009982  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 195734us
I20260709 13:02:15.010881  1534 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:15.007910  1635 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:14.987879  4021 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.012879  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:15.013360  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:14.997210  4291 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:15.015686  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:15.006734  4200 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.098s	user 0.006s	sys 0.014s
I20260709 13:02:15.000494  4261 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:15.003420  4256 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.027859  4350 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:15.046664  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:15.047549  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.049000  1635 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:15.016651  4269 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:15.030695  4359 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:15.004999  4147 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.007162  4258 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.039451  4021 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.044337  1643 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:14.986574  4182 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 13:02:15.081967  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 152610us
W20260709 13:02:15.082423  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 134758us
I20260709 13:02:15.017738  4291 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.084918  1534 leader_election.cc:400] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5.
I20260709 13:02:15.040647  4261 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.090816  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:15.091593  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:15.094897  1533 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:15.096130  1534 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:15.089718  2269 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:15.045445  4256 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:15.087692  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 79647.6us
I20260709 13:02:15.098706  4392 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:15.099576  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:15.101188  4392 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:15.101290  1681 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:15.107828  4349 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:15.108294  4349 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.108409  4048 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:15.108530  1534 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:15.108946  4345 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:15.088416  4258 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:15.090567  4317 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.110422  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:15.096311  4261 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:15.088608  4314 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:15.093918  4291 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.088388  4332 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:15.076579  4370 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:02:15.081203  4350 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:15.127516  2305 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:15.158394  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:15.115413  4392 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:15.159878  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.187191  2304 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:15.160631  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.105544  4352 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:15.161190  4269 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:15.103346  4369 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:15.128063  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.170430  2296 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:15.172547  1681 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 2.325s (SENT) [suppressed 19 similar messages]
I20260709 13:02:15.142848  1635 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:15.113479  4094 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:15.180977  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:15.159072  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.103121  4233 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:15.187695  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 194615us
I20260709 13:02:15.212265  2302 raft_consensus.cc:2368] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5.
I20260709 13:02:15.108956  4048 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.190397  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:15.227967  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.152940  4347 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:15.231655  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:15.112062  4349 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.107877  4128 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 13:02:15.110473  4345 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.185304  2299 raft_consensus.cc:3055] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:15.213438  2303 raft_consensus.cc:2410] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:15.213201  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:15.240033  1715 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.189262  4392 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:15.243883  1562 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:15.244382  4392 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.244626  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1866) took 1289 ms. Trace:
I20260709 13:02:15.245044  1562 rpcz_store.cc:276] 0709 13:02:13.955554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.858325 (+902771us) service_pool.cc:224] Handling call
0709 13:02:15.244564 (+386239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:15.245980  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.246616  2304 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:15.204270  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:15.247212  1635 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:02:15.247524  4389 log.cc:927] Time spent T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
W20260709 13:02:15.248908  4387 log.cc:927] Time spent T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:02:15.225931  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:15.251250  1718 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:15.230648  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:15.252332  1709 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:15.232257  1710 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.157565  4317 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.198289  4369 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.259321  4369 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.260512  4317 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.214196  4094 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:15.261377  4094 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.234659  1717 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:02:15.236143  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51756.1us
I20260709 13:02:15.265472  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:15.266947  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.267597  2303 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:15.267992  4147 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.268446  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.269596  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.270056  4369 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.223631  3963 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:15.271343  3963 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.272519  4369 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.273339  3963 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.280772  4369 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:15.281867  4369 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.282579  4369 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:15.284617  4358 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 1.386s	user 0.033s	sys 0.029s
I20260709 13:02:15.270651  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.287889  2303 raft_consensus.cc:2368] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6.
I20260709 13:02:15.291862  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:15.292549  1644 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.295315  1710 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:15.247545  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.297374  2304 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:15.296643  1635 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:15.297324  1715 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:15.237028  4370 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:15.180850  4314 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.238116  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:15.196112  2298 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:15.198055  4342 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.242717  2305 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:15.243402  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:15.302592  1713 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:15.188205  4332 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.298818  1533 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:15.243808  4021 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:15.301653  1711 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:15.313191  1644 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:15.231734  4048 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.239662  2299 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:15.295776  4182 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.308456  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:15.308491  4395 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:15.311053  4314 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:15.311286  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1868) took 1355 ms. Trace:
I20260709 13:02:15.327544  1569 rpcz_store.cc:276] 0709 13:02:13.956111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.544580 (+588469us) service_pool.cc:224] Handling call
0709 13:02:14.544976 (+   396us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:14.720567 (+175591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.311011 (+590444us) spinlock_profiling.cc:243] Waited 75.7 ms on lock 0x7b54002981d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:15.311223 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174095232}
I20260709 13:02:15.206110  4350 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.247572  4187 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:15.314677  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:15.325368  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:15.314563  4269 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:15.324838  2298 raft_consensus.cc:2368] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6.
I20260709 13:02:15.306785  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.334830  4269 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.337067  2299 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.338248  2299 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.325100  4182 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.343125  4269 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.345093  4182 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:15.335979  4395 raft_consensus.cc:697] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:15.346330  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:15.351828  1683 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:15.351966  2298 raft_consensus.cc:2376] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4. Re-sending same reply.
I20260709 13:02:15.352048  4395 consensus_queue.cc:237] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.352865  4182 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:15.353160  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:15.353348  4182 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:15.353771  4182 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.354718  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:15.355448  2298 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:15.356446  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.357683  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.359202  4269 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:15.359751  4269 raft_consensus.cc:697] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:15.360594  4269 consensus_queue.cc:237] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.360947  4258 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.363710  4258 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:15.365523  1609 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:15.366497  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 452) took 1414 ms. Trace:
I20260709 13:02:15.368538  1683 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:15.376269  1564 rpcz_store.cc:276] 0709 13:02:13.952233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.952337 (+   104us) service_pool.cc:224] Handling call
0709 13:02:13.952611 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.080736 (+128125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.366197 (+1285461us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b54000bc820. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c18dc0 00007fd7a1ca4a02 00007fd7a1ca0dd6 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:02:15.366430 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34440064}
I20260709 13:02:15.377601  1683 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:15.379165  1683 leader_election.cc:400] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6.
I20260709 13:02:15.379575  1683 leader_election.cc:403] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:02:15.371793  4256 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:15.372694  4124 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.549s	user 0.007s	sys 0.003s
I20260709 13:02:15.005481  1560 rpcz_store.cc:276] 0709 13:02:13.954152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.525426 (+571274us) service_pool.cc:224] Handling call
0709 13:02:14.560024 (+ 34598us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:14.664701 (+104677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:14.665915 (+  1214us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:14.675363 (+  9448us) log.cc:844] Serialized 26 byte log entry
0709 13:02:14.794804 (+119441us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:14.809005 (+ 14201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.816714 (+  7709us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:14.883592 (+ 66878us) raft_consensus.cc:1685] finished
0709 13:02:14.886539 (+  2947us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.952773 (+ 66234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31666,"raft.queue_time_us":64498,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":7220,"thread_start_us":112098,"threads_started":1,"wal-append.queue_time_us":147939}
I20260709 13:02:15.358741  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.277287  4094 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.379987  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:15.385658  4204 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:15.397064  4236 raft_consensus.cc:3055] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:15.400887  4256 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.375262  4261 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.412052  4236 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:15.412909  2299 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:02:15.413463  2299 pending_rounds.cc:85] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 6
I20260709 13:02:15.413774  2299 pending_rounds.cc:107] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:02:15.413098  4236 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.413965  4261 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.385665  4314 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:15.403764  4367 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 1.476s	user 0.005s	sys 0.009s
I20260709 13:02:15.346181  4369 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.409262  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 515) took 1934 ms. Trace:
I20260709 13:02:15.416839  1714 rpcz_store.cc:276] 0709 13:02:13.464501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.475278 (+ 10777us) service_pool.cc:224] Handling call
0709 13:02:15.398702 (+1923424us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b5400339ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:15.398871 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":967528832}
I20260709 13:02:15.406569  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 450) took 1455 ms. Trace:
I20260709 13:02:15.418315  1563 rpcz_store.cc:276] 0709 13:02:13.951575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.951699 (+   124us) service_pool.cc:224] Handling call
0709 13:02:13.959834 (+  8135us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:14.718621 (+758787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.406277 (+687656us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b5400297cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:15.406497 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1298790272}
W20260709 13:02:15.407557  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.410419  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:15.415910  3963 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:15.408867  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1761) took 1544 ms. Trace:
I20260709 13:02:15.421924  2304 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:15.422310  1641 rpcz_store.cc:276] 0709 13:02:13.864978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.946112 (+ 81134us) service_pool.cc:224] Handling call
0709 13:02:15.406887 (+1460775us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54002d8054. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:15.408812 (+  1925us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328864512}
W20260709 13:02:15.422765  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 43889.5us
I20260709 13:02:15.417212  4345 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:15.401510  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.399081  4124 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:15.423638  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:15.424789  1683 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:15.414400  4182 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:15.433156  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.434514  4332 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:15.428243  4204 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.436098  4258 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:15.434756  4182 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:15.437255  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:15.386171  4212 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:15.437925  4212 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.438186  4236 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:15.438385  4212 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.437927  2297 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.418329  4369 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:15.430404  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:15.414054  2299 raft_consensus.cc:2889] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:15.427124  4331 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.442538  1636 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:15.443004  2299 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:15.421324  2298 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:15.448413  1683 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:15.451076  3508 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:15.442445  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:15.423091  4206 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:15.463074  1683 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:15.459374  2298 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:15.465094  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.423380  4094 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.466887  2298 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:15.474802  2297 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:15.480614  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:15.471235  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:15.491540  1636 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:15.443017  4331 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.505877  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 786) took 2021 ms. Trace:
I20260709 13:02:15.514899  2301 rpcz_store.cc:276] 0709 13:02:13.483956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.484739 (+   783us) service_pool.cc:224] Handling call
0709 13:02:13.720785 (+236046us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.861192 (+140407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.505653 (+1644461us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b540068d3d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:15.505808 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":524215168}
I20260709 13:02:15.516489  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.518442  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:15.520947  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1757) took 2205 ms. Trace:
I20260709 13:02:15.521453  1642 rpcz_store.cc:276] 0709 13:02:13.315552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.315776 (+   224us) service_pool.cc:224] Handling call
0709 13:02:13.316080 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.873978 (+557898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.520703 (+1646725us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b5400154d54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:15.520888 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1077296000}
W20260709 13:02:15.496520  4339 consensus_meta.cc:287] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.727s	user 0.011s	sys 0.000s
I20260709 13:02:15.521344  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:38292 (request call id 26) took 1773 ms. Trace:
I20260709 13:02:15.483500  4200 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:15.547806  2296 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:15.469745  4332 raft_consensus.cc:2764] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:02:15.549542  1712 rpcz_store.cc:276] 0709 13:02:13.732035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.732169 (+   134us) service_pool.cc:224] Handling call
0709 13:02:15.505155 (+1772986us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b5400474354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c9b853 00007fd7a6c6960a 00007fd7a01ef7d4 00007fd7a01ef762 00007fd7a01ef691 00007fd7a01ef60c 00007fd7a01ee8c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:15.505298 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":382113664}
I20260709 13:02:15.554875  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:15.555706  1642 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:15.557324  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.553326  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 169330us
I20260709 13:02:15.558971  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:15.559541  1637 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.560442  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:15.561964  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.547196  2297 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:15.496405  4206 raft_consensus.cc:697] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:15.455299  4392 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.482460  4187 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:02:15.472776  4258 consensus_queue.cc:1059] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:02:15.544003  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:15.566237  4200 raft_consensus.cc:697] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:15.584532  1641 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:02:15.587448  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.588023  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.593601  2305 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
W20260709 13:02:15.593858  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 183520us
I20260709 13:02:15.572398  1636 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:15.565958  4314 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:15.574342  4345 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:15.597813  1642 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
W20260709 13:02:15.583945  4175 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.656s	user 0.005s	sys 0.026s
I20260709 13:02:15.600049  1712 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:02:15.600905  1712 raft_consensus.cc:606] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:02:15.601342  1712 raft_consensus.cc:614] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:15.604120  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.605392  2305 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:15.609961  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:15.611773  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:15.582938  4395 consensus_queue.cc:1059] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:02:15.615865  4395 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:15.615391  4200 consensus_queue.cc:237] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.616333  4395 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:15.617326  4395 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:15.603977  4175 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.676s	user 0.005s	sys 0.026s
W20260709 13:02:15.622119  4382 log.cc:927] Time spent T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.669s	user 0.002s	sys 0.000s
I20260709 13:02:15.623605  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:15.560928  4350 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:02:15.624751  1682 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:15.625857  3508 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:15.558636  4339 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.635232  1637 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:15.624485  1636 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:15.585561  4124 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.626412  3508 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:15.637058  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:15.637859  1636 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:15.638085  3508 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.634605  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:15.596338  2299 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.597136  4182 raft_consensus.cc:3055] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:15.603185  4345 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.652856  2297 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:15.643700  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.598939  4258 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.659004  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:15.618808  4395 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:15.665859  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 104802us
I20260709 13:02:15.654366  2269 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:15.665480  4258 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:15.671522  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.671072  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:15.672567  4258 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:15.664001  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:15.673132  1609 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:15.673273  2305 raft_consensus.cc:2376] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7. Re-sending same reply.
I20260709 13:02:15.674000  4346 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.666860  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:15.659157  1636 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:15.607860  4400 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.674760  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.681186  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 512) took 1078 ms. Trace:
I20260709 13:02:15.681735  1643 rpcz_store.cc:276] 0709 13:02:14.602495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.618560 (+ 16065us) service_pool.cc:224] Handling call
0709 13:02:15.680990 (+1062430us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b540032cdd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:15.681127 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370961664}
W20260709 13:02:15.683104  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 508) took 2138 ms (client timeout 1528 ms). Trace:
W20260709 13:02:15.683545  1640 rpcz_store.cc:269] 0709 13:02:13.544941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.546184 (+  1243us) service_pool.cc:224] Handling call
0709 13:02:15.682871 (+2136687us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b54000d07d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:15.683033 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2163641728}
I20260709 13:02:15.684669  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:15.681002  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:15.688234  1637 raft_consensus.cc:2393] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:15.676131  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:15.691473  1636 raft_consensus.cc:2393] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:15.692011  2269 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:15.693205  1640 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:15.674754  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:15.652463  4182 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:15.661511  4345 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.696341  1642 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:15.696439  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 515) took 1033 ms. Trace:
I20260709 13:02:15.697280  1640 rpcz_store.cc:276] 0709 13:02:14.664034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.692725 (+1028691us) service_pool.cc:224] Handling call
0709 13:02:15.693038 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.696165 (+  3127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.696382 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:15.642841  4233 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:02:15.700939  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.701912  2296 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
W20260709 13:02:15.703032  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 144916us
I20260709 13:02:15.666491  4315 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.704208  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:15.705348  1643 raft_consensus.cc:2410] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:15.717684  3963 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:15.718272  4182 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.719224  3963 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:15.719785  4182 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:15.722079  4352 raft_consensus.cc:3055] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:15.695415  4314 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.725884  1641 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:15.726869  4175 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:15.727514  4175 raft_consensus.cc:697] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:15.728432  4175 consensus_queue.cc:237] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:15.731248  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.733301  1641 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:15.667361  4333 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:02:15.733796  1636 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: [4.5-6.6]
I20260709 13:02:15.702736  4094 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:15.752915  1636 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:15.722720  4352 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:15.719810  3963 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:15.754020  4124 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:15.738793  1637 pending_rounds.cc:85] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 4
I20260709 13:02:15.758832  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1774) took 1063 ms. Trace:
I20260709 13:02:15.707906  4315 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.762249  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1764) took 1137 ms. Trace:
W20260709 13:02:15.746031  1535 connection.cc:256] RPC call timeout handler was delayed by -1.7798642829999949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:02:15.768457  1635 rpcz_store.cc:276] 0709 13:02:14.624784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.315011 (+690227us) service_pool.cc:224] Handling call
0709 13:02:15.315316 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.381746 (+ 66430us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:15.381794 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:15.381856 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:15.381903 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.762004 (+380101us) spinlock_profiling.cc:243] Waited 8.94 ms on lock 0x7b5400247a54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:15.762059 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.762194 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20555136}
I20260709 13:02:15.769249  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:15.769806  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:15.770329  1715 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:15.770900  1638 rpcz_store.cc:276] 0709 13:02:14.695516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.856954 (+161438us) service_pool.cc:224] Handling call
0709 13:02:15.758652 (+901698us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b54002d8f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:15.758774 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1672737024}
I20260709 13:02:15.768028  1637 pending_rounds.cc:107] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:02:15.773030  1637 raft_consensus.cc:2889] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:15.748891  2268 leader_election.cc:336] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.819s (SENT)
I20260709 13:02:15.773692  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:15.746718  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:15.766151  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 513) took 1163 ms. Trace:
I20260709 13:02:15.774571  1639 rpcz_store.cc:276] 0709 13:02:14.602724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.083352 (+480628us) service_pool.cc:224] Handling call
0709 13:02:15.765452 (+682100us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:02:15.765638 (+   186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.765823 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.765848 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:15.765890 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:15.765920 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.765974 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:15.766093 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37582208}
I20260709 13:02:15.676441  4349 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.775774  4331 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.776592  4094 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:15.777719  4349 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.777568  4094 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.779654  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:15.779462  4352 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.780344  4393 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.780534  4352 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.780882  4393 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.781692  4204 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6.
I20260709 13:02:15.770974  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:15.668566  4206 consensus_queue.cc:237] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.730818  4332 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:02:15.792084  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:15.792591  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 125815us
I20260709 13:02:15.793404  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:15.790541  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 43986us
I20260709 13:02:15.793927  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.794629  1534 leader_election.cc:400] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5.
I20260709 13:02:15.796090  1534 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:15.797082  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:15.797657  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 48843.9us
I20260709 13:02:15.794579  1564 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:15.798818  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:15.799340  1561 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:15.801312  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:15.801826  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:15.802623  1564 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:15.804425  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.737166  4412 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:15.757174  4410 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:15.805697  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.811640  1567 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:15.785877  4269 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.805094  1563 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:15.807833  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.817963  1565 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:15.808386  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.812368  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.767634  4124 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:15.807049  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.815109  1534 leader_election.cc:400] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6.
I20260709 13:02:15.821924  1534 leader_election.cc:403] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:15.816660  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.810295  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:15.818838  1564 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:15.820382  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:15.821753  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:15.703585  4400 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:15.852083  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.824684  1567 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.829927  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.854511  1563 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:15.802160  4331 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.800753  4346 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.800384  4340 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:15.801054  4147 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:15.808753  4204 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.822583  1561 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
W20260709 13:02:15.858381  4396 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.272s	user 0.001s	sys 0.001s
I20260709 13:02:15.849399  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:15.892949  1709 raft_consensus.cc:2376] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6. Re-sending same reply.
W20260709 13:02:15.893016  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 99705.4us
I20260709 13:02:15.883057  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:15.828145  4124 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.848272  1534 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:15.828791  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.846237  4349 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6.
I20260709 13:02:15.820066  4369 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:15.846407  4412 raft_consensus.cc:697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:15.895778  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:15.812007  4347 consensus_queue.cc:1059] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.070s
W20260709 13:02:15.896766  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 48626.8us
W20260709 13:02:15.913726  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1870) took 1805 ms (client timeout 1648 ms). Trace:
W20260709 13:02:15.914180  1566 rpcz_store.cc:269] 0709 13:02:14.108623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.430109 (+321486us) service_pool.cc:224] Handling call
0709 13:02:15.913550 (+1483441us) spinlock_profiling.cc:243] Waited 81.1 ms on lock 0x7b5400200654. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:15.913664 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186618112}
I20260709 13:02:15.862697  4392 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:15.897517  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:15.878033  4175 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:15.902541  1567 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:15.887841  4333 raft_consensus.cc:2764] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:02:15.879691  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.826062  4269 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.806550  3963 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.879092  4331 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.915441  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:15.915468  2269 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:15.931454  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.921025  1714 raft_consensus.cc:2393] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:15.921902  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:15.921538  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:15.920275  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:15.900225  4261 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:15.914381  4399 log.cc:927] Time spent T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:02:15.939808  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:15.924216  4315 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.941107  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:15.936141  4261 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.942126  4261 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:15.942893  3963 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:15.942998  4315 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:15.934088  4415 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:02:15.938015  4339 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.889783  4346 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:15.926503  4397 log.cc:927] Time spent T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 13:02:15.952634  1560 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:15.952272  4412 consensus_queue.cc:237] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:15.953271  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.953604  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.954412  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.954452  1560 raft_consensus.cc:2393] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:15.956203  4331 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:15.956353  4393 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:15.956768  4331 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.957274  4331 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.962844  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.964382  1561 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:02:15.965178  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 44993.2us
I20260709 13:02:15.894435  4204 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:15.967428  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:15.972896  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.923032  4256 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:02:15.965830  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:15.975252  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:15.969393  1713 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:15.975904  2301 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:15.976017  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:15.979152  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.981208  2303 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:15.980108  1561 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:15.982242  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.982895  1561 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:15.985491  1714 raft_consensus.cc:1275] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:02:15.986009  1714 pending_rounds.cc:85] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 1
I20260709 13:02:15.986224  1714 pending_rounds.cc:107] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:02:15.986439  1714 raft_consensus.cc:2889] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:15.966173  4175 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:15.988300  2296 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:15.990609  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:15.952262  4346 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:15.975472  4415 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:15.987586  4212 consensus_meta.cc:287] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.545s	user 0.013s	sys 0.000s
W20260709 13:02:15.978317  4407 log.cc:927] Time spent T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 13:02:15.964964  4333 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.984169  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:15.992049  1718 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:15.995265  1709 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:15.977383  4392 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:02:16.138180  1563 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: []
W20260709 13:02:16.412223  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 268052us
W20260709 13:02:16.092608  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 418645us
I20260709 13:02:16.605546  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:16.047055  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.606271  2302 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:16.606372  1609 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:16.275226  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.030207  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:16.608331  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 517) took 1749 ms. Trace:
I20260709 13:02:16.609170  1561 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:16.055056  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1770) took 1428 ms. Trace:
I20260709 13:02:16.657974  1644 rpcz_store.cc:276] 0709 13:02:14.626662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.315189 (+688527us) service_pool.cc:224] Handling call
0709 13:02:15.315477 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.315842 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.315885 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.316601 (+   716us) log.cc:844] Serialized 26 byte log entry
0709 13:02:15.506691 (+190090us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:15.506770 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:15.506882 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.054784 (+547902us) raft_consensus.cc:1685] finished
0709 13:02:16.054804 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.054974 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":3780,"thread_start_us":170789,"threads_started":1,"wal-append.queue_time_us":255682}
W20260709 13:02:16.021723  4414 log.cc:927] Time spent T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:02:16.370839  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:16.406126  4421 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:02:16.609047  1637 rpcz_store.cc:276] 0709 13:02:14.858745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.713061 (+854316us) service_pool.cc:224] Handling call
0709 13:02:15.713904 (+   843us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:15.789120 (+ 75216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.789168 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:15.789841 (+   673us) log.cc:844] Serialized 52 byte log entry
0709 13:02:15.904263 (+114422us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:16.272241 (+367978us) log.cc:844] Serialized 14 byte log entry
0709 13:02:16.608012 (+335771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.608093 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.608133 (+    40us) raft_consensus.cc:1685] finished
0709 13:02:16.608145 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.608277 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33397,"raft.queue_time_us":25468,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":7952,"thread_start_us":379305,"threads_started":2,"wal-append.queue_time_us":127645}
I20260709 13:02:15.978596  4296 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:16.095292  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:16.655308  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.001s	sys 0.022s
W20260709 13:02:16.031602  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.681983  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.092865  4204 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:16.662940  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:16.268620  4220 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.029388  4269 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:16.039011  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:16.590837  4182 consensus_meta.cc:287] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.870s	user 0.010s	sys 0.001s
I20260709 13:02:15.990717  4332 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:16.080092  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.690198  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:16.139480  4291 consensus_meta.cc:287] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.951s	user 0.009s	sys 0.002s
W20260709 13:02:16.055051  4048 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.716s	user 0.009s	sys 0.002s
W20260709 13:02:16.289959  4395 consensus_meta.cc:287] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.620s	user 0.005s	sys 0.005s
I20260709 13:02:15.995838  4314 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:16.680392  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
W20260709 13:02:16.682507  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.660880  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:16.701366  1563 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:15.991501  4175 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:16.683434  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.001s	sys 0.022s
I20260709 13:02:16.137100  4408 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:16.105198  4331 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:16.687872  1561 raft_consensus.cc:2368] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6.
I20260709 13:02:16.288978  4420 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.437s
I20260709 13:02:16.690148  4392 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:16.657047  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 858312us
I20260709 13:02:16.703121  4427 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.715895  2268 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:16.242563  4261 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:16.245970  4393 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.480239  4400 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:16.337518  4401 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:16.705097  4175 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:16.674425  1643 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
W20260709 13:02:16.676196  3963 consensus_meta.cc:287] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.733s	user 0.008s	sys 0.004s
W20260709 13:02:16.691828  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 596604us
I20260709 13:02:16.693811  4332 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:16.693629  4296 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:16.696986  4433 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.687052  4339 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:16.703569  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 771777us
I20260709 13:02:16.760530  1563 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:02:16.705669  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 99420.9us
I20260709 13:02:16.740926  4420 consensus_queue.cc:1243] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:16.705523  4048 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:16.288080  4206 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.427s
I20260709 13:02:16.762725  4420 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:16.763598  4420 raft_consensus.cc:3055] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:16.706362  1568 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:16.738343  4261 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:16.318208  4410 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:16.766824  4296 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:16.768079  4410 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:16.020848  4419 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.679s
I20260709 13:02:16.714439  4408 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:16.734977  4094 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:16.700143  4220 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:16.662825  4333 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:16.739521  1567 raft_consensus.cc:2368] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6.
I20260709 13:02:16.738240  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:16.092186  2299 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:16.703437  4314 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.757313  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:16.763769  4048 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:16.763991  4420 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:16.771152  1566 raft_consensus.cc:2393] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 6: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:16.657040  4212 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:16.727033  1562 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:16.703759  4291 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:16.775918  1609 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:16.809625  4291 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:16.754228  4393 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:16.780795  1717 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.026s	sys 0.009s
W20260709 13:02:16.811708  1717 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.026s	sys 0.009s
I20260709 13:02:16.812724  4393 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:16.812825  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 533) took 1042 ms. Trace:
I20260709 13:02:16.813514  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 858) took 1346 ms. Trace:
W20260709 13:02:16.798300  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 60254.3us
W20260709 13:02:16.786454  4124 consensus_meta.cc:287] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.888s	user 0.012s	sys 0.001s
I20260709 13:02:16.770694  4332 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:16.805558  4408 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:16.815212  1561 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:02:16.759593  4400 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:16.815822  2268 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:16.793277  4314 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:16.865995  1711 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:16.815871  1718 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:16.816430  1716 raft_consensus.cc:606] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:16.866871  1716 raft_consensus.cc:614] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:16.813401  1717 rpcz_store.cc:276] 0709 13:02:15.770225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.771764 (+  1539us) service_pool.cc:224] Handling call
0709 13:02:15.785298 (+ 13534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.785707 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.785746 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.786272 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.205234 (+418962us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:16.812180 (+606946us) log.cc:844] Serialized 14 byte log entry
0709 13:02:16.812412 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.812523 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.812575 (+    52us) raft_consensus.cc:1685] finished
0709 13:02:16.812595 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.812754 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4237,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":5392,"spinlock_wait_cycles":528768,"thread_start_us":961955,"threads_started":2,"wal-append.queue_time_us":898431}
I20260709 13:02:16.868230  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:16.818261  1712 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:16.843624  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:16.845840  1639 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:16.813984  2299 rpcz_store.cc:276] 0709 13:02:15.454247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.471632 (+ 17385us) service_pool.cc:224] Handling call
0709 13:02:16.800534 (+1328902us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:16.778436  4094 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:16.754544  4427 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:16.864099  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 88482.5us
W20260709 13:02:16.848457  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 156888us
I20260709 13:02:16.810518  4291 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:02:16.841272  3963 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:16.770659  4401 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:16.801759  4392 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:16.886361  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.778113  4339 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:16.885694  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:16.790166  4206 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.079s
W20260709 13:02:16.890542  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 74804.7us
I20260709 13:02:16.886466  1609 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:16.870775  4345 consensus_meta.cc:287] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.161s	user 0.013s	sys 0.000s
I20260709 13:02:16.773622  4433 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:16.893623  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:16.895905  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:16.848977  4448 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:16.794559  4412 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
W20260709 13:02:16.912719  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 115988us
W20260709 13:02:16.913311  1710 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.018s	sys 0.019s
I20260709 13:02:16.913853  1683 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:16.914132  1710 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.018s	sys 0.020s
W20260709 13:02:16.914978  1569 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.004s	sys 0.011s
W20260709 13:02:16.915372  1569 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.004s	sys 0.011s
I20260709 13:02:16.862238  4220 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:16.856509  4415 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:16.899564  1717 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:16.918766  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 532) took 1148 ms. Trace:
I20260709 13:02:16.919176  1710 rpcz_store.cc:276] 0709 13:02:15.769965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.771269 (+  1304us) service_pool.cc:224] Handling call
0709 13:02:15.771552 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.785484 (+ 13932us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.785526 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.795199 (+  9673us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.298125 (+502926us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:16.915549 (+617424us) log.cc:844] Serialized 14 byte log entry
0709 13:02:16.916323 (+   774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.916582 (+   259us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.916703 (+   121us) raft_consensus.cc:1685] finished
0709 13:02:16.916759 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.918708 (+  1949us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38324,"raft.queue_time_us":14717,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":12194,"thread_start_us":965901,"threads_started":2,"wal-append.queue_time_us":885336}
I20260709 13:02:16.815596  4204 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:02:16.885136  4094 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:16.923666  1683 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:16.914440  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 530) took 1145 ms. Trace:
I20260709 13:02:16.914744  4182 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:16.924561  1711 rpcz_store.cc:276] 0709 13:02:15.769435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.770653 (+  1218us) service_pool.cc:224] Handling call
0709 13:02:16.914216 (+1143563us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54001c1954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:16.914357 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2486423808}
I20260709 13:02:16.939019  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:16.940059  1711 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:16.849491  4124 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:16.942238  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:16.958077  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:16.945554  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:16.890698  4392 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.870078  4331 consensus_queue.cc:1059] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:02:16.952783  1637 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:16.960978  1711 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:16.961912  1717 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:16.904838  4345 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:16.964006  4392 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:16.896585  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 138051us
W20260709 13:02:16.964990  4422 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:02:16.965792  4428 log.cc:927] Time spent T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:02:16.976727  1717 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:16.981339  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 526) took 1539 ms. Trace:
I20260709 13:02:16.981760  1641 rpcz_store.cc:276] 0709 13:02:15.441512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.749432 (+307920us) service_pool.cc:224] Handling call
0709 13:02:15.749780 (+   348us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:15.783681 (+ 33901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.783710 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:15.784259 (+   549us) log.cc:844] Serialized 52 byte log entry
0709 13:02:16.249125 (+464866us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:16.249167 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.249241 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.981130 (+731889us) raft_consensus.cc:1685] finished
0709 13:02:16.981150 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.981286 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33165,"raft.queue_time_us":33291,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":18635,"thread_start_us":432113,"threads_started":1,"wal-append.queue_time_us":986215}
I20260709 13:02:16.983385  4048 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.983934  4048 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:16.986974  4048 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:16.979919  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 534) took 1193 ms. Trace:
I20260709 13:02:16.988389  1715 rpcz_store.cc:276] 0709 13:02:15.786576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.786826 (+   250us) service_pool.cc:224] Handling call
0709 13:02:15.787197 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.790121 (+  2924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.790156 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.790571 (+   415us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.605718 (+815147us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:16.605766 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.605832 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.979694 (+373862us) raft_consensus.cc:1685] finished
0709 13:02:16.979719 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:16.979860 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2543,"raft.queue_time_us":3232,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":10902,"thread_start_us":312087,"threads_started":1,"wal-append.queue_time_us":968325}
I20260709 13:02:16.966248  4446 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:16.993854  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.943018  4393 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:16.936898  4182 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:16.999817  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 113433us
I20260709 13:02:17.001984  1710 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:17.003532  1638 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:17.011235  1609 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:16.975713  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 79290.6us
I20260709 13:02:17.017264  1533 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:17.008958  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 165474us
I20260709 13:02:17.029644  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:17.031028  1641 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:17.034327  4440 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:02:17.015694  4182 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:17.039295  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 1922) took 1051 ms. Trace:
I20260709 13:02:16.917793  4415 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:17.040051  4436 log.cc:927] Time spent T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:02:16.905071  4420 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.043282  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2254) took 1224 ms. Trace:
I20260709 13:02:17.043196  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:17.044224  1710 raft_consensus.cc:2368] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6.
I20260709 13:02:17.044554  4457 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.013145  4452 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:17.045516  4452 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:17.045753  4460 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:17.046298  4345 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:17.024410  4048 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:17.047324  1560 raft_consensus.cc:3055] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:17.047825  1560 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:17.047789  4048 raft_consensus.cc:697] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:17.047909  4433 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:17.039815  1714 rpcz_store.cc:276] 0709 13:02:15.987954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.000689 (+ 12735us) service_pool.cc:224] Handling call
0709 13:02:16.001045 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.001273 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:16.001303 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:16.019450 (+ 18147us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.742510 (+723060us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:16.742558 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.742789 (+   231us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.039009 (+296220us) raft_consensus.cc:1685] finished
0709 13:02:17.039037 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.039223 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":138182,"threads_started":1,"wal-append.queue_time_us":840343}
I20260709 13:02:17.017545  4455 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:17.068918  1565 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:17.069882  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2253) took 1259 ms. Trace:
I20260709 13:02:17.070379  1565 rpcz_store.cc:276] 0709 13:02:15.810945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.826178 (+ 15233us) service_pool.cc:224] Handling call
0709 13:02:17.068175 (+1241997us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:02:17.068464 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.069516 (+  1052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.069803 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6212352}
I20260709 13:02:17.040182  4427 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:17.043838  1564 rpcz_store.cc:276] 0709 13:02:15.819310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.823973 (+  4663us) service_pool.cc:224] Handling call
0709 13:02:16.085445 (+261472us) spinlock_profiling.cc:243] Waited 6.07 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:02:16.085669 (+   224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.086053 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:16.086085 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:16.086604 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.560614 (+474010us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:16.847905 (+287291us) log.cc:844] Serialized 14 byte log entry
0709 13:02:16.848248 (+   343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.848434 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.042974 (+194540us) raft_consensus.cc:1685] finished
0709 13:02:17.043076 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.043206 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":174458,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":8148,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":15800,"spinlock_wait_cycles":13957888,"thread_start_us":661311,"threads_started":2,"wal-append.queue_time_us":784485}
I20260709 13:02:17.021199  4296 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:17.046303  4460 raft_consensus.cc:697] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:17.082159  4339 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:17.043855  4212 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:17.078615  4403 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:17.094120  4403 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:17.085500  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:17.095009  1710 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
W20260709 13:02:17.095618  4416 log.cc:927] Time spent T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 1.235s	user 0.017s	sys 0.004s
I20260709 13:02:17.066736  4455 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.099704  4455 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:17.099865  4124 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:17.101547  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:17.102180  1639 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:17.102591  1636 raft_consensus.cc:2410] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:17.103386  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:17.104224  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:17.111901  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:17.113204  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:17.089522  1641 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:17.087196  4427 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.113698  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:17.121451  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:17.122138  1635 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:17.115960  1636 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:17.123924  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:17.117122  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:17.118386  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:17.125573  1710 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:17.126056  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:17.132148  1641 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:17.132745  1636 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:17.045143  4457 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:17.138623  2303 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:17.149194  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:17.150202  2304 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:17.087679  1560 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.154060  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:17.154356  1560 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:17.155447  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:17.046594  4452 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.156281  1710 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:17.159368  4048 consensus_queue.cc:237] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:17.163600  4460 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.164779  4175 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.166105  2305 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:17.167563  4212 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:17.171321  4332 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.173506  1643 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:17.177201  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:17.187855  2304 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
W20260709 13:02:17.220597  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.223721  1635 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
W20260709 13:02:17.224450  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.235697  1560 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:02:17.236352  1560 pending_rounds.cc:85] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 5
I20260709 13:02:17.236693  1560 pending_rounds.cc:107] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:02:17.237061  1560 raft_consensus.cc:2889] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:17.237958  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2263) took 1328 ms. Trace:
I20260709 13:02:17.198024  1683 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:17.254390  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:17.238492  1560 rpcz_store.cc:276] 0709 13:02:15.909390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.956604 (+ 47214us) service_pool.cc:224] Handling call
0709 13:02:15.956914 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.047219 (+1090305us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b5400046620. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c18dc0 00007fd7a1ca31c0 00007fd7a1ca4231 00007fd7a1ca0d8f 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c
0709 13:02:17.237636 (+190417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.237886 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47241472}
I20260709 13:02:17.271809  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:17.162086  4392 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.153412  3963 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:17.174750  4332 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:17.285706  4457 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:17.291814  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.297477  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:17.269740  4340 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:02:17.284789  4430 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.305419  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2258) took 1424 ms. Trace:
I20260709 13:02:17.316802  1567 rpcz_store.cc:276] 0709 13:02:15.881050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.940861 (+ 59811us) service_pool.cc:224] Handling call
0709 13:02:17.293439 (+1352578us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b54000a7dd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:17.305338 (+ 11899us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1834361088}
W20260709 13:02:17.316835  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 186776us
I20260709 13:02:17.327757  1715 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:02:17.284406  4187 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:02:17.307317  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 278543us
W20260709 13:02:17.307672  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.153215  4446 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:17.239899  4212 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:02:17.137136  4427 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.293054  4182 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.261404  1565 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
W20260709 13:02:17.340622  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:17.652302  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.653266  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:17.559558  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.548873  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2255) took 1700 ms. Trace:
I20260709 13:02:17.654217  1564 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:17.329699  4430 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.660388  1569 rpcz_store.cc:276] 0709 13:02:15.848629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.860146 (+ 11517us) service_pool.cc:224] Handling call
0709 13:02:15.892138 (+ 31992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:15.927429 (+ 35291us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:16.207887 (+280458us) log.cc:844] Serialized 14 byte log entry
0709 13:02:16.915760 (+707873us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:16.915813 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:16.916814 (+  1001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.548570 (+631756us) spinlock_profiling.cc:243] Waited 35 ms on lock 0x7b5400270454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:17.548617 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.548782 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":775762,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":977,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":70,"spinlock_wait_cycles":80525568,"thread_start_us":950983,"threads_started":2,"wal-append.queue_time_us":871292}
I20260709 13:02:17.239369  4331 raft_consensus.cc:3055] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:02:17.654260  4094 consensus_meta.cc:287] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.695s	user 0.010s	sys 0.000s
I20260709 13:02:17.656642  4427 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:17.340858  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:17.329185  4187 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:17.342495  4314 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:17.626013  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 428114us
W20260709 13:02:17.661916  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.685009  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:17.690460  1710 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
W20260709 13:02:17.625808  4393 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.000s	sys 0.019s
W20260709 13:02:17.694309  4393 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.000s	sys 0.020s
I20260709 13:02:17.694674  4393 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:17.696022  4175 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:17.695935  4393 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.687819  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:17.700198  1560 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:17.691076  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.693763  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:17.709062  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:17.694070  4187 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.715102  1711 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 7. Ops: []
I20260709 13:02:17.715765  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:17.716426  1710 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:17.671976  4331 raft_consensus.cc:740] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:17.726446  1561 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
W20260709 13:02:17.730516  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 400982us
W20260709 13:02:17.718340  4400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.821s	user 0.000s	sys 0.013s
I20260709 13:02:17.580134  4220 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.165s
W20260709 13:02:17.758015  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 101501us
I20260709 13:02:17.671820  4094 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.689297  4427 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:02:17.766578  4467 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.757682  4476 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
W20260709 13:02:17.756924  4400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.000s	sys 0.014s
I20260709 13:02:17.789752  1560 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:17.790460  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:17.790843  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:17.791227  1563 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:17.792486  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:17.795065  2299 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:17.686527  3963 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.806319  2298 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:02:17.805208  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:17.804155  1561 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:17.807529  2300 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:17.806314  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 863) took 1840 ms. Trace:
I20260709 13:02:17.807749  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:17.813136  2298 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:17.814061  2298 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:17.809376  2297 rpcz_store.cc:276] 0709 13:02:15.965890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.967140 (+  1250us) service_pool.cc:224] Handling call
0709 13:02:15.980831 (+ 13691us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:15.999209 (+ 18378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.999258 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:17.170462 (+1171204us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b4c0036dd80. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ff6da20 00007fd79ff7ea29 00007fd79ff44644 00007fd79ff6bf2d 00007fd79ff6bb93 00007fd7a1c9c18b 00007fd7a1c9ccb5 00007fd7a1c9e004 00007fd7a1c9d729 00007fd7a1ca2d96 00007fd7a1c97adb
0709 13:02:17.171564 (+  1102us) log.cc:844] Serialized 590 byte log entry
0709 13:02:17.335784 (+164220us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:17.335843 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.335943 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:17.789607 (+453664us) raft_consensus.cc:1685] finished
0709 13:02:17.789625 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.806245 (+ 16620us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":397134,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":59563,"spinlock_wait_cycles":94205184,"thread_start_us":163682,"threads_started":1,"wal-append.queue_time_us":597385}
I20260709 13:02:17.757583  4466 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.817122  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:17.817408  4466 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.818084  1561 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:17.820289  4466 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:17.823360  1532 leader_election.cc:336] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.708s (SENT)
I20260709 13:02:17.823971  1532 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:17.813621  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:17.826076  2300 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:17.826944  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:17.769241  4465 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.827468  2300 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEARNER]: Advancing to term 6
I20260709 13:02:17.755832  4475 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:17.761438  4147 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:17.835996  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 75707.4us
I20260709 13:02:17.789772  1562 raft_consensus.cc:1275] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:02:17.845405  1563 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:17.812260  4456 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.514s	user 0.001s	sys 0.000s
I20260709 13:02:17.849182  2297 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:17.794675  4400 raft_consensus.cc:697] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:17.853150  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.007s	sys 0.023s
W20260709 13:02:17.854068  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.007s	sys 0.023s
I20260709 13:02:17.812680  4392 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:17.769568  4175 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
W20260709 13:02:17.854866  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 832984us
I20260709 13:02:17.872550  1714 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:17.869305  2268 connection.cc:256] RPC call timeout handler was delayed by -2.0708758050000142s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 13:02:17.876157  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:17.884388  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:17.885236  1563 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:17.863775  4475 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:17.887807  1562 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:17.889617  1562 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:17.899259  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:17.899996  1563 raft_consensus.cc:2376] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6. Re-sending same reply.
W20260709 13:02:17.900125  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 76803.5us
I20260709 13:02:17.842943  4465 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.905221  2300 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEARNER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:17.810545  4467 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.844071  4147 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:17.910066  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1680) took 1908 ms (client timeout 2089 ms). Trace:
I20260709 13:02:17.913738  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:17.913862  1718 rpcz_store.cc:269] 0709 13:02:16.001723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.660599 (+658876us) service_pool.cc:224] Handling call
0709 13:02:17.909846 (+1249247us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b5400474854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:17.909995 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287284992}
I20260709 13:02:17.914564  2300 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:17.812439  4331 consensus_queue.cc:260] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.875001  4468 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:02:17.926062  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:17.926978  1532 leader_election.cc:336] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.931s (SENT)
I20260709 13:02:17.927266  2300 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:17.928634  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:17.927964  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:17.929842  1563 raft_consensus.cc:2376] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5. Re-sending same reply.
I20260709 13:02:17.763962  4446 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:17.886148  4481 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:02:17.940341  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 77046.9us
I20260709 13:02:17.875895  4463 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
W20260709 13:02:17.947016  1713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 529) took 2162 ms (client timeout 1921 ms). Trace:
I20260709 13:02:17.948189  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:17.896202  4450 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:17.950814  1562 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 3.
I20260709 13:02:17.951911  1718 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:17.946610  1561 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
W20260709 13:02:17.953541  1532 leader_election.cc:336] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.848s (SENT)
I20260709 13:02:17.954259  1532 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:17.950309  1713 rpcz_store.cc:269] 0709 13:02:15.769158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.770413 (+  1255us) service_pool.cc:224] Handling call
0709 13:02:17.931644 (+2161231us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b54003daed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:17.931751 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9316224}
I20260709 13:02:17.960147  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:17.960750  1713 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:17.982795  1563 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:17.971985  2300 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:17.969529  4314 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:17.943572  4400 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:17.982853  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.944342  4331 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:17.945897  4182 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:17.932137  4048 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:17.932646  4430 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:17.902184  4471 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
W20260709 13:02:18.001641  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 119706us
I20260709 13:02:17.992249  4481 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:18.006481  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:18.010218  2268 connection.cc:256] RPC call timeout handler was delayed by -1.7333758049999801s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19210000000000002s for a response
I20260709 13:02:18.012691  1718 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:18.011530  2268 connection.cc:256] RPC call timeout handler was delayed by -1.6739758049999978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19870000000000002s for a response
W20260709 13:02:18.013839  2268 connection.cc:256] RPC call timeout handler was delayed by -1.521487852000007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
W20260709 13:02:18.020087  2268 leader_election.cc:336] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.528s (SENT)
I20260709 13:02:18.020826  2268 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:18.028054  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:18.031389  4417 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 1.894s	user 0.004s	sys 0.024s
I20260709 13:02:18.036491  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 865) took 2070 ms. Trace:
I20260709 13:02:18.037024  2305 rpcz_store.cc:276] 0709 13:02:15.966445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.976184 (+  9739us) service_pool.cc:224] Handling call
0709 13:02:15.976526 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.166520 (+1189994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.036228 (+869708us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400168354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:18.036427 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2735498624}
W20260709 13:02:18.016484  4473 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.219s	user 0.001s	sys 0.001s
I20260709 13:02:18.011961  4094 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:18.044799  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:18.034723  4401 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:18.048420  4489 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:18.049417  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1690) took 1150 ms. Trace:
I20260709 13:02:17.986959  4450 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:18.056042  4467 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:18.000495  4475 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:18.064066  4461 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:18.075480  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 55452.5us
I20260709 13:02:18.117015  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 516) took 3306 ms. Trace:
I20260709 13:02:18.025663  4491 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:18.083222  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:18.121201  1562 raft_consensus.cc:2376] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7. Re-sending same reply.
I20260709 13:02:18.085858  4489 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:18.085491  4403 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:18.088841  1569 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:02:18.112195  1718 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:18.080199  1717 rpcz_store.cc:276] 0709 13:02:16.899083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.981935 (+ 82852us) service_pool.cc:224] Handling call
0709 13:02:16.982824 (+   889us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:17.308663 (+325839us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.308759 (+    96us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:17.327945 (+ 19186us) log.cc:844] Serialized 52 byte log entry
0709 13:02:17.792139 (+464194us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:17.891650 (+ 99511us) log.cc:844] Serialized 14 byte log entry
0709 13:02:17.892968 (+  1318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.893502 (+   534us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.049168 (+155666us) raft_consensus.cc:1685] finished
0709 13:02:18.049194 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.049344 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15695,"prepare.queue_time_us":159575,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":125092,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":12187,"thread_start_us":848920,"threads_started":3,"wal-append.queue_time_us":614585}
W20260709 13:02:18.049417  4484 log.cc:927] Time spent T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:02:18.127527  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:18.117456  1640 rpcz_store.cc:276] 0709 13:02:14.810881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.703795 (+892914us) service_pool.cc:224] Handling call
0709 13:02:15.704272 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:15.897373 (+193101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:15.897405 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:15.922483 (+ 25078us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.292940 (+370457us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:16.293021 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.110721 (+1817700us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400324bd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:18.110775 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.110830 (+    55us) raft_consensus.cc:1685] finished
0709 13:02:18.110842 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.116950 (+  6108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3342,"raft.queue_time_us":68280,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":2031,"spinlock_wait_cycles":435619712,"thread_start_us":322114,"threads_started":1,"wal-append.queue_time_us":803145,"wal-append.run_cpu_time_us":1562,"wal-append.run_wall_time_us":1002043}
W20260709 13:02:18.095034  4483 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:02:18.128476  1562 raft_consensus.cc:2376] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6. Re-sending same reply.
I20260709 13:02:18.147701  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:18.148756  1562 raft_consensus.cc:2410] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:18.124166  4489 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:18.041190  4048 consensus_queue.cc:1059] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 13:02:18.111599  4460 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:18.152415  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:18.153698  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 1930) took 1043 ms. Trace:
W20260709 13:02:18.120707  4454 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 1.074s	user 0.003s	sys 0.018s
W20260709 13:02:18.145571  4455 consensus_meta.cc:287] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.045s	user 0.012s	sys 0.002s
W20260709 13:02:18.163923  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 176678us
I20260709 13:02:18.166115  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:18.167784  1569 raft_consensus.cc:2376] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7. Re-sending same reply.
I20260709 13:02:18.168238  1562 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:18.166049  1718 rpcz_store.cc:276] 0709 13:02:17.110144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.045338 (+935194us) service_pool.cc:224] Handling call
0709 13:02:18.110841 (+ 65503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.127463 (+ 16622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.153633 (+ 26170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.156343  4403 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:18.179898  1534 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:18.173059  4485 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.172205  4455 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:18.215380  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 97183.8us
W20260709 13:02:18.216192  2268 leader_election.cc:336] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.594s (SENT)
W20260709 13:02:18.217545  2268 leader_election.cc:336] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.778s (SENT)
I20260709 13:02:18.220322  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 1923) took 2232 ms. Trace:
I20260709 13:02:18.223395  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:18.224215  2298 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:18.185263  4048 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:18.226285  1712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 527) took 2457 ms (client timeout 1987 ms). Trace:
I20260709 13:02:18.226485  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:18.226764  2301 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:02:18.227126  1569 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:18.224681  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 870) took 1087 ms. Trace:
I20260709 13:02:18.220737  1709 rpcz_store.cc:276] 0709 13:02:15.988204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.679124 (+690920us) service_pool.cc:224] Handling call
0709 13:02:16.793632 (+114508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.794101 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:16.794148 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:16.842232 (+ 48084us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.974145 (+131913us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:16.974194 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.974273 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.092253 (+1117980us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:18.216924 (+124671us) raft_consensus.cc:1685] finished
0709 13:02:18.216947 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.220246 (+  3299us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":42023,"spinlock_wait_cycles":44429568,"thread_start_us":83807,"threads_started":1,"wal-append.queue_time_us":156505}
I20260709 13:02:18.227628  2296 rpcz_store.cc:276] 0709 13:02:17.137302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.137444 (+   142us) service_pool.cc:224] Handling call
0709 13:02:17.146726 (+  9282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.147236 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.147276 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.658334 (+511058us) log.cc:844] Serialized 26 byte log entry
0709 13:02:17.731838 (+ 73504us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:17.731885 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:17.731976 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.219208 (+487232us) raft_consensus.cc:1685] finished
0709 13:02:18.219254 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.224622 (+  5368us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3969,"raft.queue_time_us":610170,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":73053,"threads_started":1,"wal-append.queue_time_us":215617}
W20260709 13:02:18.226892  1712 rpcz_store.cc:269] 0709 13:02:15.768524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.769594 (+  1070us) service_pool.cc:224] Handling call
0709 13:02:18.226103 (+2456509us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b54004051d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:18.226217 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2130015616}
W20260709 13:02:18.230105  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:18.237504  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:18.238677  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.124557  4461 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:18.251574  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.252710  1569 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:18.185460  4489 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:18.266654  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:18.238320  4493 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:18.215381  4485 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:18.281857  4499 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:02:18.298293  4461 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:18.299818  4501 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:02:18.302028  4455 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:18.302961  4496 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:02:18.314417  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 1925) took 1205 ms. Trace:
I20260709 13:02:18.317204  1710 rpcz_store.cc:276] 0709 13:02:17.108681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.717040 (+608359us) service_pool.cc:224] Handling call
0709 13:02:17.728487 (+ 11447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.746829 (+ 18342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.746880 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.747426 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.157908 (+410482us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:18.157940 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.158004 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.314190 (+156186us) raft_consensus.cc:1685] finished
0709 13:02:18.314209 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.314353 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35578,"raft.queue_time_us":128165,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":9959,"thread_start_us":366075,"threads_started":1,"wal-append.queue_time_us":424997}
I20260709 13:02:18.325531  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:18.330395  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:38292 (request call id 28) took 2239 ms. Trace:
I20260709 13:02:18.279266  4493 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:18.332739  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:18.334417  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1692) took 1334 ms. Trace:
I20260709 13:02:18.336624  1715 rpcz_store.cc:276] 0709 13:02:16.999839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.999975 (+   136us) service_pool.cc:224] Handling call
0709 13:02:17.000295 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.328343 (+328048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.334151 (+1005808us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54003dc7d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:18.334357 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752622976}
I20260709 13:02:18.334702  4481 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:02:18.335978  1563 raft_consensus.cc:2376] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6. Re-sending same reply.
I20260709 13:02:18.335454  1716 rpcz_store.cc:276] 0709 13:02:16.091364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.649079 (+557715us) service_pool.cc:224] Handling call
0709 13:02:18.330187 (+1681108us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54001487d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c9b853 00007fd7a6c6960a 00007fd7a01ef7d4 00007fd7a01ef762 00007fd7a01ef691 00007fd7a01ef60c 00007fd7a01ee8c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:18.330328 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267550592}
I20260709 13:02:18.359576  2302 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 2. Current term is 3. Ops: []
I20260709 13:02:18.362797  4493 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:18.364502  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.364660  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.365185  1715 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:18.365918  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 669) took 1350 ms. Trace:
I20260709 13:02:18.366283  2299 rpcz_store.cc:276] 0709 13:02:17.015306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.017006 (+  1700us) service_pool.cc:224] Handling call
0709 13:02:17.063427 (+ 46421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.795555 (+732128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.365701 (+570146us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b5400650254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:18.365861 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1682243968}
I20260709 13:02:18.367419  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.368196  1709 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:18.368081  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:18.368882  1565 raft_consensus.cc:2410] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:18.369733  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.370168  1569 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:18.372337  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:18.372964  1563 raft_consensus.cc:2368] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6.
I20260709 13:02:18.374521  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.375126  1712 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:18.379355  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.380064  1710 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
W20260709 13:02:18.382112  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 461) took 2452 ms (client timeout 1528 ms). Trace:
W20260709 13:02:18.382663  1566 rpcz_store.cc:269] 0709 13:02:15.929670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.931212 (+  1542us) service_pool.cc:224] Handling call
0709 13:02:18.381880 (+2450668us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b54001ff254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:18.382027 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1929275648}
I20260709 13:02:18.389634  1712 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:18.381147   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:18.391075  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.391760  4048 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:18.397136  4485 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:18.396726  4048 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:18.410986  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.414340  4453 raft_consensus.cc:3055] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:18.414848  4453 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:18.420765  4453 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:18.430857  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 72542.2us
I20260709 13:02:18.432339  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.418845  4425 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.433676  1712 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:18.433933  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:18.435593  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:18.436687  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.437094  1710 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:18.438007  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:02:18.442500  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2249) took 2638 ms (client timeout 1718 ms). Trace:
W20260709 13:02:18.442984  1568 rpcz_store.cc:269] 0709 13:02:15.803806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:15.806803 (+  2997us) service_pool.cc:224] Handling call
0709 13:02:18.442325 (+2635522us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b54001ce654. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:18.442431 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2033909120}
I20260709 13:02:18.443526  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:18.443709  1712 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:18.444409  1640 raft_consensus.cc:2410] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:02:18.445923  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 529) took 1601 ms. Trace:
I20260709 13:02:18.446310  1637 rpcz_store.cc:276] 0709 13:02:16.844651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.844789 (+   138us) service_pool.cc:224] Handling call
0709 13:02:16.865069 (+ 20280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:16.970421 (+105352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.445697 (+1475276us) spinlock_profiling.cc:243] Waited 87.4 ms on lock 0x7b5400247f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:18.445868 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":200954368}
I20260709 13:02:18.447909  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.438620  4453 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:18.452471  1669 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:37201 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:37201 timed out after 15.000s (SENT)
I20260709 13:02:18.455978  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.493899  1712 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:18.463979  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.502060  1710 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:18.502684  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:18.470232  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.505600  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.506197  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.461282  1635 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:18.523758  1709 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:18.512575  4487 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:02:18.438444  4425 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:18.533123  4425 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:18.536298  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.536325  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.537228  1712 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:18.539681  1713 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:18.541320  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.541726  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.541934  1710 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:18.542263  1713 raft_consensus.cc:2376] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7. Re-sending same reply.
I20260709 13:02:18.543603  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.543759  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.544817  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.545883  1712 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:18.547734  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.548681  1718 raft_consensus.cc:2376] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5. Re-sending same reply.
W20260709 13:02:18.548931  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 112590us
W20260709 13:02:18.549749  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.700594  4469 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:18.712903  1681 connection.cc:256] RPC call timeout handler was delayed by -1.4686643799999501s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2008s for a response
W20260709 13:02:18.726495  1681 connection.cc:256] RPC call timeout handler was delayed by -1.3278261799999882s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23250000000000004s for a response
W20260709 13:02:18.726994  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.722582  4465 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:18.617023  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 531) took 1771 ms. Trace:
I20260709 13:02:18.560045  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.730588  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.731966  1718 raft_consensus.cc:2376] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5. Re-sending same reply.
I20260709 13:02:18.733063  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.733965  1718 raft_consensus.cc:2368] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6.
I20260709 13:02:18.553234  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.564659  4487 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:18.559862  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:18.575121  4471 consensus_queue.cc:1059] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:02:18.565891  4430 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.579370  4345 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:18.608022  4478 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.693s	user 0.003s	sys 0.000s
I20260709 13:02:18.540874  4333 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:18.714638   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:18.549607  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.716595  1467 catalog_manager.cc:5697] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 3 to 4, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 4 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:18.555610  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:18.584203  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.744102  1709 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:18.745795  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:18.746630  1532 leader_election.cc:336] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.633s (SENT)
I20260709 13:02:18.747021  1532 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:18.747983  1532 leader_election.cc:336] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.635s (SENT)
I20260709 13:02:18.748364  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:18.748653  4256 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:18.749068  4340 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:02:18.749387  1532 leader_election.cc:336] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.637s (SENT)
I20260709 13:02:18.737040  4469 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:18.750313  1532 leader_election.cc:336] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.664s (SENT)
I20260709 13:02:18.575862  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.649116  1712 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:18.751550  1717 raft_consensus.cc:2376] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7. Re-sending same reply.
I20260709 13:02:18.752043  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.752377  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.753113  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.753444  4469 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:18.753515  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.619975  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:18.754274  1717 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:18.754348  1567 raft_consensus.cc:2376] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6. Re-sending same reply.
W20260709 13:02:18.727591  1681 connection.cc:256] RPC call timeout handler was delayed by -1.1280261799999494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25470000000000004s for a response
W20260709 13:02:18.755117  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.756201  1709 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:18.555852  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:18.728968  1638 rpcz_store.cc:276] 0709 13:02:16.845250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.846237 (+   987us) service_pool.cc:224] Handling call
0709 13:02:16.846592 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.011320 (+164728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.616741 (+1605421us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54001e9954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:18.616958 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353517440}
I20260709 13:02:18.757612  2303 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:18.734766  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.738615  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.710744  4450 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:18.742832  1564 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:18.753661  1712 raft_consensus.cc:2376] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6. Re-sending same reply.
I20260709 13:02:18.725483  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:18.760028  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1886) took 1075 ms. Trace:
I20260709 13:02:18.760555  1567 rpcz_store.cc:276] 0709 13:02:17.685106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.619654 (+934548us) service_pool.cc:224] Handling call
0709 13:02:18.754905 (+135251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.761273  1710 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:18.757161  1681 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:18.761915  1718 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:18.763280  4212 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:18.764022  4491 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:18.764458  4491 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:18.765215  4491 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:18.766408  4491 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:18.772663  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.772854  1715 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:18.773456  1710 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:18.774262  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.775005  1715 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:18.775022  1717 raft_consensus.cc:2410] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:02:18.783304  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.783846  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.784075  1717 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:18.758940  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:18.788726  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.791268  2303 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:18.792579  1683 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:18.794062  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.795317  1718 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:18.771858  4450 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:18.807047  1715 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
W20260709 13:02:18.808847  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 51749.8us
I20260709 13:02:18.771669  4206 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:18.811878  4206 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:18.814170  4206 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:18.816996  4187 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:18.744087  4430 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:18.775759  4469 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:18.743961  4345 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:18.821209  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:18.809841  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:18.810252  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:18.829671  1638 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:18.831400  1710 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:18.832552  1681 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:18.834097  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:18.835037  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1792) took 1717 ms. Trace:
I20260709 13:02:18.835377  1643 rpcz_store.cc:276] 0709 13:02:17.145111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.808857 (+1663746us) service_pool.cc:224] Handling call
0709 13:02:18.834975 (+ 26118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:18.837920  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.847550  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:18.849239  2298 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:02:18.852489  1641 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:18.856339  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:18.857463  1643 raft_consensus.cc:2410] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:02:18.861858  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.831172  4469 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:18.872071  4469 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:18.876683  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.878242  4487 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:18.878228  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1895) took 1033 ms. Trace:
I20260709 13:02:18.879232  4487 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:18.879112  1560 rpcz_store.cc:276] 0709 13:02:17.844627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.844768 (+   141us) service_pool.cc:224] Handling call
0709 13:02:17.845106 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.845418 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.845449 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:18.634198 (+788749us) spinlock_profiling.cc:243] Waited 921 us on lock 0x7b2c00000658. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:02:18.634713 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.782712 (+147999us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:18.831786 (+ 49074us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.849096 (+ 17310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.849170 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.875974 (+ 26804us) spinlock_profiling.cc:243] Waited 8.09 ms on lock 0x7b1800304000. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:18.877884 (+  1910us) raft_consensus.cc:1685] finished
0709 13:02:18.877919 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.878146 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11025,"prepare.queue_time_us":48907,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":168686,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":17830,"spinlock_wait_cycles":31393024,"thread_start_us":116375,"threads_started":1,"wal-append.queue_time_us":189607}
I20260709 13:02:18.881973  4475 raft_consensus.cc:3055] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:18.882301  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.882436  4475 raft_consensus.cc:740] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:18.883167  1709 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:18.883590  2268 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:18.883469  4475 consensus_queue.cc:260] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:18.885229  4481 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:18.886543  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.887183  4481 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:18.888722  4430 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:18.884971  4475 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:18.860780  4530 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:18.891453  4530 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:18.892329  4530 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:18.893967  4345 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:18.894276  4256 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.895181  4256 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:18.897996  4256 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:18.899437  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.900933  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:18.901626  4420 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:18.903256  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:18.904366  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:18.905383  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:18.905929  1644 raft_consensus.cc:2376] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6. Re-sending same reply.
I20260709 13:02:18.906450  1638 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:18.907171  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:18.907791  1533 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:18.908104  2303 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:02:18.909204  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:18.909617  4340 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:18.910202  4340 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:18.885679  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:18.911084  1710 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:18.868566  4206 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.911684  4206 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:18.913662  4206 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:18.915040  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:18.915613  1715 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:18.901592  1639 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
W20260709 13:02:18.920666  1532 leader_election.cc:336] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.743s (SENT)
I20260709 13:02:18.926474  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:18.927111  2303 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:18.927300  2300 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:18.935083  1532 leader_election.cc:336] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.841s (SENT)
I20260709 13:02:18.940567  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:18.949080  2300 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:18.953114  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:18.953722  1607 connection.cc:256] RPC call timeout handler was delayed by -1.0036855290000517s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19770000000000001s for a response
I20260709 13:02:18.953166  1533 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:18.957396  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:18.956302  4425 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.952960  4345 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:02:18.958214  2303 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
W20260709 13:02:18.961153  1532 leader_election.cc:336] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.848s (SENT)
I20260709 13:02:18.975307  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:18.986071  2297 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:18.979102  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1682) took 2279 ms. Trace:
I20260709 13:02:18.987900  1711 rpcz_store.cc:276] 0709 13:02:16.699827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.973107 (+273280us) service_pool.cc:224] Handling call
0709 13:02:16.973426 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.715612 (+742186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.978840 (+1263228us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b540020ba54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:18.979039 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1705301248}
I20260709 13:02:18.983872  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:18.983583  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:18.995946  1533 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:19.002079  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1890) took 1241 ms. Trace:
I20260709 13:02:19.002686  1564 rpcz_store.cc:276] 0709 13:02:17.760953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.407341 (+646388us) service_pool.cc:224] Handling call
0709 13:02:18.407791 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:18.770833 (+363042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.001720 (+230887us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54001ff254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:19.002001 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":770041216}
I20260709 13:02:18.975462  4460 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:19.039812  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.055126  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.080700  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 288206us
I20260709 13:02:18.977697  4147 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:19.083513  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.089386  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.989624  4453 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.961236  4340 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:18.961910  4444 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.118s
I20260709 13:02:19.096551  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:19.017769  4505 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:18.961717  4425 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:19.011044  4048 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:19.011924  4450 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.099351  1636 raft_consensus.cc:2393] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:19.100443  4505 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.080837  4400 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:02:19.018891  4256 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:19.096408  4453 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:19.110519  4400 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:19.111135  4400 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:19.111609  4400 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:19.112169  4425 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:19.112744  4425 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:19.112872  4489 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:19.113520  1532 leader_election.cc:336] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.008s (SENT)
I20260709 13:02:19.113245  4425 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:18.971175  4512 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.122s
I20260709 13:02:18.982517  4514 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.113s
I20260709 13:02:19.116557  1682 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:18.961184  4517 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.140s
I20260709 13:02:19.093830  4212 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.018234  4452 consensus_queue.cc:1059] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 13:02:19.110021  4505 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:18.963256  4233 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.115877  4512 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:19.054695  4345 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:02:19.102841  4448 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:02:19.159060  4425 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:19.133008  4512 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:19.167109  4204 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:19.167795  4512 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.133181  4212 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.168704  4204 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:19.141891  4233 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:19.169389  4512 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:19.155501  4452 raft_consensus.cc:3055] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:19.145428  4533 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:19.156002  4531 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:19.120870  4450 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.170713  4212 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:19.254462  1467 catalog_manager.cc:5697] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 6 to 7, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 7 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:19.265587  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.279294  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 446669us
I20260709 13:02:19.280092  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:19.283434  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 1910) took 1237 ms. Trace:
I20260709 13:02:19.284072  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:19.174829  4452 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:19.178833  4204 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.208521  4505 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.213881  4531 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:19.179509  4533 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:19.376224  1467 catalog_manager.cc:5697] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:19.343981  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1694) took 2283 ms. Trace:
I20260709 13:02:19.379105  1714 rpcz_store.cc:276] 0709 13:02:17.060799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.060982 (+   183us) service_pool.cc:224] Handling call
0709 13:02:17.061392 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.873016 (+811624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.343345 (+1470329us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b54003395d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:19.343741 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1865135360}
I20260709 13:02:19.379948  1716 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:19.380640  4453 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:19.381402  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:19.381537  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 397618us
W20260709 13:02:19.384032  1532 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.089s (SENT)
I20260709 13:02:19.385111  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:19.385885  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:19.386291  1644 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:19.387040  1638 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:19.387691  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:19.381451  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:19.389353  1716 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:19.390334  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:19.392349  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:19.346877  1713 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:19.335561  1569 rpcz_store.cc:276] 0709 13:02:18.046246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.275401 (+229155us) service_pool.cc:224] Handling call
0709 13:02:19.282875 (+1007474us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b54003e7454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:19.283123 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100633472}
I20260709 13:02:19.353459  1715 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:19.353897  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:19.354431  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:19.378473  4505 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:19.342500  1636 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:19.387807  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:19.392747  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:19.391321  1467 catalog_manager.cc:5697] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:19.393394  4452 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.345440  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:19.344521  4206 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.378048  4533 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.396737  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:19.403650  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.400352  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:19.344005  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 872) took 2205 ms. Trace:
W20260709 13:02:19.386533  1532 leader_election.cc:336] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.145s (SENT)
I20260709 13:02:19.367096  4233 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:19.391270  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:19.416975  2301 rpcz_store.cc:276] 0709 13:02:17.137978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.138095 (+   117us) service_pool.cc:224] Handling call
0709 13:02:17.651801 (+513706us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.230576 (+578775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.342961 (+1112385us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b54004e5f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:19.343664 (+   703us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1321840000}
W20260709 13:02:19.439112  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 63864.4us
I20260709 13:02:19.440241  4452 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:19.442349  1681 leader_election.cc:400] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6.
I20260709 13:02:19.443491  1564 raft_consensus.cc:2410] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:19.443888  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.444238  4514 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.444914  1568 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:19.445235  4514 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.446555  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.446837  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.447824  1566 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:19.447956  1562 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:19.402386  4546 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:19.451512  4546 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.438823  4491 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:19.442762  1681 leader_election.cc:403] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 13:02:19.461867  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 55317.5us
I20260709 13:02:19.429571  1532 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:19.456219  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:19.444279  4531 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:02:19.472592  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:19.473968  1718 raft_consensus.cc:2410] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:02:19.474495  4514 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:19.475284  1717 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:19.475378  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:19.476373  1560 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:19.476497  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:19.478577  1717 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:19.484145  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:19.484575  4489 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.485292  4489 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.485795  4495 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:19.495005  4256 raft_consensus.cc:3055] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:19.486821  4430 consensus_queue.cc:1059] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:02:19.495586  4256 raft_consensus.cc:740] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:19.470160  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.497076  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:19.508077  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:19.492300  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.491673  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.499202  4533 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:19.487489  4469 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:19.499030  4430 raft_consensus.cc:3055] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:19.445384  4212 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:02:19.489001  4408 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:19.517282  4314 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:02:19.517884  1712 raft_consensus.cc:2393] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:19.516788  4469 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.523377  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:19.466306  1467 catalog_manager.cc:5697] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 3 to 4, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 4 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:19.515048  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 352232us
I20260709 13:02:19.493716  4206 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.533831  4530 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6.
I20260709 13:02:19.534399  1567 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 6: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:19.534883  1563 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:19.535758  4460 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:19.535633  4256 consensus_queue.cc:260] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.541805  4546 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:19.573904  4256 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:19.575831  4489 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:19.542869  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:19.578042  1561 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:19.579051  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.579357  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:19.579921  1561 raft_consensus.cc:2410] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:19.580899  1609 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:19.582293  1609 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:19.542183  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:19.544904  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:19.584830  1564 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:19.585878  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.547369  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.587571  1566 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:19.545864  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:19.548316  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.589202  1565 raft_consensus.cc:2376] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6. Re-sending same reply.
I20260709 13:02:19.589454  1568 raft_consensus.cc:2410] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:02:19.557211  4461 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:02:19.524250  4430 raft_consensus.cc:740] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:19.562994  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.563068  4469 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:19.563345  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 177039us
W20260709 13:02:19.563829  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.588517  1716 raft_consensus.cc:1240] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:19.563946  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:19.564050  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:19.555904  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.569370  4314 raft_consensus.cc:3055] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:19.569588  4475 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:19.570549  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.571774  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.571285  1467 catalog_manager.cc:5697] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 4, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 4 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:19.573916  4450 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:19.583633  1713 raft_consensus.cc:2376] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6. Re-sending same reply.
I20260709 13:02:19.588512  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.548682  2297 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:02:19.586854  1564 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 6: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:19.580919  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:19.583405  4296 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:19.568089  1710 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:19.596249  1563 raft_consensus.cc:2376] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6. Re-sending same reply.
W20260709 13:02:19.599031  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 84071.5us
I20260709 13:02:19.597317  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:19.606509  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:19.606230  1532 leader_election.cc:336] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.146s (SENT)
I20260709 13:02:19.609421  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.610232  1569 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:19.610416  1714 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
W20260709 13:02:19.610975  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 168836us
I20260709 13:02:19.614007  1532 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:19.616686  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:19.611549  4296 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:19.620741  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.622781  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.623221  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:19.623633  1567 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:19.623996  1566 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:19.624684  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.625798  1567 raft_consensus.cc:2410] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:19.624758  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.609668  4204 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:19.627624  1564 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 6: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:19.620834  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:19.629822  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:19.630311  1711 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:19.630496  1563 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:19.632200  4489 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:19.622067  4430 consensus_queue.cc:260] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.633544  4430 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:19.635809  1532 leader_election.cc:336] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.167s (SENT)
I20260709 13:02:19.636353  1532 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:19.620684  4296 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.638268  4546 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:19.638957  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:19.639927  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 186531us
I20260709 13:02:19.640065  4475 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:19.640232  1464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45150 (request call id 324) took 16972 ms (client timeout 15000 ms). Trace:
I20260709 13:02:19.640530  4475 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.641033  4475 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:19.642323  1718 raft_consensus.cc:2376] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6. Re-sending same reply.
I20260709 13:02:19.643306  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:19.644171  1569 raft_consensus.cc:2410] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:19.645044  4448 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:19.645385  1640 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:19.617915  1467 catalog_manager.cc:5697] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 1 to 4, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 4 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:19.620640  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.649648  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:19.650144  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:19.650830  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:19.621032  1715 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:19.652473  4487 raft_consensus.cc:3055] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:19.652971  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:19.653227  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.654170  1568 raft_consensus.cc:2410] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:19.654085  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:19.656664  1533 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:19.657344  1636 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
W20260709 13:02:19.657750  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.658223  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:19.658253  4491 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:19.657691  4514 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:19.659983  1533 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:19.660252  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:19.657737  1636 pending_rounds.cc:85] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 6
I20260709 13:02:19.661173  1640 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:19.684397  1567 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:02:19.665726  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 46815.1us
I20260709 13:02:19.684957  1638 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:19.595721  4314 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:19.639117  4546 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:19.622484  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.691226  1560 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
W20260709 13:02:19.686599  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.640753  1464 rpcz_store.cc:269] 0709 13:02:02.528943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.529299 (+   356us) service_pool.cc:224] Handling call
0709 13:02:04.081898 (+1552599us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b540013f954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0b090 00007fd7a1ca9f3a 00007fd7a7a601cc 00007fd7a7b5e37e 00007fd7a601f2a4 00007fd7a601f232 00007fd7a601f161 00007fd7a601f0dc 00007fd7a601e392 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:19.470200 (+15388302us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:11.168021 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:11.180630 (+ 12609us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:11.180695 (+    65us) write_op.cc:435] Acquired schema lock
0709 13:02:11.180716 (+    21us) tablet.cc:662] Decoding operations
0709 13:02:11.427098 (+246382us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:11.434908 (+  7810us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:11.444054 (+  9146us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:02:11.631145 (+187091us) tablet.cc:701] Row locks acquired
0709 13:02:11.631197 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:02:11.631396 (+   199us) write_op.cc:270] Start()
0709 13:02:11.631565 (+   169us) write_op.cc:276] Timestamp: P: 1783602131631311 usec, L: 0
0709 13:02:11.631588 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:02:11.632278 (+   690us) log.cc:844] Serialized 3674 byte log entry
0709 13:02:12.697122 (+1064844us) op_driver.cc:464] REPLICATION: finished
0709 13:02:12.847306 (+150184us) write_op.cc:301] APPLY: starting
0709 13:02:12.847443 (+   137us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:12.847769 (+   326us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:12.847947 (+   178us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:12.885408 (+ 37461us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:12.885780 (+   372us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:02:12.885825 (+    45us) write_op.cc:312] APPLY: finished
0709 13:02:12.891357 (+  5532us) log.cc:844] Serialized 71 byte log entry
0709 13:02:12.891634 (+   277us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:12.891813 (+   179us) write_op.cc:454] Released schema lock
0709 13:02:12.891905 (+    92us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:13.062452 (+170547us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b1800189160. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb4522 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:13.062611 (+   159us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:13.062651 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":213601,"reported_tablets":33,"spinlock_wait_cycles":42188160,"child_traces":[["op",{"apply.queue_time_us":138574,"apply.run_cpu_time_us":7596,"apply.run_wall_time_us":57689,"num_ops":9,"prepare.queue_time_us":189402,"prepare.run_cpu_time_us":36041,"prepare.run_wall_time_us":569803,"raft.queue_time_us":685504,"raft.run_cpu_time_us":26290,"raft.run_wall_time_us":389791,"replication_time_us":911528,"spinlock_wait_cycles":7765248,"thread_start_us":801515,"threads_started":4,"wal-append.queue_time_us":191841,"wal-append.run_cpu_time_us":15755,"wal-append.run_wall_time_us":2096055}]]}
I20260709 13:02:19.687598  1562 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
W20260709 13:02:19.637820  1532 leader_election.cc:336] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.227s (SENT)
I20260709 13:02:19.684338  1636 pending_rounds.cc:107] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:02:19.695910  1636 raft_consensus.cc:2889] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:19.696604  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 84741.3us
I20260709 13:02:19.700078  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.701594  2304 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:19.704461  2298 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:19.706122  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:19.705905  4314 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:19.706995  2296 raft_consensus.cc:2393] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:19.707484  4314 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:19.708993  4448 consensus_queue.cc:1059] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:19.710026  4448 raft_consensus.cc:3055] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:19.710376  4448 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:19.711170  4491 raft_consensus.cc:697] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:19.711388  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:19.715221  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.681818  4487 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:19.720363  1561 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:19.721984  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.692180  4505 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:19.724227  4487 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.725526  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:19.728807  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 69036.6us
I20260709 13:02:19.729444  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2258) took 1238 ms. Trace:
I20260709 13:02:19.729959  4206 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:19.729619  4233 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:19.731662  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.722314  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.689221  4489 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:02:19.741912  1563 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:19.711197  4415 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:19.751307  1567 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:19.752264  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:19.736235  1643 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:19.712558  4491 consensus_queue.cc:237] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.748334  1566 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:19.754613  1636 raft_consensus.cc:2410] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:19.755946  1562 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
W20260709 13:02:19.767218  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.736536  1637 rpcz_store.cc:276] 0709 13:02:18.448974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.449102 (+   128us) service_pool.cc:224] Handling call
0709 13:02:18.449607 (+   505us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:18.450031 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.450057 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:19.010041 (+559984us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b2c001080d8. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:02:19.120741 (+110700us) log.cc:844] Serialized 590 byte log entry
0709 13:02:19.510649 (+389908us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:19.557037 (+ 46388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.568813 (+ 11776us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.665640 (+ 96827us) raft_consensus.cc:1685] finished
0709 13:02:19.665666 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.687139 (+ 21473us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"spinlock_wait_cycles":10092928,"thread_start_us":67393,"threads_started":1,"wal-append.queue_time_us":312227}
I20260709 13:02:19.777752  1560 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
W20260709 13:02:19.778872  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 171026us
I20260709 13:02:19.698027  4461 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:02:19.783154  1561 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:19.755615  4415 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.795598  1566 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:19.796788  1563 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:19.804888  2296 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:19.809705  1562 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:19.796171  4415 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:19.809821  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:19.830152  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 192405us
I20260709 13:02:19.807729  4340 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:19.822422  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:19.835415  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:19.836009  1562 raft_consensus.cc:2376] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6. Re-sending same reply.
W20260709 13:02:19.837633  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 56930.6us
I20260709 13:02:19.842970  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:19.851235  1563 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:19.843683  1566 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:19.848551  1635 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:19.840077  4505 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.837641  4340 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:19.856082  2297 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:19.845191  4489 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:19.860693  4427 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:02:19.865461  4489 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.866350  4512 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:19.867054  4512 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:19.867846  1642 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:02:19.868232  1642 pending_rounds.cc:85] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 3
I20260709 13:02:19.855505  4460 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.869248  4460 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.872845  4460 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:19.873646  4147 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:02:19.875711  4557 log.cc:927] Time spent T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:02:19.877241  4446 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:19.879412  4487 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:19.882189  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:19.883248  4461 raft_consensus.cc:2955] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Committing config change with OpId 4.6: config changed from index 3 to 6, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:02:19.884670  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 311762us
I20260709 13:02:19.810524  4448 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:19.888464  4448 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:19.888922  4555 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:19.857416  4400 consensus_meta.cc:287] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.663s	user 0.007s	sys 0.017s
I20260709 13:02:19.890750  4400 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:19.891759  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 44214.8us
I20260709 13:02:19.893929  4400 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:19.894387  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:19.895162  4514 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:19.896569  1561 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:19.862234  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:19.868680  1642 pending_rounds.cc:107] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:02:19.907269  1642 raft_consensus.cc:2889] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:19.893990   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:19.909526  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 64630.2us
I20260709 13:02:19.913668  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:19.924131  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 131824us
I20260709 13:02:19.931221  1562 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:19.868263  4512 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:19.934439  1560 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:19.903908  4340 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:19.938589  1681 leader_election.cc:336] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.502s (SENT)
I20260709 13:02:19.943262  1562 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:02:19.946106  1566 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:02:19.946357  1563 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:02:19.948704  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.949757  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 112789us
I20260709 13:02:19.951992  4408 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:19.955104  4256 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:19.955780  4408 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.956037  4256 raft_consensus.cc:2764] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:02:19.956465  4408 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:19.957970  1561 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:19.947608  1681 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:19.961772  2268 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:19.920261  4546 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:02:19.967299  4546 consensus_queue.cc:1243] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:19.967541  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:19.968765  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:19.969897  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:19.915380  4505 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:19.970455  1636 raft_consensus.cc:2376] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6. Re-sending same reply.
I20260709 13:02:19.972111  4505 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:19.971941  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:19.974171  2269 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:19.975314  4403 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.976162  4403 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:19.977892  4446 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:19.959610  4475 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:19.981277  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 43803.8us
I20260709 13:02:19.982901  2268 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:20.005897  1561 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:19.958986  4460 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.978744  4446 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:20.013991  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 68329.2us
I20260709 13:02:20.015338  1635 raft_consensus.cc:2410] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:20.032963  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:20.039240  4466 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:20.041450  2304 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:20.042460  4475 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:20.045040  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.946007  4453 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
W20260709 13:02:20.048516  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 74446.2us
I20260709 13:02:20.056123  4489 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:20.058122  4403 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:20.059095  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 140226us
W20260709 13:02:20.059798  1681 leader_election.cc:336] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.616s (SENT)
I20260709 13:02:20.060164  1681 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:20.061553  1681 leader_election.cc:336] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.628s (SENT)
I20260709 13:02:20.061709  4204 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:20.062114  1681 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:20.063218  4468 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:20.064615  2302 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:20.067754  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:20.010669  4505 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.068480  4505 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:20.069437  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:20.070081  2302 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:20.039296  4538 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:20.071168  4538 raft_consensus.cc:697] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:20.072096  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:20.073145  2303 raft_consensus.cc:2393] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:20.073508  2296 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:20.072201  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:20.072151  4538 consensus_queue.cc:237] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:20.012300  4495 consensus_queue.cc:1059] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:02:20.077968  4495 raft_consensus.cc:3055] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:20.078348  4495 raft_consensus.cc:740] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:20.079137  4471 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:20.080178  4533 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.079321  4495 consensus_queue.cc:260] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:20.082284  4544 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:20.083268  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2252) took 3419 ms. Trace:
I20260709 13:02:20.083632  1642 rpcz_store.cc:276] 0709 13:02:16.664146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.665133 (+   987us) service_pool.cc:224] Handling call
0709 13:02:16.665369 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.941072 (+3275703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.082873 (+141801us) spinlock_profiling.cc:243] Waited 3.19 s on lock 0x7b540032c8d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:20.083100 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7343320448}
I20260709 13:02:20.087667  4469 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:20.089720  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:20.090593  1642 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:02:20.095266  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.080935  4533 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:20.101874  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.008286  4408 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:20.104993  1565 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 3.
W20260709 13:02:20.106666  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 73802.9us
W20260709 13:02:20.107879  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 245621us
I20260709 13:02:20.110241  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:20.112483  4495 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:20.014724  4450 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.122552  1609 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:20.128273  2303 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:20.133792  4450 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:20.142623  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.145943  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 84454.2us
I20260709 13:02:20.121027  4544 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:20.048623  4460 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:20.160497  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 266177us
W20260709 13:02:20.171214  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 48793.3us
W20260709 13:02:20.178042  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.179713  1714 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:02:20.181106  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.131596  1636 raft_consensus.cc:2955] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 3 to 6, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:20.185199  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:20.188937  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:20.134377  4487 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 13:02:20.193673  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 46738.2us
I20260709 13:02:20.134202  4468 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
W20260709 13:02:20.194527  1681 leader_election.cc:336] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.654s (SENT)
I20260709 13:02:20.156534  4544 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:20.199370  1566 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:20.200433  4408 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:20.201992  4469 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:20.203415  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:20.205440  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:20.210513  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:20.211648  1565 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:20.206063  1642 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
W20260709 13:02:20.217000  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.211624  2268 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.722s (SENT) [suppressed 80 similar messages]
I20260709 13:02:20.202697  4532 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 13:02:20.191236  4505 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:20.203435  4544 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:20.167846  4340 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:20.207561  4408 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:20.225838  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:20.213801  4314 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:20.193209  4448 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:02:20.217571  4469 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:20.249567  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.253315  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:20.268599  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.276381  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:20.277518  1532 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:20.277740  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:20.281416  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:20.287909  4578 log.cc:927] Time spent T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:02:20.287629  4403 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:20.288928  4403 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:20.290177  4403 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:20.306377  4430 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:20.307643  4512 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:20.312759  4415 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:20.288942  4408 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:20.271561  4427 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:20.319875  4571 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.174s	user 0.002s	sys 0.000s
W20260709 13:02:20.320115  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 102994us
W20260709 13:02:20.321813  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 142520us
W20260709 13:02:20.306843  4579 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:02:20.225095  4546 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:02:20.339504  1642 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:20.342501  2302 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:20.343549  2304 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:02:20.349924  4466 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:20.320130  4427 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:20.361325  1639 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:02:20.361850  1639 pending_rounds.cc:85] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 6
I20260709 13:02:20.362092  1639 pending_rounds.cc:107] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:02:20.362329  1639 raft_consensus.cc:2889] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:20.367276  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 89860.6us
I20260709 13:02:20.367633  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:20.368214  1532 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:20.366767  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:20.369837  1714 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 8: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:20.369939  4468 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:20.370579  4468 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:20.391754  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 51114.1us
W20260709 13:02:20.394424  4583 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:02:20.394662  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:20.395869  1640 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 13:02:20.398116  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 116800us
I20260709 13:02:20.306417  4533 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:20.373567  1534 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:20.410256  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:20.393445  1683 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:20.341735  4460 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:20.435523  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:20.394539  2305 raft_consensus.cc:2955] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 3 to 6, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:20.373018  4468 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:20.326774  4314 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:20.446252  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:20.415318  4048 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.341178  4469 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:20.466198  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 244251us
W20260709 13:02:20.466274  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 50223.5us
I20260709 13:02:20.441733  4532 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:20.476692  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:20.477770  4187 consensus_queue.cc:1059] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:02:20.481946  4580 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:20.482424  4586 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
W20260709 13:02:20.483381  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 91582.3us
I20260709 13:02:20.483691  4587 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:20.484256  4514 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:20.486650  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:20.487586  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 70878.7us
I20260709 13:02:20.495461  1642 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
W20260709 13:02:20.504792  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.508844  2297 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:20.465406  4468 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:20.469080  4427 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:20.525666  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 157541us
I20260709 13:02:20.518317  4450 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:20.553660  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 67115us
I20260709 13:02:20.498533  4048 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:02:20.593230  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 76178.4us
I20260709 13:02:20.533311  4495 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:20.553121  1532 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:20.609366  1638 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:20.521274  4580 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:20.624272  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:20.626168  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 291385us
I20260709 13:02:20.549643  4256 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 13:02:20.550573  4512 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:02:20.610709  4544 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:20.641229  4514 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:20.683166  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 59028us
W20260709 13:02:20.659873  4581 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
W20260709 13:02:20.685913  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.646334  4575 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:20.689193  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:20.691560  4491 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.689885  4314 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:20.706775  4580 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:20.694908  4469 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:20.705721  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 159372us
W20260709 13:02:20.717391  1681 leader_election.cc:336] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.707s (SENT)
I20260709 13:02:20.717545  1532 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:20.719678  4591 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:02:20.717048  4491 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:20.729379  2303 consensus_meta.cc:287] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.587s	user 0.006s	sys 0.005s
I20260709 13:02:20.729972  2303 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
W20260709 13:02:20.723341  4588 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
W20260709 13:02:20.739332  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 49511us
I20260709 13:02:20.745628  4599 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:20.750419  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:20.751158  4570 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
W20260709 13:02:20.753250  4592 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 13:02:20.753312  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 683) took 1050 ms. Trace:
I20260709 13:02:20.755910  4533 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:20.763540  4048 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:20.739499  4580 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:20.766180  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.754747  1607 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:20.777246  1681 leader_election.cc:336] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.718s (SENT)
I20260709 13:02:20.777926  1681 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:20.781667  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 65240.8us
I20260709 13:02:20.784578  1532 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:20.785749  4505 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:20.754033  2301 rpcz_store.cc:276] 0709 13:02:19.703136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.707696 (+  4560us) service_pool.cc:224] Handling call
0709 13:02:19.708069 (+   373us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:19.751422 (+ 43353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:19.751454 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:19.794592 (+ 43138us) log.cc:844] Serialized 52 byte log entry
0709 13:02:19.939060 (+144468us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:19.985998 (+ 46938us) log.cc:844] Serialized 14 byte log entry
0709 13:02:20.009956 (+ 23958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.010050 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:20.753042 (+742992us) raft_consensus.cc:1685] finished
0709 13:02:20.753070 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.753240 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31153,"prepare.queue_time_us":23501,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":57647,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":19882,"thread_start_us":108387,"threads_started":1,"wal-append.queue_time_us":712211}
I20260709 13:02:20.775635  4533 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:20.805308  4544 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:20.805527  4597 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:02:20.806097  4593 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:02:20.807016  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:20.808493  4512 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:20.810571  1532 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:20.805341  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 50706.4us
I20260709 13:02:20.811770  4512 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:20.812777  1607 leader_election.cc:400] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6.
W20260709 13:02:20.812922  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.813128  1607 leader_election.cc:403] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:02:20.814898  1532 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:20.816583  4512 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:20.817101  4512 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:20.817579  4512 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:20.819803  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:20.820703  2299 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
W20260709 13:02:20.810588  4596 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:02:20.815374  4533 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6.
I20260709 13:02:20.821887  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:20.799191  4204 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:20.825362  4187 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:20.831990  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 54818.7us
I20260709 13:02:20.831326  4187 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.835793  4187 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:20.849917  4487 raft_consensus.cc:3055] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:20.851342  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:20.853143  1681 leader_election.cc:336] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.749s (SENT)
W20260709 13:02:20.854280  1681 leader_election.cc:336] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.759s (SENT)
I20260709 13:02:20.854846  1681 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:20.855957  4450 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:02:20.856650  1681 leader_election.cc:336] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.764s (SENT)
W20260709 13:02:20.857852  1681 leader_election.cc:336] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.765s (SENT)
I20260709 13:02:20.858255  1681 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:20.858345  4533 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:20.858887  4533 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:20.859434  4533 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:20.864081  4340 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:20.850477  4487 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:20.868041  4487 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:20.868610  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:20.869576  4487 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:20.869692  4544 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:20.870213  4544 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:20.870642  4544 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:20.874368  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 185272us
I20260709 13:02:20.875146  4495 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:20.875254  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:20.876220  4495 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:20.876602  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:20.876699  4495 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:20.877308  4495 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:20.877732  4587 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:20.878197  4587 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:20.878556  4587 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:20.881055  4505 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:20.882809  4603 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:02:20.883733  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.884265  4314 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:02:20.885721  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:20.887769  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.891224  4314 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:20.893079  4408 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.893841  4408 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:20.841401  4491 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:20.895541  4448 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:20.896553  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:20.897414  2298 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:20.899122  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:20.901212  4340 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:20.913921  2268 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:20.914616  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:20.920449  1717 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:02:20.923854  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:20.924499  2299 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:20.924381  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:20.924741  1710 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:02:20.911569  4256 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:02:20.926195  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:20.927656  1711 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:20.928369  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:20.929667  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:20.930259  1713 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:20.902258  4340 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.933547  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:20.902462  4491 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.939406  1639 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:20.932024  1714 pending_rounds.cc:85] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 2
I20260709 13:02:20.940344  1714 pending_rounds.cc:107] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:02:20.940726  1714 raft_consensus.cc:2889] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:20.938649  1715 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:20.927094  1716 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:20.949858  1718 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:20.952543  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:20.924829  4538 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:20.973133  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 59276.1us
I20260709 13:02:20.929204  4450 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:20.939247  4340 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:20.984015  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:20.937994  4461 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.962838  4512 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:20.944653  4446 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:20.990557  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 132777us
I20260709 13:02:20.992432  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:20.950021  4491 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:20.993291  4610 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:21.033824  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 120168us
W20260709 13:02:21.036022  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 70165.4us
W20260709 13:02:21.031558  1681 leader_election.cc:336] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.790s (SENT)
I20260709 13:02:21.040961  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:21.049634  1636 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:21.042542  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:20.982806  4538 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.023784  4446 raft_consensus.cc:697] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:21.002965  4514 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:02:21.032611  4147 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:21.010726  4533 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.003563  4461 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.073997  4147 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:21.083854  4611 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.085384  4489 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:02:21.087809  4147 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:21.088029  4611 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.072894  4427 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:21.079964  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:21.085400  4408 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.099462  4427 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:02:21.104507  4618 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:02:21.104806  4427 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:21.107165  4624 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:02:21.107440  4512 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.107779  4204 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:21.085669  4430 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:02:21.111217  4625 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.111771  4625 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:21.113888  4625 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:21.114022  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:21.114974  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 88395.5us
I20260709 13:02:21.119067  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:21.122480  4461 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.123202  4461 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:21.125470  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:21.126895  2268 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:21.080931  4544 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:21.151551  1681 leader_election.cc:336] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.798s (SENT)
I20260709 13:02:21.084227  4460 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:02:21.057628  4610 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.167106  4446 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.170002  4187 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.212736  4469 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:21.217717  4340 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:21.222157  1638 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:21.227557  1642 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:21.161970  4514 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:21.240804  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 313937us
I20260709 13:02:21.251957  4610 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:21.253835  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 119268us
W20260709 13:02:21.261284  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.262072  1681 leader_election.cc:336] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.868s (SENT)
I20260709 13:02:21.262544  1681 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:21.267019  4495 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.267673  4340 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:21.269320  4495 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:21.271138  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:21.271862  2301 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:21.249240  4514 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:21.275957  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 98164.5us
I20260709 13:02:21.279592  1532 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:21.280751  1532 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:21.284622  1532 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:21.285274  4505 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:21.286446  4505 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.286862  4505 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:21.288546  4614 log.cc:927] Time spent T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.464s	user 0.003s	sys 0.000s
I20260709 13:02:21.290740  4638 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:02:21.291198  4512 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:21.291605  4512 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.291970  4512 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:21.293083  4623 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:02:21.294011  4611 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.294641  4611 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:21.294862  4415 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.260247  4577 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.295668  1532 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:02:21.280439  4621 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
I20260709 13:02:21.312296  4256 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:21.314533  4256 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.314942  4256 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:21.283928  4340 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:02:21.320969  4635 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.312865  4546 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:02:21.323421  4608 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:21.323933  4608 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:21.324420  4608 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.326092  4256 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.326674  4204 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:21.327313  4204 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.324015  4461 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:21.321586  4635 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:21.341769  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 84336us
W20260709 13:02:21.335016  4615 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:02:21.345347  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:21.346096  1718 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.346968  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:21.347992  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:21.350473  1711 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.333324  4461 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:21.313066  4538 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:21.329232  4450 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:21.305040  4495 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:02:21.335507  4491 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.337672  4533 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:21.358538  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 231707us
I20260709 13:02:21.358089  1713 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 13:02:21.362103  4634 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:02:21.362950  4427 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:21.363577  4632 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:02:21.298130  4610 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.048s
W20260709 13:02:21.407933  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:21.420169  1681 leader_election.cc:336] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.885s (SENT)
I20260709 13:02:21.312676  4577 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:21.424695  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 55454us
I20260709 13:02:21.427137  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:21.427259  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 52388.4us
I20260709 13:02:21.433450  4637 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.434089  4637 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.436878  1466 catalog_manager.cc:5697] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:21.428767  4546 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.438649  4546 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.441298  4546 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.442286  4147 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.444680  4147 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:21.445299  4256 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:21.448211  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 394384us
W20260709 13:02:21.449795  4487 consensus_meta.cc:287] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.000s
W20260709 13:02:21.450583  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.444774  4514 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.451534  4204 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.458144  1709 raft_consensus.cc:2410] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:21.420589  4544 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:21.458906  1711 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:21.473162  2302 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:21.477962  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.478251  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:21.483886  4512 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.429478  4491 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.376004  4450 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.477980  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:21.506350  2301 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
W20260709 13:02:21.502813  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.442812  4403 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:21.507279  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:21.513949  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:21.515965  4427 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:21.517832  4471 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.511196  4403 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:21.518728  4403 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:21.518491  4471 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:21.519661  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.520562  4471 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.524592  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:21.517927  4450 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.525241  1640 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.525494  4427 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.526490  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:21.526523  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:21.527246  2301 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.527673  4514 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:21.528275  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:21.525998  4427 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.528275  4514 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:21.533842  2304 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:21.537496  1718 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:21.539124  2301 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:21.467446  4147 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:21.544477  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:21.545446  1636 raft_consensus.cc:2410] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:21.431658  4430 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:02:21.549023  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:21.550383  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:21.552425  4541 consensus_queue.cc:1059] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:02:21.552605  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:21.553382  4541 raft_consensus.cc:3055] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:21.553540  1638 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:21.553864  4541 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:21.556125  4514 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:21.552608  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:21.559878  1682 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:21.507004  4187 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.562510  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:21.533396  4427 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:21.578655  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 118497us
W20260709 13:02:21.579085  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 146392us
I20260709 13:02:21.579990  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:21.561837  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:21.587168  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:21.591663  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:21.597250  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 613402us
I20260709 13:02:21.598114  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:21.601035  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:21.602720  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:21.565402  4487 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:21.554807  4541 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:21.566254  4546 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:21.606907  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:21.609047  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:21.616315  1565 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:21.606540  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:21.606003  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:21.610971  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:21.596590  1641 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:21.587383  4460 raft_consensus.cc:3055] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:21.617720  1567 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:21.617720  1710 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:21.620043  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:21.625530  1566 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:02:21.626801  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:21.634701  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:21.635676  1560 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:21.639073  1569 raft_consensus.cc:2368] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7.
W20260709 13:02:21.642628  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 48199.5us
I20260709 13:02:21.545527  4491 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.652622  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:21.654713  1642 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:21.657663  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 59672.1us
I20260709 13:02:21.610368  4487 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:21.669567  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 54756us
I20260709 13:02:21.663787  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:21.673290  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:21.661024  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:21.653676  4333 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:21.658999  4541 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:21.675056  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:21.676163  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:21.676378  4651 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:21.676795  4651 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.676900  2302 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:21.677165  4651 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.677644  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:21.679034  4572 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:02:21.677088  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:21.678956  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:21.680572  4466 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:21.681130  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:21.683377  1718 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:21.664901  1609 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:21.673537  1715 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:21.624495  4460 raft_consensus.cc:740] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:21.695197  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:21.699824  2297 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:21.700747  1563 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.701545  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:21.702309  1718 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.600742  4505 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:21.703495  1565 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:21.681097  4645 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.210s
I20260709 13:02:21.718015  4577 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:21.681069  4466 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:21.723495  4466 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:21.724218  4491 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.723763  1642 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:21.698158  4512 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.725777  1640 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:21.669636  4514 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.686156  4487 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:21.669998  4187 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:21.736900  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:21.728310  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 63523.8us
I20260709 13:02:21.732426  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:21.745218  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 71012.7us
I20260709 13:02:21.750605  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:21.747898  4314 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:21.752656  4495 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:21.780462  2298 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:21.786882  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:21.778450  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 128799us
I20260709 13:02:21.802059  1710 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:21.761806  4635 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:21.815588  1563 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:21.815768  1560 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:21.752231  4187 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:21.776870  4649 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.823768  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:02:21.803740  4587 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.925s	user 0.012s	sys 0.000s
I20260709 13:02:21.782995  4314 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:21.827857  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:21.786473  4495 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.765637  4514 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:21.849407  1718 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
W20260709 13:02:21.837642  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 164439us
I20260709 13:02:21.851257  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:21.861526  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.861920  1636 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:21.865214  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:21.865909  1710 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 3.
I20260709 13:02:21.867866  2300 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:21.838320  4469 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:21.868546  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 109201us
I20260709 13:02:21.843216  4314 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:21.840790  4495 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:21.875753  2268 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:21.872406  1718 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:21.778820  4460 consensus_queue.cc:260] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.883466  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:21.884361  1718 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:21.827445  4608 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.889576  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:21.852694  4637 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:21.898584  4646 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 13:02:21.909207  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:21.910161  1716 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:21.919167  4645 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.919775  4645 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:21.922017  4645 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:21.923676  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:21.926218  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:21.928541  4645 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.929239  4645 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:21.930127  4639 log.cc:927] Time spent T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.611s	user 0.000s	sys 0.003s
I20260709 13:02:21.931362  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:21.933202  1534 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:21.935510  4645 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:21.943691  1466 catalog_manager.cc:5697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:21.946884  1466 catalog_manager.cc:5697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 2 to 4, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 4 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:21.934769  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:21.962352  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.964499  4514 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:21.965000  4580 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:21.965400  4514 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:21.965790  4580 raft_consensus.cc:697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:21.966535  4505 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:21.967482  4505 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:21.968297  4505 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:21.971083  4599 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:21.835721  4649 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:21.976444  4649 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:21.978315  4641 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:21.979172  4641 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:21.977919  4580 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.980140  4641 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.984501  4651 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.985461  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:21.988652  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.875809  4635 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:22.005972  1683 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:22.010596  2303 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:21.881156  1533 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:22.021658  1532 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:22.024847  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:21.841256  4587 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:21.881140  4469 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:22.058830  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 177753us
I20260709 13:02:22.060592  2305 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:22.070109  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:22.091307  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:22.091583  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:22.026866  4450 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:22.055138  4638 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:22.112859  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:22.073908  4611 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:22.086473  4415 consensus_queue.cc:1059] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:22.080703  1637 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:22.012982  4468 consensus_queue.cc:1059] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:02:22.102552  1716 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:22.116986  1532 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:22.123986  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 248311us
I20260709 13:02:22.124684  2300 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:02:22.022707  4635 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.068388  4514 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:02:22.132349  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:22.131536  2300 pending_rounds.cc:85] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 6
I20260709 13:02:22.135185  2300 pending_rounds.cc:107] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.7
I20260709 13:02:22.137172  1532 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:22.136336  2300 raft_consensus.cc:2917] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Aborting config change with OpId 7.7: Aborted: Op aborted by new leader
I20260709 13:02:22.143108  4650 raft_consensus.cc:3055] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:22.143726  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:22.146569  2300 raft_consensus.cc:2889] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:22.149276  1716 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:22.145346  1532 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:22.144925  4546 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:22.114951  4637 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:22.096588  4466 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.091202  4649 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:02:22.137318  4661 raft_consensus.cc:3055] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:22.135996  4611 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:22.159322  4048 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:22.179916  1716 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:22.150537  4505 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:22.184060  1532 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:22.149053  4650 raft_consensus.cc:740] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:22.202209  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:22.162635  4491 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:22.208859  4657 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.337s	user 0.000s	sys 0.001s
I20260709 13:02:22.209308  4587 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:22.176117  4514 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:02:22.210659  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 342015us
W20260709 13:02:22.185703  4541 consensus_meta.cc:287] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.000s
I20260709 13:02:22.204725  4461 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:22.177533  4661 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:22.194278  4505 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:22.225392  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.161248  4544 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:02:22.196166  4187 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:22.260083  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 235300us
I20260709 13:02:22.267056  1716 raft_consensus.cc:2410] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:02:22.223496  4403 consensus_meta.cc:287] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.704s	user 0.011s	sys 0.001s
I20260709 13:02:22.283912  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:22.180857  4637 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.238866  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:22.236958  1643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.008s	sys 0.012s
W20260709 13:02:22.291700  1643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.009s	sys 0.012s
I20260709 13:02:22.292909  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:22.217937  4666 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:22.294199  1641 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:22.220808  4466 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:22.220293  4514 raft_consensus.cc:697] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:22.294013  1466 catalog_manager.cc:5697] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 2 to 3, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 3 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:22.237494  4505 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:22.322849  1638 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:02:22.328763  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1749) took 1391 ms. Trace:
I20260709 13:02:22.238049  4608 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:22.348389  1709 rpcz_store.cc:276] 0709 13:02:20.902327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.902484 (+   157us) service_pool.cc:224] Handling call
0709 13:02:22.293108 (+1390624us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b54001360d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:22.293261 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":811598208}
I20260709 13:02:22.217168  4491 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:22.322540  4657 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:02:22.294632  4461 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:22.292675  4333 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:22.215364  4495 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:22.382278  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.385126  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 494077us
I20260709 13:02:22.391592  4650 consensus_queue.cc:260] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:22.393811  4469 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:22.322767  4403 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.233641  4546 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:22.398505  4487 consensus_meta.cc:287] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.644s	user 0.008s	sys 0.000s
I20260709 13:02:22.413859  1641 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:22.420868  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:22.435278  1638 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:22.377175  4670 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:02:22.446396  1639 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:22.416289  4538 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:22.450304  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:22.451152  2300 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:22.376662  4466 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:02:22.437878  4469 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.457751  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:22.461362  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:22.446488  4670 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:22.470697  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 326678us
I20260709 13:02:22.361356  4611 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.473345  1709 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:22.464533  4650 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:22.441329  4487 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.460873  4635 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:22.500470  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:22.440155  4661 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.474295  4670 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:22.459226  4538 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.509572  2268 leader_election.cc:400] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7.
I20260709 13:02:22.510403  1715 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:22.510092  2268 leader_election.cc:403] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:22.469242  4469 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.523403  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:22.478489  4514 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.520020  4661 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:22.521189  4666 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:22.528492  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:22.521854  4048 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:22.571902  4689 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:22.572696  4689 raft_consensus.cc:697] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:22.572597  4048 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.573686  4689 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:22.531224  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.531383  4538 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:22.539086  4670 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.577427  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:22.548756  4487 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:22.550369  4469 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:22.551528  4314 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.659s	user 0.011s	sys 0.002s
I20260709 13:02:22.551919  4446 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:02:22.553086  4671 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:02:22.567998  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:22.569474  4491 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:22.522405  4608 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:02:22.569439  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:22.576376  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 47970.7us
I20260709 13:02:22.604416  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:22.579528  4691 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:22.526955  4427 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:22.595952  4495 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:22.607105  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:22.571928  4666 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:22.620944  4641 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:22.595813  4674 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:02:22.593896  4403 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:22.623219  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:22.623720  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:22.625267  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:22.631723  4670 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:22.647003  4460 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:22.655431  4691 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.598575  4314 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.676095  4641 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.677215  4314 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:22.606216  4469 raft_consensus.cc:3055] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:22.678182  4469 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:22.679520  4469 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.681560  4469 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:22.682442  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 488) took 1084 ms. Trace:
I20260709 13:02:22.683741  2300 raft_consensus.cc:2410] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:02:22.657112  4653 log.cc:927] Time spent T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:02:22.653798  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:22.686566  2302 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:22.662052  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:22.665942  4660 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.636s	user 0.001s	sys 0.004s
I20260709 13:02:22.621753  4538 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:22.621707  4446 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:22.672020  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1828) took 1112 ms. Trace:
W20260709 13:02:22.673080  4673 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:02:22.692147  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1763) took 1339 ms. Trace:
I20260709 13:02:22.692832  1643 rpcz_store.cc:276] 0709 13:02:21.559420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.561015 (+  1595us) service_pool.cc:224] Handling call
0709 13:02:21.585364 (+ 24349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.608113 (+ 22749us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.608145 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.608698 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.292228 (+683530us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:02:22.292284 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.292501 (+   217us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.671639 (+379138us) raft_consensus.cc:1685] finished
0709 13:02:22.671667 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.671915 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71576,"raft.queue_time_us":58147,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":9215,"thread_start_us":683095,"threads_started":1,"wal-append.queue_time_us":870647}
I20260709 13:02:22.682859  1561 rpcz_store.cc:276] 0709 13:02:21.598025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.598167 (+   142us) service_pool.cc:224] Handling call
0709 13:02:21.631921 (+ 33754us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:21.833765 (+201844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.833901 (+   136us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:21.850288 (+ 16387us) log.cc:844] Serialized 78 byte log entry
0709 13:02:22.213587 (+363299us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:22.262657 (+ 49070us) log.cc:844] Serialized 14 byte log entry
0709 13:02:22.681214 (+418557us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b18003d07e0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb2ec0a 00007fd7a826e219 00007fd79ebb6a40 00007fd79ebb14e4 00007fd79ebb1455 00007fd7a56993c7 00007fd7a1cab9a7 00007fd7a1cbb0bf 00007fd7a1cbb038
0709 13:02:22.681436 (+   222us) log.cc:844] Serialized 14 byte log entry
0709 13:02:22.681781 (+   345us) log.cc:844] Serialized 14 byte log entry
0709 13:02:22.681997 (+   216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.682151 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.682196 (+    45us) raft_consensus.cc:1685] finished
0709 13:02:22.682209 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.682348 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":347853,"prepare.queue_time_us":474597,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":1928,"raft.queue_time_us":157086,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":207,"spinlock_wait_cycles":79833728,"thread_start_us":462302,"threads_started":2,"wal-append.queue_time_us":591052}
I20260709 13:02:22.812377  4446 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:22.812901  4147 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:22.813496  4147 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.815901  4147 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:22.813482  4446 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.819104  4491 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.819904  4491 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
W20260709 13:02:22.826507  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 130477us
I20260709 13:02:22.827488  2302 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:22.678563  4641 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:22.829617  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:22.698334  1717 rpcz_store.cc:276] 0709 13:02:21.346767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.346968 (+   201us) service_pool.cc:224] Handling call
0709 13:02:21.367448 (+ 20480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.426520 (+ 59072us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.426560 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.432214 (+  5654us) log.cc:844] Serialized 26 byte log entry
0709 13:02:21.837888 (+405674us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:21.869755 (+ 31867us) log.cc:844] Serialized 14 byte log entry
0709 13:02:21.872363 (+  2608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.872437 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.561780 (+689343us) spinlock_profiling.cc:243] Waited 73.2 ms on lock 0x7b18003bc440. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:22.685708 (+123928us) raft_consensus.cc:1685] finished
0709 13:02:22.685810 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.685946 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5089,"prepare.queue_time_us":31710,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":3788,"raft.queue_time_us":7420,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":14066,"spinlock_wait_cycles":168473856,"thread_start_us":328858,"threads_started":1,"wal-append.queue_time_us":451311}
I20260709 13:02:22.834503  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:22.838776  1641 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:22.842073  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:22.846356  4403 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:22.848300  4628 consensus_queue.cc:1059] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:22.849614  4628 raft_consensus.cc:3055] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:22.849471  4427 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:22.850100  4628 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:22.826609  4691 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:22.850400  4427 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.704841  4637 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:22.827883  4048 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.851496  4538 raft_consensus.cc:697] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:22.851346  4314 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:02:22.900887  4491 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:22.937546  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1824) took 1426 ms. Trace:
W20260709 13:02:22.943349  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.938292  1635 rpcz_store.cc:276] 0709 13:02:21.511598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.512275 (+   677us) service_pool.cc:224] Handling call
0709 13:02:21.512808 (+   533us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:21.551810 (+ 39002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.551857 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:21.604534 (+ 52677us) log.cc:844] Serialized 590 byte log entry
0709 13:02:21.810897 (+206363us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:21.810976 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.811059 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.664989 (+853930us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:22.930300 (+265311us) raft_consensus.cc:1685] finished
0709 13:02:22.930329 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.937468 (+  7139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":142,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":15225,"spinlock_wait_cycles":62526848,"thread_start_us":148908,"threads_started":1,"wal-append.queue_time_us":253005}
I20260709 13:02:22.902849  4048 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.929606  4628 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.964336  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 487) took 1367 ms. Trace:
I20260709 13:02:22.959534  4645 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:22.963409  4505 consensus_meta.cc:287] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.587s	user 0.012s	sys 0.000s
I20260709 13:02:22.965068  1564 rpcz_store.cc:276] 0709 13:02:21.597310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.597470 (+   160us) service_pool.cc:224] Handling call
0709 13:02:21.632422 (+ 34952us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.838223 (+205801us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:21.879526 (+ 41303us) log.cc:844] Serialized 14 byte log entry
0709 13:02:22.231309 (+351783us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.326322 (+ 95013us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.326509 (+   187us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:22.920867 (+594358us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b48000030d8. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a6bcaaa1 00007fd7a6bc1b37 00007fd7a6bc19ca 00007fd7a6cb926f 00007fd7a6cb8f37 00007fd7a6ccd55c 00007fd7a6ccd508 00007fd7a6ccd488 00007fd7a6ccd443 00007fd7a6ccc6ec 00007fd7a5bf6a96
0709 13:02:22.958183 (+ 37316us) log.cc:844] Serialized 14 byte log entry
0709 13:02:22.958454 (+   271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.958611 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.964106 (+  5495us) raft_consensus.cc:1685] finished
0709 13:02:22.964129 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.964275 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":425248,"prepare.queue_time_us":399089,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":85,"raft.queue_time_us":167076,"raft.run_cpu_time_us":4673,"raft.run_wall_time_us":610036,"spinlock_wait_cycles":34386304,"thread_start_us":722245,"threads_started":3,"wal-append.queue_time_us":550768}
I20260709 13:02:22.979780  4427 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:22.981156  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:22.990769  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.961266  4538 consensus_queue.cc:237] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:22.992615  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:22.999322  4699 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:02:22.974889  4505 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:23.001811  1635 raft_consensus.cc:2410] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:02:23.008601  4662 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.886s	user 0.003s	sys 0.000s
I20260709 13:02:23.017349  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 706) took 1265 ms. Trace:
I20260709 13:02:23.017815  2303 rpcz_store.cc:276] 0709 13:02:21.752346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.752567 (+   221us) service_pool.cc:224] Handling call
0709 13:02:21.760387 (+  7820us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.011161 (+250774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.017059 (+1005898us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b5400708fd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:23.017245 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":575041408}
I20260709 13:02:23.019253  4608 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7.
I20260709 13:02:23.027120  2304 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:23.036816  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:23.044374  4647 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:23.045096  4647 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:23.049077  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.050458  4691 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:23.051019  4691 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:23.053081  4691 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:23.053967  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 579) took 1197 ms. Trace:
I20260709 13:02:23.054543  4505 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:23.054365  1639 rpcz_store.cc:276] 0709 13:02:21.857053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.857217 (+   164us) service_pool.cc:224] Handling call
0709 13:02:21.857612 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.447045 (+589433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.053754 (+606709us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b5400324bd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:23.053911 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1353637760}
W20260709 13:02:23.055441  4700 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:02:23.058192  1639 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:23.060107  1644 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:23.061064  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:23.065657  2301 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:23.059486  4415 consensus_queue.cc:1059] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:02:23.069157  4048 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:23.077796  4340 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.079615  2302 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:02:23.081645  4610 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.083366  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:23.078351  4340 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:23.088986  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:23.096143  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:23.095213  4048 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:23.078791  4505 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.097951  4256 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:23.087865  4610 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:23.097472  4505 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:23.100390  4610 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:23.100493  4654 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 1.269s	user 0.003s	sys 0.000s
I20260709 13:02:23.100661  4505 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:23.101864  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 703) took 1632 ms. Trace:
I20260709 13:02:23.103389  4340 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:23.102236  2299 rpcz_store.cc:276] 0709 13:02:21.469211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.469384 (+   173us) service_pool.cc:224] Handling call
0709 13:02:21.469682 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.511588 (+ 41906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.511618 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.521435 (+  9817us) log.cc:844] Serialized 26 byte log entry
0709 13:02:21.649798 (+128363us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:21.649844 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.649932 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.099889 (+1449957us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:23.101589 (+  1700us) raft_consensus.cc:1685] finished
0709 13:02:23.101610 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.101804 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46046,"raft.queue_time_us":42022,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"spinlock_wait_cycles":13206784,"thread_start_us":111398,"threads_started":1,"wal-append.queue_time_us":241077}
I20260709 13:02:23.106824  2299 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:23.107127  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:23.107892  2303 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:23.109829  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:23.118909  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:23.119796  1644 raft_consensus.cc:2410] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:02:23.117022  4647 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:23.121973  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:23.128743  2302 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:23.130887  4333 consensus_queue.cc:1059] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:23.132000  4333 consensus_queue.cc:1243] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:02:23.132576  4333 raft_consensus.cc:3055] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:23.132988  4333 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:23.118734  4340 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:02:23.134023  4333 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:23.120600  4610 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:02:23.129994  4689 consensus_queue.cc:1059] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:02:23.135941  4544 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:23.189569  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:23.135872  4427 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:23.188767  4505 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:23.199705  4505 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:23.136919  4651 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:23.137614  4460 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:23.151075  4628 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:23.151579  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:23.123483  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 581) took 1388 ms. Trace:
W20260709 13:02:23.156466  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.156787  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 883649us
I20260709 13:02:23.209597  1683 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:23.164745  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:23.176075  1562 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:23.211433  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:23.212056  1562 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:23.212958  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:23.213536  1562 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:23.214270  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:23.181734  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.188751  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:23.191071  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:23.194315  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:23.192556  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:23.219767  1569 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:23.219978  1565 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:23.193264  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:23.166015  4541 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:23.194306  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:23.223146  1568 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:23.191833  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:23.223297  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 71845.9us
I20260709 13:02:23.223812  1567 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:23.220160  1566 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:23.224510  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:23.193562  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:23.220846  1561 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:23.225157  1563 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:23.225353  1560 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
W20260709 13:02:23.223706  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.207926  1715 rpcz_store.cc:276] 0709 13:02:21.735576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.827533 (+ 91957us) service_pool.cc:224] Handling call
0709 13:02:21.985397 (+157864us) spinlock_profiling.cc:243] Waited 975 us on lock 0x7b5c00139e8c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a560b161 00007fd7a57aca79 00007fd7a6c669b6 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:23.123243 (+1137846us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b5400135bd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:23.123407 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1207756928}
I20260709 13:02:23.227407  4256 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:23.228217  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:23.233906  1565 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:23.227425  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:23.234567  4468 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.234977  1643 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:23.235096  4664 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:23.202071  4505 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:23.253165  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:23.236009  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.236567  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:23.190083  4544 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:23.238122  1639 raft_consensus.cc:2955] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:23.238210  1681 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:23.239953  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.261247  1563 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:23.241648  1562 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:23.242430  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1775) took 1122 ms. Trace:
I20260709 13:02:23.222666  4333 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:23.248417  1564 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:23.251025  1718 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:23.251705  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:23.252251  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:23.274597  2304 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:23.275439  1534 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:23.268038  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:23.267135  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.246347  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:23.262813  1565 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:23.272291  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:23.279628  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:23.281241  2303 raft_consensus.cc:2393] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:23.256923  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:23.266464  1682 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:23.222767  4541 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:23.282433  1567 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:23.294152  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:23.288628  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:23.290880  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:23.295169  1563 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:23.249742  4664 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.252418  4187 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:23.262694  1713 rpcz_store.cc:276] 0709 13:02:22.120364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.120562 (+   198us) service_pool.cc:224] Handling call
0709 13:02:22.133376 (+ 12814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:22.133586 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:22.135066 (+  1480us) log.cc:844] Serialized 14 byte log entry
0709 13:02:23.155505 (+1020439us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b1800384dc0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb2ec0a 00007fd7a826e219 00007fd79ebb6a40 00007fd79ebb14e4 00007fd79ebb1455 00007fd7a1c99060 00007fd7a1cab716 00007fd7a1cbc287 00007fd7a1cbc208
0709 13:02:23.242125 (+ 86620us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.242176 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:23.242215 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.242275 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.242380 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2663,"prepare.queue_time_us":3706,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":1035482,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":6218,"spinlock_wait_cycles":15540992,"thread_start_us":86387,"threads_started":1}
I20260709 13:02:23.252183  4204 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.317489  2304 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:23.319192  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:23.319797  1710 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:23.320135  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:23.250222  4471 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:23.330356  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.300997  1718 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:23.305135  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.332203  1711 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:23.260092  4461 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:23.338343  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 100233us
I20260709 13:02:23.313241  2296 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:23.313495  1532 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:23.314109  1560 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:23.284639  4678 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:23.331084  2302 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:23.282081  4487 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:23.350347  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:23.229528  4256 raft_consensus.cc:2955] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:23.273808  4651 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:23.296607  4635 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:23.342087  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:23.314321  4638 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.176s	user 0.006s	sys 0.021s
I20260709 13:02:23.268111  4668 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:02:23.253546  4468 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:23.315069  4187 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.311090  4664 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:23.303519  4505 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:23.309696  4512 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:23.359136  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.384624  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:23.356231  4638 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.220s	user 0.006s	sys 0.022s
I20260709 13:02:23.401479  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.413273  1718 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:23.417057  1710 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 4.
I20260709 13:02:23.568636  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.568732  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.417536  1713 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:02:23.419759  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 69545us
I20260709 13:02:23.571187  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:23.420935  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 72706.3us
I20260709 13:02:23.385658  4187 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:23.419826  2268 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:23.573872  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 154136us
I20260709 13:02:23.426191  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:23.575187  1563 raft_consensus.cc:2410] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:02:23.392241  4505 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.410498  4691 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.430845  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:23.431490  4048 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:23.425979  4686 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:23.435542  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 75573.4us
I20260709 13:02:23.414577  4544 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:23.351668  4461 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:23.437350  4668 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:23.443310  4546 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:23.400877  4541 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:23.416095  4469 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.446861  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:23.676826  1565 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:23.451304  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:23.452371  1564 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:02:23.452992  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.398030  4638 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:23.384743  4487 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:23.456539  4608 raft_consensus.cc:3055] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:23.381314  4651 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:23.341377  4204 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:23.461431  1641 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:23.465013  1569 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:23.465406  1561 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:23.466230  1568 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
W20260709 13:02:23.469385  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 155366us
I20260709 13:02:23.470110  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:23.348109  4471 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:23.498351  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.504173  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.425421  4711 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:23.533118  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 213045us
W20260709 13:02:23.551426  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.706494  1711 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:23.706996  4711 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:23.706835  4638 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:23.707507  4711 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:23.707993  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.708156  1711 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:23.710522  1709 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:23.712260  4608 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:23.712548  4204 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:23.712770  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:23.687865  4468 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:23.637708  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.681465  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:23.713985  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:23.638792  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:23.655825  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.648825  4691 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:23.716070  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:23.716513  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.717000  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.717170  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:23.717444  1683 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:23.717872  1713 raft_consensus.cc:2410] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:02:23.714655  1637 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:23.719075  4686 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:23.719262  4635 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:23.719946  4635 raft_consensus.cc:697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:23.720238  4637 consensus_queue.cc:1059] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:02:23.656725  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 85635.7us
I20260709 13:02:23.722098  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:23.722394  1718 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:23.722576  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:23.723215  1569 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:23.723654  1562 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:02:23.660630  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.633688  4505 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:23.665912  4725 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.635572  4512 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:23.747916  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:23.634114  4661 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:23.680660  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:23.663129  4546 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.657785  4668 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:23.695767  1642 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:23.715251  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:23.681054  1715 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:23.706067  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:23.729674  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:23.678970  1717 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
W20260709 13:02:23.658419  4663 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.436s	user 0.008s	sys 0.026s
I20260709 13:02:23.720901  4471 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:23.717712  4691 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:23.754079  1718 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:23.719661  4333 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:02:23.790587  4663 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.574s	user 0.008s	sys 0.026s
I20260709 13:02:23.755033  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.790737  4711 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:23.791771  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.789812  4468 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:23.792773  4468 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.793170  4468 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:23.793071  4664 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:23.790834  4333 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:23.763440  4546 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:23.796211  4333 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:23.796963  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.764814  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:23.799403  4691 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:23.799108  4668 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:23.799813  4691 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:23.800231  4691 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:23.800384  4668 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:23.801543  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:23.802327  4489 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:23.802444  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:23.802778  4489 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:23.798573  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.803150  4541 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:23.803453  4489 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:23.804209  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:23.805068  1718 raft_consensus.cc:2368] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9.
I20260709 13:02:23.805115  1710 raft_consensus.cc:2410] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:02:23.805967  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:23.790129  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 112193us
I20260709 13:02:23.807669  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:23.766538  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:23.774750  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:23.787274  1638 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:02:23.789805  4446 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:23.715267  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:23.793012  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.818235  1713 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:23.805927  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.789189  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.696058  4666 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.740975  1714 raft_consensus.cc:2410] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:23.791929  1565 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:02:23.744306  4514 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:23.836099  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 120927us
I20260709 13:02:23.814363  1709 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:23.829843  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:23.833240  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:23.835183  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:23.822163  1569 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:23.857388  1682 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:23.837461  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:23.857740  1563 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
W20260709 13:02:23.859352  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 194570us
I20260709 13:02:23.762686  4204 raft_consensus.cc:3055] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:23.721266  4637 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.111s
I20260709 13:02:23.863404  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.697543  4469 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:23.829526  4660 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:02:23.817603  4446 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:23.864027  1681 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:23.870347  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:23.899618  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.874876  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:23.901010  1715 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:23.881239  1638 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:23.719758  4686 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:23.756654  4685 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 13:02:23.869566  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:23.720988  4635 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:23.713253  4608 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:23.756273  4403 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 13:02:23.837206  4471 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:23.758935  4641 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.107s
W20260709 13:02:23.907860  4724 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:23.908205  4726 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:23.915161  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:23.921826  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 57895.6us
I20260709 13:02:23.924118  4340 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:23.927171  1565 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:23.870709  4333 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
W20260709 13:02:24.023280  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 301273us
I20260709 13:02:23.933001  1711 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:23.931944  4487 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:23.934633  4628 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.712s	user 0.003s	sys 0.007s
W20260709 13:02:23.939616  4730 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 13:02:23.940487  4723 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:02:23.950613  4686 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:23.952045  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:23.914309  4720 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:02:23.960098  1718 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:24.088967  1718 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:02:23.859354  4415 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:23.866891  4204 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:24.101006  4628 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.877972  4661 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:23.973546  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:23.910369  4446 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:23.983700  4461 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:23.985554  4649 consensus_queue.cc:1059] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:23.987150  4450 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:02:23.989856  4544 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:23.995707  1710 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:02:24.126439  1710 pending_rounds.cc:85] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 6
I20260709 13:02:23.973650  4608 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:23.759101  4725 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:23.854564  4666 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.002734  4651 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:24.003943  4403 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:24.005174  4512 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:24.023211  1681 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:24.036041  4340 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.030606  1562 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:23.930397  4187 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.024497  1532 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:24.080308  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:24.085439  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:24.092769  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.090658  1718 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:02:24.147065  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 289782us
W20260709 13:02:24.104431  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.036033  4711 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:24.035706  4471 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:24.136970  4649 consensus_queue.cc:1243] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:02:24.133989  1714 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:02:24.126713  1710 pending_rounds.cc:107] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.7
I20260709 13:02:24.022681  4505 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.096920  4415 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:24.176515  4415 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:24.176672  4489 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.179436  1716 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:24.110715  4661 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:24.142838  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 119736us
I20260709 13:02:23.933614  4469 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:24.150518  1569 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:02:24.106271  4628 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:24.148499  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.150655  1562 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:02:24.153716  1682 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:24.156414  1564 raft_consensus.cc:2393] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:02:24.158232  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 133807us
W20260709 13:02:24.123309  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 320831us
I20260709 13:02:24.204053  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:24.166393  1710 raft_consensus.cc:2917] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Aborting config change with OpId 7.7: Aborted: Op aborted by new leader
I20260709 13:02:24.164080  4471 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:24.172122  4187 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:24.173410  4340 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:24.174664  4512 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:24.150974  4726 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:24.175477  4505 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:24.181833  1713 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 4.
I20260709 13:02:24.223394  2298 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:02:24.187718  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 317508us
I20260709 13:02:24.223853  1560 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:24.224606  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:24.151234  4685 consensus_queue.cc:1243] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:24.124346  4546 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:24.208477  1710 raft_consensus.cc:2889] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:24.209995  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 60739.5us
I20260709 13:02:24.210207  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:24.212826  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:24.219121  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.225314  1532 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:24.125984  4204 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.187592  4403 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:24.204739  4661 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:24.212126  4649 raft_consensus.cc:1064] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:02:24.152184  4691 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:24.241503  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.163527  4468 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:24.213380  4450 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:24.222223  4726 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:24.214219  4469 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:24.252424  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:24.253964  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:24.255065  1532 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:24.285343  1532 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:24.255795  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 45684.7us
I20260709 13:02:24.286386  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:24.262329  1568 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 7. Ops: [5.4-5.4]
I20260709 13:02:24.287326  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:24.287567  4649 raft_consensus.cc:1068] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Unable to promote non-voter 6208c2fa49f643b5a50ab8f6d292fab2: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:24.226401  4489 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:24.287961  1637 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:24.286175  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:24.252498  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:24.227308  4721 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:24.290072  1635 raft_consensus.cc:2410] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:02:24.286983  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:24.294616  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.241945  4403 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:24.311074  4403 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:24.307649  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:24.314316  4649 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:24.315409  4641 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:24.315930  4721 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.316423  4721 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:24.243592  4685 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:24.318499  4689 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:24.319064  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:24.320303  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:24.242192  4187 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:02:24.320888  4669 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:02:24.308140  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 55821.1us
I20260709 13:02:24.321460  4460 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:24.322043  4460 raft_consensus.cc:697] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:24.322247  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:24.323463  4460 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:24.325440  4736 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:02:24.326159  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:24.321508  4669 raft_consensus.cc:697] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:24.326853  1640 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:24.327715  4669 consensus_queue.cc:237] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.328636  4666 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:24.329258  4666 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:24.330255  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:24.236479  4720 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:02:24.331118  4489 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:24.331364  4735 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:02:24.331950  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 588) took 1079 ms. Trace:
I20260709 13:02:24.332994  1718 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:24.336409  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:24.230049  4461 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:02:24.309058  1712 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:24.344724  2305 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:24.309478  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:24.319191  4689 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:24.353765  4725 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:24.353575  4689 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.284274  4726 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:24.355482  4415 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:24.357934  4415 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:24.331611  4489 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:24.371258  4489 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.357828  4647 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:02:24.352389  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:24.358443  4461 consensus_queue.cc:1243] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:24.341209  4743 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:02:24.359120  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:24.359699  4649 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:24.360169  4685 consensus_queue.cc:1059] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:24.360201  4546 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:24.332302  1716 rpcz_store.cc:276] 0709 13:02:23.252820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.252947 (+   127us) service_pool.cc:224] Handling call
0709 13:02:24.331742 (+1078795us) spinlock_profiling.cc:243] Waited 925 ms on lock 0x7b5400148cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:24.331894 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2128142208}
I20260709 13:02:24.383925  4649 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:24.384482  4649 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:24.299556  4741 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:24.335840  4734 log.cc:927] Time spent T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:02:24.363941  4446 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.365183  1638 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:24.369992  4637 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.393827  4647 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:02:24.399222  4637 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.264745  4740 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:24.401053  4740 raft_consensus.cc:697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:24.401427  4637 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:24.402150  2302 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:24.283308  4733 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:24.403267  4733 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.404417  4446 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:24.404675  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:24.386504  4661 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.405582  2300 raft_consensus.cc:2410] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:24.406031  4733 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:24.296382  4737 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:24.374522  1569 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:24.380534  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:24.406906  4737 raft_consensus.cc:697] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:24.407351  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:24.408404  4661 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:24.408128  4737 consensus_queue.cc:237] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:24.410597  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.345104  4746 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.375299  4333 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.397971  4685 raft_consensus.cc:3055] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:24.398415  4691 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:24.407430  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:24.421310  1569 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:24.390195  2268 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:24.291164  4738 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.097s
W20260709 13:02:24.413692  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.393539  1641 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:24.388420  4743 consensus_queue.cc:1243] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:24.424032  2268 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:24.392019  4747 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:24.411126  4744 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:24.426383  4333 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.428685  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:24.445720  1569 raft_consensus.cc:2376] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7. Re-sending same reply.
I20260709 13:02:24.428856  4647 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.434122  4204 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:24.434311  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:24.436551  4728 log.cc:927] Time spent T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.321s	user 0.000s	sys 0.002s
I20260709 13:02:24.485060  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:24.486125  1569 raft_consensus.cc:2376] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7. Re-sending same reply.
I20260709 13:02:24.488248  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:24.419632  4733 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:24.487800  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:24.421176  4685 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:24.425755  4637 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.498567  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:24.513346  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:24.538769  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:24.540359  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:24.443723  4608 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:24.542296  4608 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:24.477600  4691 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:24.545046  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:24.545317  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 46859.1us
I20260709 13:02:24.553483  1569 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:24.557399  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:24.558357  1716 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:24.483695  4744 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:24.537026  4735 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:02:24.605492  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:24.611896  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:24.614563  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:24.578529  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:24.571991  1713 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:02:24.396600  4741 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.574389  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:24.511854  4733 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.587231  4333 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:24.595659  4468 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:24.493813  4647 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:24.566233  4715 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.008s	sys 0.032s
I20260709 13:02:24.598378  1717 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:24.402284  4740 consensus_queue.cc:237] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.494707  4446 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.070s
W20260709 13:02:24.653090  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 339477us
I20260709 13:02:24.663527  2297 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:24.599960  4628 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:24.422047  4746 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.668043  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:24.688534  1716 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:24.572476  4608 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:24.652845  4715 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.008s	sys 0.032s
I20260709 13:02:24.627933  4691 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:24.695196  2269 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:24.705152  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1928) took 1324 ms. Trace:
W20260709 13:02:24.705508  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 419072us
I20260709 13:02:24.663345  4538 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:24.664412  4733 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:24.705839  1566 rpcz_store.cc:276] 0709 13:02:23.380399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.382261 (+  1862us) service_pool.cc:224] Handling call
0709 13:02:23.382632 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:23.439168 (+ 56536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:23.439202 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:23.439721 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.866028 (+426307us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:02:23.866085 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.866184 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.435814 (+569630us) spinlock_profiling.cc:243] Waited 6.87 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:24.700471 (+264657us) raft_consensus.cc:1685] finished
0709 13:02:24.700501 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.705058 (+  4557us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56094,"raft.queue_time_us":56222,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":122,"spinlock_wait_cycles":15805184,"thread_start_us":425874,"threads_started":1,"wal-append.queue_time_us":538864}
I20260709 13:02:24.569798  4637 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.706533  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:24.712000  2269 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:24.737265  1716 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:24.740155  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:24.740458  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:24.740998  1709 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:24.749166  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1939) took 1301 ms. Trace:
I20260709 13:02:24.754904  1718 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:02:24.749671  1565 rpcz_store.cc:276] 0709 13:02:23.447768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.854830 (+407062us) service_pool.cc:224] Handling call
0709 13:02:23.877258 (+ 22428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.020779 (+143521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.725913 (+705134us) spinlock_profiling.cc:243] Waited 49.7 ms on lock 0x7b5400276854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:24.749046 (+ 23133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114221952}
I20260709 13:02:24.757057  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:02:24.757529  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 45630us
I20260709 13:02:24.757937  1718 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:24.719584  4669 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:24.764783  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:24.765509  1709 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:24.760582  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:24.750154  2301 raft_consensus.cc:2955] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:24.763775  1711 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:24.775933  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:24.780638  4403 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:24.752025  4514 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:24.783283  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 359494us
I20260709 13:02:24.794229  2296 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:24.794642  2300 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:24.711292  4691 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.762404  4340 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:24.814694  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:24.822592  2304 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:24.764608  4669 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.830169  1718 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:24.832624  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:24.813107  1607 leader_election.cc:400] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9.
I20260709 13:02:24.766127  4491 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:24.765606  4610 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:24.828073  2304 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:24.834965  1532 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:24.840518  1533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.012s	sys 0.012s
W20260709 13:02:24.841516  1533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.012s	sys 0.013s
I20260709 13:02:24.835798  1607 leader_election.cc:403] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:02:24.836601  1718 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:24.827847  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:24.843477  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:24.844446  1643 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:24.724454  4538 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.852029  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:24.863292  4647 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:24.693400  4685 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:24.866755  4637 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:24.853135  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:24.894479  1532 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:24.897918  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.897784  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:24.899766  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 86807.8us
W20260709 13:02:24.911481  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 58390.6us
I20260709 13:02:24.912302  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:24.914494  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:24.841856  4669 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:24.848213  4256 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:24.852670  4668 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:02:24.940229  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 65132.2us
I20260709 13:02:24.927757  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:24.941286  1532 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:24.941582  1638 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:24.915280  1718 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:24.936581  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:24.850534  4689 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:24.887058  4512 consensus_meta.cc:287] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.657s	user 0.008s	sys 0.003s
I20260709 13:02:24.907241  4333 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9.
I20260709 13:02:24.958266  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:24.910369  4726 consensus_meta.cc:287] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.000s
I20260709 13:02:24.851404  4651 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 13:02:24.910719  4647 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.935109  4446 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:24.966619  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 52942.1us
I20260709 13:02:24.852789  4491 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.968070  1643 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:24.968689  1532 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:24.790622  4744 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:24.939952  4314 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:24.972733  1642 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:02:24.977783  1642 pending_rounds.cc:85] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 3
I20260709 13:02:24.978096  1642 pending_rounds.cc:107] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:02:24.978394  1642 raft_consensus.cc:2889] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:24.852236  4610 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:24.938983  4763 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:02:24.944433  4668 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.909430  4757 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 13:02:24.947477  4514 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:24.965935  4747 raft_consensus.cc:1064] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:02:24.957175  4415 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:24.928614  4048 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 13:02:24.963506  4512 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:24.984249  1638 raft_consensus.cc:2410] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:02:25.001566  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:24.974480  4333 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:24.978426  4691 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:24.980458  4446 raft_consensus.cc:697] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:02:24.990408  4649 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.605s	user 0.007s	sys 0.006s
I20260709 13:02:24.965723  4726 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:24.991892  4747 raft_consensus.cc:1068] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Unable to promote non-voter 6208c2fa49f643b5a50ab8f6d292fab2: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:24.984448  4645 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:24.992298  4761 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:24.990402  4661 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:24.985100  4668 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:24.990367  4741 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:24.975217  4647 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:24.977263  4546 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
W20260709 13:02:25.061153  4765 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:02:25.036963  4314 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:25.097775  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 139600us
I20260709 13:02:25.054255  4746 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.063894  4645 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.067102  4489 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:25.100546  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 115101us
I20260709 13:02:25.050005  4514 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:02:25.090747  4761 raft_consensus.cc:697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:25.095971  4635 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.055687  4649 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:25.126207  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:25.127924  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:25.113656  4645 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:25.130784  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:25.109681  4314 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.110001  4750 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:02:25.129993  4746 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:25.156490  1710 raft_consensus.cc:3055] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:25.160115  1710 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:25.164335  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:25.137838  4333 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:25.121728  4489 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.134562  4775 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:25.147114  4538 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:25.129112  4635 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.161036  1710 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:25.190877  4771 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
W20260709 13:02:25.189605  4762 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:02:25.125057  4446 consensus_queue.cc:237] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.196333  4538 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:25.201297  4538 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.201849  4314 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.203125  4726 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.197588  4610 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.206949  4647 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:25.207748  4649 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:25.211687  4777 log.cc:927] Time spent T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
W20260709 13:02:25.213121  4608 consensus_meta.cc:287] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.000s
I20260709 13:02:25.213618  4608 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:25.214496  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1936) took 1767 ms. Trace:
I20260709 13:02:25.214898  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:25.214869  1561 rpcz_store.cc:276] 0709 13:02:23.446857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.726046 (+279189us) service_pool.cc:224] Handling call
0709 13:02:23.726446 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.116187 (+389741us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:24.418315 (+302128us) log.cc:844] Serialized 14 byte log entry
0709 13:02:24.722590 (+304275us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.754836 (+ 32246us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.755030 (+   194us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:24.755077 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.755170 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.214304 (+459134us) raft_consensus.cc:1685] finished
0709 13:02:25.214323 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.214443 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16276,"prepare.queue_time_us":508230,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":10933,"raft.queue_time_us":188992,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":80617,"thread_start_us":936009,"threads_started":3,"wal-append.queue_time_us":622936}
I20260709 13:02:25.198629  4761 consensus_queue.cc:237] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:25.217800  4721 consensus_meta.cc:287] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.901s	user 0.012s	sys 0.001s
I20260709 13:02:25.218200  4721 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:25.193240  1710 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:25.194778  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:25.172219  4491 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.235746  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:25.233258  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 69857.8us
I20260709 13:02:25.243914  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:25.247743  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1947) took 1012 ms. Trace:
I20260709 13:02:25.220572  4721 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:25.206956  4664 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.216284  4726 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:25.233150  4628 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:25.249418  1567 rpcz_store.cc:276] 0709 13:02:24.235016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.236073 (+  1057us) service_pool.cc:224] Handling call
0709 13:02:24.260397 (+ 24324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.264847 (+  4450us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:24.440792 (+175945us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b18003ebde0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb1455 00007fd7a56993c7 00007fd7a1cab9a7 00007fd7a1cbb0bf 00007fd7a1cbb038
0709 13:02:24.883126 (+442334us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.011019 (+127893us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:25.089803 (+ 78784us) log.cc:844] Serialized 26 byte log entry
0709 13:02:25.090207 (+   404us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:25.090865 (+   658us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.109879 (+ 19014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.109999 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.247406 (+137407us) raft_consensus.cc:1685] finished
0709 13:02:25.247453 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.247652 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43998,"prepare.queue_time_us":429250,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":1826,"raft.queue_time_us":103939,"raft.run_cpu_time_us":519,"raft.run_wall_time_us":42713,"spinlock_wait_cycles":33219968,"thread_start_us":525436,"threads_started":2,"wal-append.queue_time_us":204710}
I20260709 13:02:25.232733  4635 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.267946  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:25.263123  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 134656us
I20260709 13:02:25.278192  1710 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:02:25.280215  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:25.290390  4678 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
W20260709 13:02:25.309456  1607 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 2.053s (SENT) [suppressed 9 similar messages]
W20260709 13:02:25.310573  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.310911  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.293309  4649 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:25.275597  4726 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.300340  4314 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:02:25.292788  4628 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:02:25.320693  4747 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:25.347362  4785 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:25.348069  4785 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:25.348831  4471 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:25.349062  4785 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:25.345494  4747 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.312131  4775 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
W20260709 13:02:25.355191  4204 consensus_meta.cc:287] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.861s	user 0.013s	sys 0.000s
I20260709 13:02:25.356040  1710 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:25.356781  4747 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.358676  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:25.361923  4645 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:02:25.363238  4778 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.240s	user 0.005s	sys 0.000s
W20260709 13:02:25.365444  4733 consensus_meta.cc:287] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.633s	user 0.008s	sys 0.004s
I20260709 13:02:25.375936  1636 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:25.376379  1636 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:25.377079  1636 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:25.377698  4645 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:25.379909  4726 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.381937  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 511) took 1350 ms. Trace:
I20260709 13:02:25.386795  4340 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:25.387543  4340 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:25.389492  4733 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:25.391942  4733 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:25.366803  4668 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.394743  4668 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.407152  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:25.413621  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:25.418475  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:25.382499  1563 rpcz_store.cc:276] 0709 13:02:24.031419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.031602 (+   183us) service_pool.cc:224] Handling call
0709 13:02:24.089004 (+ 57402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.165689 (+ 76685us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:24.394717 (+229028us) log.cc:844] Serialized 14 byte log entry
0709 13:02:24.696079 (+301362us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:24.696142 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:24.696208 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.380470 (+684262us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b1800387160. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb822a6 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419
0709 13:02:25.381593 (+  1123us) spinlock_profiling.cc:243] Waited 76.4 ms on lock 0x7b54000f6254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:25.381663 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.381839 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":458842,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":136,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":4506,"spinlock_wait_cycles":185072768,"thread_start_us":529552,"threads_started":2,"wal-append.queue_time_us":389762}
I20260709 13:02:25.423347  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:25.425884  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:25.427652  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:25.428236  1713 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:25.429147  1643 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:25.430720  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:25.431207  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:25.435595  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:25.436726  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:25.437522  1714 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:25.440217  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:25.455102  4747 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:25.457166  4746 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:25.427291  4461 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:25.457652  4746 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:25.451053  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:25.462981  2298 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:25.463478  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:25.464289  1714 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:25.365604  4669 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.482228  1561 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:02:25.458112  4461 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:25.487159  1467 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60118 (request call id 229) took 15476 ms (client timeout 15000 ms). Trace:
I20260709 13:02:25.522764  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:25.524077  1714 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:25.429558  4733 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:02:25.526012  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:25.530656  1565 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:25.531505  1682 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:25.536970  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:25.525527  4789 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.458592  4204 consensus_queue.cc:1059] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:02:25.479357  4725 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:25.487733  1467 rpcz_store.cc:269] 0709 13:02:09.990080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.041521 (+ 51441us) service_pool.cc:224] Handling call
0709 13:02:25.466333 (+15424812us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:19.864826 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:19.865572 (+   746us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:19.865628 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:02:19.865649 (+    21us) tablet.cc:662] Decoding operations
0709 13:02:19.866572 (+   923us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:19.866651 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:19.866676 (+    25us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:02:19.867472 (+   796us) tablet.cc:701] Row locks acquired
0709 13:02:19.867516 (+    44us) write_op.cc:260] PREPARE: finished
0709 13:02:19.867693 (+   177us) write_op.cc:270] Start()
0709 13:02:19.867879 (+   186us) write_op.cc:276] Timestamp: P: 1783602139867623 usec, L: 0
0709 13:02:19.867899 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:02:19.868647 (+   748us) log.cc:844] Serialized 2835 byte log entry
0709 13:02:20.734795 (+866148us) op_driver.cc:464] REPLICATION: finished
0709 13:02:20.819545 (+ 84750us) write_op.cc:301] APPLY: starting
0709 13:02:20.819649 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:20.819853 (+   204us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:20.819977 (+   124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:20.823091 (+  3114us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:20.849641 (+ 26550us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:02:20.849736 (+    95us) write_op.cc:312] APPLY: finished
0709 13:02:20.889578 (+ 39842us) log.cc:844] Serialized 59 byte log entry
0709 13:02:20.889809 (+   231us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:20.889960 (+   151us) write_op.cc:454] Released schema lock
0709 13:02:20.890055 (+    95us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:20.895372 (+  5317us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:20.895408 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":915700,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":71861,"apply.run_cpu_time_us":3981,"apply.run_wall_time_us":75794,"num_ops":7,"prepare.queue_time_us":150948,"prepare.run_cpu_time_us":12732,"prepare.run_wall_time_us":136108,"raft.queue_time_us":370177,"raft.run_cpu_time_us":29596,"raft.run_wall_time_us":107266,"replication_time_us":834014,"thread_start_us":592370,"threads_started":4,"wal-append.queue_time_us":244589,"wal-append.run_cpu_time_us":27215,"wal-append.run_wall_time_us":1766980}]]}
I20260709 13:02:25.557463  1713 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:25.558022  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:25.559067  1713 pending_rounds.cc:85] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 5
I20260709 13:02:25.559615  1713 pending_rounds.cc:107] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:02:25.560235  1713 raft_consensus.cc:2889] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:25.559464  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.529964  4738 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.539348  4649 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:25.574034  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:25.575457  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:25.575935  1565 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:25.576953  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:25.577571  1565 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:25.579334  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:25.579381  2298 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:25.580107  1567 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:25.583513  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:25.594684  4805 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:02:25.597231  4803 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:02:25.599340  1711 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:02:25.575083  4810 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.602339  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.602769  4810 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.603649  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.604358  1564 raft_consensus.cc:2393] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 6: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:25.563405  4725 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.605038  4810 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.605662  4810 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:25.606267  4810 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:25.606956  4725 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.573318  4738 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.609263  4738 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.578814  4471 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:25.611816  4669 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:25.612347  4461 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:25.612432  4746 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:25.613778  4415 consensus_meta.cc:287] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.620s	user 0.013s	sys 0.002s
I20260709 13:02:25.613972  4746 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:25.614229  4415 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:25.615211  1566 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:25.616391  1638 raft_consensus.cc:2410] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:25.553507  4789 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.605091  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.618440  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:25.645336  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:25.649354  4796 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.655479  4796 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.658301  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 1943) took 1948 ms. Trace:
I20260709 13:02:25.658947  1712 rpcz_store.cc:276] 0709 13:02:23.709619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.944994 (+235375us) service_pool.cc:224] Handling call
0709 13:02:23.945357 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.351372 (+406015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.657977 (+1306605us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54001482d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:25.658214 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":836046208}
I20260709 13:02:25.618260  1564 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:25.674403  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:25.675513  1718 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:25.676551  1565 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:25.602487  1711 raft_consensus.cc:2955] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:02:25.707587  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.582465  4649 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:25.735430  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 160058us
I20260709 13:02:25.662374  4770 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:25.738814  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.619602  4810 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:25.721876  4738 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:25.742336  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1934) took 2296 ms. Trace:
I20260709 13:02:25.747076  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.695277  4796 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.748481  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:25.650620  4471 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 13:02:25.749284  1564 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:25.754952  1682 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:25.749455  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.756667  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:25.749677  1568 rpcz_store.cc:276] 0709 13:02:23.446351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.723252 (+276901us) service_pool.cc:224] Handling call
0709 13:02:23.761966 (+ 38714us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.287704 (+525738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.741281 (+1453577us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b5400200654. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:25.742273 (+   992us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1150251904}
I20260709 13:02:25.750993  1566 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:25.758773  4314 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:25.760697  4314 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:25.731909  4340 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:25.772651  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:25.756613  4761 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.775022  1641 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:25.761292  4812 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:02:25.784260  4747 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:25.790659  4810 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:25.768453  4610 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:25.751351  4204 consensus_queue.cc:1059] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:02:25.815698  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:25.817881  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:25.771950  4415 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:25.774801  4314 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:25.823282  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.755105  4738 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.825692  1564 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:25.821691  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:25.775614  4733 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:25.828337  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:25.758006  4770 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:25.831331  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.830570  1641 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:25.840806  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.846315  1568 raft_consensus.cc:2393] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:25.797282  4761 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.837128  4628 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:25.858095  4789 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.858712  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.860409  1568 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:25.833420  4744 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:25.862707  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:25.863878  1568 raft_consensus.cc:2368] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6.
I20260709 13:02:25.858536  4747 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:02:25.864759  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:25.865446  1568 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:25.866226  4761 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.866194  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.826431  4610 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.845293  1682 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:25.875068  4812 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:25.875561  4812 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:25.875905  4812 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:25.876034  4770 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:25.871316  4628 raft_consensus.cc:697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:25.877856  4628 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:25.852283  4814 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:02:25.882081  4743 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:25.834766  4415 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.841660  4733 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:25.889562  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.879184  1567 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:25.903199  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.903965  1567 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:25.904611  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:25.905973  1568 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:25.871168  4403 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:25.916296  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.919296  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.913784  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:25.933183  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 93178.4us
I20260709 13:02:25.935114  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.868775  4744 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:25.937353  1561 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:25.914259  4403 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:25.886222  4743 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:25.942009  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:25.944175  1567 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:25.951725  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:25.952531  1560 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:25.959200  1568 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:25.959235  1560 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:25.962958  1567 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:02:25.953177  4340 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:25.974496  4831 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:25.977211  4489 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:25.984089  4610 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.988287  4743 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:25.989533  4415 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:25.990727  4733 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:25.972049  4816 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:02:25.999963  1567 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:02:25.999990  1564 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:26.001259  1560 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:25.973631  4403 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:26.006170  4403 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:26.006798  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 922) took 2258 ms. Trace:
I20260709 13:02:26.007223  2302 rpcz_store.cc:276] 0709 13:02:23.748181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.748451 (+   270us) service_pool.cc:224] Handling call
0709 13:02:23.806622 (+ 58171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.402734 (+596112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.006561 (+1603827us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b54005dc0d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:26.006729 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1369230720}
I20260709 13:02:26.007892  1567 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: [4.3-7.4]
I20260709 13:02:25.982208  4738 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:26.009305  1567 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:02:26.016211  4661 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:26.019940  4661 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:26.020804  4812 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:26.025419  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:26.025846  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.026132  1561 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:26.027081  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:26.027968  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:26.028749  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:26.029183  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:25.981424  4649 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:26.030988  4666 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:26.032780  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:26.028080  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:26.026283  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.031989  4744 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:25.959299  4823 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:02:26.040525  1642 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:26.044059  1564 raft_consensus.cc:2955] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 3 to 6, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:26.039155  4649 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:26.057979  4781 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.905s	user 0.000s	sys 0.003s
I20260709 13:02:26.071671  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 878) took 1371 ms. Trace:
I20260709 13:02:26.072161  2297 rpcz_store.cc:276] 0709 13:02:24.738506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.738657 (+   151us) service_pool.cc:224] Handling call
0709 13:02:24.777377 (+ 38720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.934821 (+157444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.934878 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:24.934975 (+    97us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:24.935030 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.071422 (+1136392us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54006414d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:26.071470 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.071605 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350465408}
I20260709 13:02:26.074617  1713 pending_rounds.cc:85] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 4
I20260709 13:02:26.075184  1713 pending_rounds.cc:107] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:02:26.075479  1713 raft_consensus.cc:2889] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:26.076611  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:26.078559  1712 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:02:26.081249  4635 raft_consensus.cc:3055] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:26.081774  4635 raft_consensus.cc:740] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:26.044543  4746 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:26.094208  1609 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:26.096259  4823 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:26.096517  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:26.097432  1715 raft_consensus.cc:2410] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:02:26.039333  4788 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:02:26.099133  4661 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:26.106906  2299 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.233s	user 0.020s	sys 0.032s
I20260709 13:02:26.047616  4744 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:02:26.048462  4812 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:26.103256  4726 raft_consensus.cc:3055] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:02:26.172621  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 220976us
I20260709 13:02:26.179297  4747 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:26.179929  4747 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:26.181422  4760 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 1.357s	user 0.005s	sys 0.000s
I20260709 13:02:26.093221  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:26.121398  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:26.134142  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:26.185205  1637 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.185271  1644 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:26.186395  1712 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:26.134833  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:26.187534  1641 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 13:02:26.136291  2299 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.262s	user 0.032s	sys 0.032s
I20260709 13:02:26.136253  1532 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:26.098603  4538 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:26.189410  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 53302.8us
I20260709 13:02:26.103734  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:26.190336  1635 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:26.138690  1718 raft_consensus.cc:1240] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6. Ops: []
W20260709 13:02:26.112092  4828 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:02:26.149462  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:26.193030  1640 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:26.150355  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:26.155220  4635 consensus_queue.cc:260] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:26.189670  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:26.199383  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:26.156302  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 615) took 1001 ms. Trace:
I20260709 13:02:26.091884  4649 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.159873  4788 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.135465  4746 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.244159  1561 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:26.194456  1715 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
I20260709 13:02:26.201195  1716 rpcz_store.cc:276] 0709 13:02:25.155252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.155447 (+   195us) service_pool.cc:224] Handling call
0709 13:02:25.155929 (+   482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:25.210563 (+ 54634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:25.210595 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:25.211258 (+   663us) log.cc:844] Serialized 26 byte log entry
0709 13:02:25.284830 (+ 73572us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:25.284899 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.155842 (+870943us) spinlock_profiling.cc:243] Waited 54.3 ms on lock 0x7b54003395d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:26.155896 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.155948 (+    52us) raft_consensus.cc:1685] finished
0709 13:02:26.155966 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.156225 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"spinlock_wait_cycles":124811008,"thread_start_us":73196,"threads_started":1,"wal-append.queue_time_us":129455}
W20260709 13:02:26.205437  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:26.291901  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.292991  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:26.293032  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:26.294200  1607 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:26.201778  1533 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:26.295177  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 93477.7us
I20260709 13:02:26.295279  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:26.298220  1533 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:26.210813  4661 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:26.313066  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:26.314663  4649 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:26.321380  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:26.212064  4635 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:26.216326  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:26.332188  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.332502  4661 consensus_queue.cc:1059] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:26.333422  4661 raft_consensus.cc:3055] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:26.333760  4661 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:26.245534  1532 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:02:26.246389  4835 log.cc:927] Time spent T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:02:26.334681  4661 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.161170  4314 raft_consensus.cc:3055] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:26.336073  4645 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:26.336172  4314 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:26.336660  4645 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.337108  4645 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.246819  4834 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.337299  4314 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.253933  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 876) took 1558 ms. Trace:
I20260709 13:02:26.255224  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:26.338980  4314 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:26.340132  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:26.340775  1643 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:26.336681  4661 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:26.341689  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:26.342362  1643 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:26.250351  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:26.256283  4538 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:26.343509  4538 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.349254  4685 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.354525  4645 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.355466  1636 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:26.359297  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:26.362285  4645 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:26.365607  4661 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:26.366340  4661 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:26.369766  2268 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:26.277580  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.171110  4726 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:26.374739  4726 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.376000  4726 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:26.376549  4741 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:26.377112  4741 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:26.377619  4741 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:26.279614  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:26.379333  1642 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:26.207795  4491 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:26.383459  4491 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:26.384775  4491 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:26.282388  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:26.388608  1563 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:26.388729  4741 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.390162  1563 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:26.376608  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:26.372702  4661 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.391993  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:26.392395  4446 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:26.243453  4725 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:26.283808  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:26.287287  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:26.287585  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:26.413234  1566 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.338653  2299 rpcz_store.cc:276] 0709 13:02:24.695573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.696607 (+  1034us) service_pool.cc:224] Handling call
0709 13:02:24.697098 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.825532 (+128434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.825575 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.874053 (+ 48478us) log.cc:844] Serialized 26 byte log entry
0709 13:02:26.188576 (+1314523us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:26.188621 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.253447 (+ 64826us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54006077d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:26.253523 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.253588 (+    65us) raft_consensus.cc:1685] finished
0709 13:02:26.253600 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.253856 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294670080,"thread_start_us":1314182,"threads_started":1,"wal-append.queue_time_us":488773,"wal-append.run_cpu_time_us":1726,"wal-append.run_wall_time_us":1001960}
I20260709 13:02:26.288815  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:26.308838  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:26.309479  4048 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:26.310786  4544 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:26.311264  4744 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:26.313016  4737 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:26.331015  4792 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:02:26.334962  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 89485.5us
I20260709 13:02:26.373171  4747 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:26.392789  4757 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:26.393903  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:26.263316  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:26.371034  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:26.282886  4746 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:26.400972  2269 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:26.259099  4809 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.660s	user 0.002s	sys 0.002s
I20260709 13:02:26.402632  1568 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:26.403573  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:26.346570  4538 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:26.416458  1569 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.468644  4610 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:02:26.338240  4834 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:26.439056  4741 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:26.479878  4741 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:26.480312  4741 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.480676  4741 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:26.430229  4048 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:26.432919  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 124163us
I20260709 13:02:26.487629  1607 leader_election.cc:400] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6.
I20260709 13:02:26.487993  1607 leader_election.cc:403] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:02:26.434947  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:26.489825  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:26.490166  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:26.440189  4505 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:26.496448  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:26.499447  1566 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:26.502570  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:26.503530  1639 raft_consensus.cc:2393] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:26.503624  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:26.439549  4740 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:26.431999  4785 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.411391  4446 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:26.429939  4725 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.439873  4314 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:26.528815  4314 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.528882  4834 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:26.529295  4314 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:26.485563  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:26.428987  4788 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.440928  4737 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:26.515789  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:26.427615  1642 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:26.465781  1636 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:02:26.521188  1569 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:26.495184  4048 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:26.470739  4757 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.442096  4770 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:02:26.526152  4733 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:02:26.527896  4446 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:26.526286  4747 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:26.561357  1561 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 4.
I20260709 13:02:26.561527  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:26.528308  4725 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:26.537959  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:26.504647  4538 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:26.549345  1565 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:02:26.552480  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.524051  4649 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:02:26.521873  1637 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:26.540268  4786 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.554787  4757 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:26.539377  1636 pending_rounds.cc:85] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 4
I20260709 13:02:26.590516  1636 pending_rounds.cc:107] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 6.5
W20260709 13:02:26.591491  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 197654us
I20260709 13:02:26.456316  4792 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.534260  4737 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:26.524685  4836 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:26.578352  1563 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:26.553843  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:26.526080  4505 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.539805  4628 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.592789  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:26.527180  4740 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.531810  4785 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.564735  1569 raft_consensus.cc:1217] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Deduplicated request from leader. Original: 5.4->[6.5-6.5]   Dedup: 6.5->[]
I20260709 13:02:26.586989  4544 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 5. Current term is 6.
I20260709 13:02:26.540270  4831 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:26.612531  2297 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:26.613241  1569 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:02:26.591872  1636 raft_consensus.cc:2889] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:26.620710  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1857) took 2543 ms. Trace:
I20260709 13:02:26.621239  1636 rpcz_store.cc:276] 0709 13:02:24.077602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.079280 (+  1678us) service_pool.cc:224] Handling call
0709 13:02:24.079704 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.375835 (+1296131us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54003c7420. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c18dc0 00007fd7a1ca31c0 00007fd7a1ca4231 00007fd7a1ca0d8f 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c
0709 13:02:26.355068 (+979233us) spinlock_profiling.cc:243] Waited 976 ms on lock 0x7b58004abad0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1bfd0aa 00007fd7a1c7f447 00007fd7a1c97fe7 00007fd7a1c997de 00007fd7a1ca3b49 00007fd7a1ca4416 00007fd7a1ca0d8f 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4
0709 13:02:26.620306 (+265238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.620626 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2582268672}
I20260709 13:02:26.607545  4836 raft_consensus.cc:697] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:26.603241  4786 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:26.650888  4650 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
W20260709 13:02:26.661571  4838 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:02:26.668571  4792 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:26.608767  4628 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:26.673029  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2324) took 1058 ms. Trace:
I20260709 13:02:26.677749  1564 rpcz_store.cc:276] 0709 13:02:25.614779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.001258 (+386479us) service_pool.cc:224] Handling call
0709 13:02:26.001488 (+   230us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.039534 (+ 38046us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:26.093567 (+ 54033us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.448031 (+354464us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:26.468592 (+ 20561us) log.cc:844] Serialized 26 byte log entry
0709 13:02:26.468724 (+   132us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:26.494806 (+ 26082us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.497428 (+  2622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.497499 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.672763 (+175264us) raft_consensus.cc:1685] finished
0709 13:02:26.672796 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.672945 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16284,"prepare.queue_time_us":25874,"raft.queue_time_us":29721,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":38730,"spinlock_wait_cycles":34659200,"thread_start_us":272744,"threads_started":1,"wal-append.queue_time_us":374208}
W20260709 13:02:26.686441  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 78560.8us
I20260709 13:02:26.687247  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:26.693732  1637 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:26.674616  4845 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:26.672443  4770 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:02:26.645419  4446 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.731283  4836 consensus_queue.cc:237] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:26.732374  4628 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:26.738397  1638 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:02:26.753269  1638 pending_rounds.cc:85] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 4
I20260709 13:02:26.753530  1638 pending_rounds.cc:107] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:02:26.753765  1638 raft_consensus.cc:2889] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:26.754136  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 1946) took 2519 ms. Trace:
I20260709 13:02:26.739573  1643 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:26.755134  4786 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:26.744197  4785 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:26.702069  4844 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:26.759013  4785 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.759779  4785 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:26.754509  1562 rpcz_store.cc:276] 0709 13:02:24.234572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.235206 (+   634us) service_pool.cc:224] Handling call
0709 13:02:24.235664 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.243459 (+  7795us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:26.112379 (+1868920us) spinlock_profiling.cc:243] Waited 7.19 ms on lock 0x7b18003870a0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb1455 00007fd7a56993c7 00007fd7a1cab9a7 00007fd7a1cbb0bf
0709 13:02:26.112545 (+   166us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.417724 (+305179us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:26.430172 (+ 12448us) log.cc:844] Serialized 26 byte log entry
0709 13:02:26.430357 (+   185us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:26.439290 (+  8933us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.439524 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.439616 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.531391 (+ 91775us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b18003870a0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:26.753254 (+221863us) raft_consensus.cc:1685] finished
0709 13:02:26.753277 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.753426 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7272,"prepare.queue_time_us":415193,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":1899,"raft.queue_time_us":24298,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":42049,"spinlock_wait_cycles":37574528,"thread_start_us":703064,"threads_started":2,"wal-append.queue_time_us":443409}
I20260709 13:02:26.761858  4785 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:26.764813  4740 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:26.766121  4538 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:26.751998  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 54125.5us
W20260709 13:02:26.767695  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.771749  1533 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:26.773587  1533 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:26.775161  4761 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:26.775689  4761 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:26.776149  4761 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:26.776288  4795 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:26.776906  4795 raft_consensus.cc:697] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:02:26.778793  4848 log.cc:927] Time spent T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:02:26.779345  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:26.780249  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:26.780772  1641 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:02:26.781159  1641 pending_rounds.cc:85] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 3
W20260709 13:02:26.754485  4839 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:02:26.781387  1641 pending_rounds.cc:107] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:02:26.767534  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 617) took 1509 ms. Trace:
I20260709 13:02:26.781661  1641 raft_consensus.cc:2889] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:26.770398  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:26.781762  1710 rpcz_store.cc:276] 0709 13:02:25.258545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.279647 (+ 21102us) service_pool.cc:224] Handling call
0709 13:02:26.767360 (+1487713us) spinlock_profiling.cc:243] Waited 74.8 ms on lock 0x7b540033a4d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:26.767473 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171976064}
W20260709 13:02:26.746811  4817 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.899s	user 0.002s	sys 0.001s
I20260709 13:02:26.782707  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1809) took 1211 ms. Trace:
I20260709 13:02:26.784442  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 1971) took 1376 ms. Trace:
I20260709 13:02:26.783146  1711 rpcz_store.cc:276] 0709 13:02:25.571684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.601786 (+ 30102us) service_pool.cc:224] Handling call
0709 13:02:25.602106 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:25.602340 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:25.746135 (+143795us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.901378 (+155243us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.154737 (+253359us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b1800328c60. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:26.364446 (+209709us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b1800328c60. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb4522 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:26.782277 (+417831us) spinlock_profiling.cc:243] Waited 4.59 ms on lock 0x7b1800328c60. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb2ec0a 00007fd7a826e219 00007fd79ebb6a40 00007fd79ebb14e4 00007fd79ebb1455 00007fd7a1c99060 00007fd7a1cab716 00007fd7a1cbc287 00007fd7a1cbc208
0709 13:02:26.782405 (+   128us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.782463 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:26.782504 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.782564 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.782660 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1783,"prepare.queue_time_us":1977,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":883954,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":4420,"spinlock_wait_cycles":28565888,"thread_start_us":154585,"threads_started":1,"wal-append.queue_time_us":187113}
I20260709 13:02:26.760941  4741 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:26.793599  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:26.794245  2298 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:26.796129  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:26.784811  1709 rpcz_store.cc:276] 0709 13:02:25.408412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:25.582073 (+173661us) service_pool.cc:224] Handling call
0709 13:02:25.591364 (+  9291us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:25.651348 (+ 59984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:25.651396 (+    48us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:25.721842 (+ 70446us) log.cc:844] Serialized 78 byte log entry
0709 13:02:25.835020 (+113178us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:25.847595 (+ 12575us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.849806 (+  2211us) log.cc:844] Serialized 14 byte log entry
0709 13:02:25.942874 (+ 93068us) log.cc:844] Serialized 14 byte log entry
0709 13:02:26.271926 (+329052us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b18001276c0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb822a6 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1
0709 13:02:26.272536 (+   610us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b18001276c0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:26.360154 (+ 87618us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b18001276c0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:26.784130 (+423976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:26.784212 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:26.784260 (+    48us) raft_consensus.cc:1685] finished
0709 13:02:26.784272 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:26.784390 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":902230,"prepare.queue_time_us":92067,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":102,"raft.queue_time_us":74910,"raft.run_cpu_time_us":430,"raft.run_wall_time_us":73795,"spinlock_wait_cycles":44922368,"thread_start_us":177274,"threads_started":3,"wal-append.queue_time_us":136452}
I20260709 13:02:27.008893  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:27.012697  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 216105us
I20260709 13:02:27.013588  1609 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:27.014276  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.014756  2300 raft_consensus.cc:2410] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:02:27.015321  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 211134us
I20260709 13:02:27.056332  2298 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:26.786253  4415 raft_consensus.cc:3055] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:02:27.070322  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 47417.1us
I20260709 13:02:27.066519  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:26.793085  4487 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:27.070228  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.032s	sys 0.065s
W20260709 13:02:27.094532  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.032s	sys 0.065s
W20260709 13:02:27.096411  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 695540us
I20260709 13:02:27.097939  2269 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:27.109699  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:27.109894  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1812) took 1037 ms. Trace:
I20260709 13:02:27.110410  1714 rpcz_store.cc:276] 0709 13:02:26.072052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.088553 (+ 16501us) service_pool.cc:224] Handling call
0709 13:02:26.089028 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.089313 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:26.089698 (+   385us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.109468 (+1019770us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:27.109539 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:27.109601 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.109698 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.109839 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1019453,"prepare.queue_time_us":168,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42}
I20260709 13:02:27.109159  4810 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:02:27.022464  4685 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:02:27.131744  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:27.132436  2301 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:27.122162  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:27.069996  4546 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.126439  2269 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:27.051512  4747 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:02:27.137362  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:27.070879  4415 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:27.067581  4750 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:02:27.048611  4666 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:02:27.107875  4831 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.168661  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:27.173390  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:27.135460  4740 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:27.163630  4744 consensus_meta.cc:287] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.722s	user 0.011s	sys 0.000s
I20260709 13:02:27.175307  2269 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:02:27.172271  4314 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.638s	user 0.011s	sys 0.000s
W20260709 13:02:27.196698  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 134866us
W20260709 13:02:27.200250  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.152315  4668 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:27.203676  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 219953us
I20260709 13:02:27.151583  4747 consensus_queue.cc:1243] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:27.202147  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:27.180840  4461 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.169245  4770 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:27.212168  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 140806us
I20260709 13:02:27.151723  4546 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:27.179215  4649 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.198769  4713 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.199226  4744 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:27.221815  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:27.220045  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 44857us
I20260709 13:02:27.173661  4489 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:02:27.110522  4795 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.181427  4740 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.220806  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:27.174077  4831 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.229439  4814 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:02:27.234264  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 219368us
I20260709 13:02:27.235083  1681 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:27.232125  2299 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:27.233495  1636 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:27.237308  1635 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:02:27.237843  1635 pending_rounds.cc:85] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 6
I20260709 13:02:27.238161  1635 pending_rounds.cc:107] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:02:27.238484  1635 raft_consensus.cc:2889] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:27.236296  1681 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:27.221740  4678 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:27.240079  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:27.204071  4314 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.260708  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:27.225307  4538 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:27.252899  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:27.210314  4668 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.249079  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:27.205878  4822 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:02:27.230445  4770 raft_consensus.cc:697] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:27.232381  4806 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.268399  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:27.233814  4740 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:27.269824  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:27.277909  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:27.278390  1635 raft_consensus.cc:2393] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:27.211323  4461 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:27.232475  4847 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:27.225082  4810 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:02:27.241633  4726 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.225087  4649 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:27.291975  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:27.284900  1567 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:02:27.299688  4858 log.cc:927] Time spent T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
W20260709 13:02:27.300143  4856 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 13:02:27.300880  4859 log.cc:927] Time spent T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:02:27.301611  4461 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:27.259735  4635 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:27.296916  1637 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:27.241245  4468 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:27.241021  4471 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:27.219270  4415 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:27.263233  4741 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:27.312570  1643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.001s	sys 0.001s
W20260709 13:02:27.320227  1643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.002s	sys 0.001s
W20260709 13:02:27.311476  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 50728.8us
W20260709 13:02:27.299780  4849 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.515s	user 0.001s	sys 0.000s
I20260709 13:02:27.292761  4538 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:27.259960  4678 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.301996  4813 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:27.330329  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 84990us
I20260709 13:02:27.225463  4713 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.355505  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:27.313176  4461 raft_consensus.cc:2764] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:02:27.347425  1714 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:27.350060  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:27.352792  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:27.354584  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:27.357059  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:27.362486  1714 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:27.242821  4814 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:27.364789  1712 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:27.354063  4822 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:27.373157  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:27.330941  4721 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:27.310736  4726 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:27.373291  1681 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:27.373680  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:27.373237  1711 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
W20260709 13:02:27.347955  4725 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.755s	user 0.011s	sys 0.000s
I20260709 13:02:27.310842  4669 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:02:27.316589  4750 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:02:27.355006  4685 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:27.324481  4468 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.325131  4471 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.436936  1717 raft_consensus.cc:2410] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:02:27.374233  4489 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:27.446719  2298 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
W20260709 13:02:27.448889  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 196506us
I20260709 13:02:27.290182  4806 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:27.453219  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 80011.1us
W20260709 13:02:27.453022  4857 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.233s	user 0.006s	sys 0.007s
I20260709 13:02:27.457073  1681 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:27.457736  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:27.458370  1681 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:27.458683  4866 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:27.459242  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1813) took 1386 ms. Trace:
I20260709 13:02:27.459383  4866 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:27.459646  1713 rpcz_store.cc:276] 0709 13:02:26.072378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.073603 (+  1225us) service_pool.cc:224] Handling call
0709 13:02:26.074217 (+   614us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:27.123453 (+1049236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.123483 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:27.174378 (+ 50895us) log.cc:844] Serialized 616 byte log entry
0709 13:02:27.282814 (+108436us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:27.282886 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.283171 (+   285us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:27.458995 (+175824us) raft_consensus.cc:1685] finished
0709 13:02:27.459028 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.459178 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1063148,"raft.queue_time_us":1061116,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":13321,"thread_start_us":69403,"threads_started":1,"wal-append.queue_time_us":163245}
I20260709 13:02:27.446990  4871 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:27.461376  4871 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.463856  4871 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:27.446962  4461 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:27.463851  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:27.465324  4461 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.466777  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:27.387749  4847 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:02:27.467679  4862 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:27.469095  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:27.469907  4862 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:27.471897  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:27.458403  1715 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:27.471316  4489 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.383690  4822 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:27.483480  1718 raft_consensus.cc:2410] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:27.388024  4861 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:02:27.532181  4737 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.925s	user 0.009s	sys 0.002s
I20260709 13:02:27.524782  4738 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:02:27.390617  4721 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:27.534824  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 53706.8us
I20260709 13:02:27.388166  4725 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.479655  4862 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:27.525058  4813 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:27.544078  1681 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:27.379099  4770 consensus_queue.cc:237] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:27.534421  4757 consensus_meta.cc:287] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.927s	user 0.010s	sys 0.001s
I20260709 13:02:27.532665  4810 consensus_queue.cc:1243] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:27.565727  4314 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:27.566792  1710 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:27.544260  4546 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:27.578378  4725 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:27.590378  4879 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:27.590663  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:27.590991  4879 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.591709  1711 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 4.
W20260709 13:02:27.571681  4865 log.cc:927] Time spent T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:02:27.572348  4668 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:27.447652  4685 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.577121  4678 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:27.578318  4468 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:27.605559  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:27.605888  4713 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:27.579675  4806 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:27.580567  4726 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:27.585989  4761 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.809s	user 0.008s	sys 0.003s
I20260709 13:02:27.608697  4761 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.609925  1560 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:27.611057  4761 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:27.586397  4831 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:27.589192  4738 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:27.574944  4757 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.593191  4879 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:27.570061  1566 raft_consensus.cc:2368] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8.
I20260709 13:02:27.597765  1715 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:27.534114  4737 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.606206  1711 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:27.527907  4538 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.607023  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2325) took 1272 ms. Trace:
I20260709 13:02:27.602036  4813 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:27.607766  4685 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:27.607815  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 63882.5us
I20260709 13:02:27.616135  4831 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.612485  4738 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.637233  4831 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:27.630875  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1878) took 1454 ms. Trace:
I20260709 13:02:27.641402  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:27.641660  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:27.641987  1715 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:27.624576  1640 rpcz_store.cc:276] 0709 13:02:26.334151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.788290 (+454139us) service_pool.cc:224] Handling call
0709 13:02:27.045907 (+257617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.077125 (+ 31218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.077180 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:27.077871 (+   691us) log.cc:844] Serialized 26 byte log entry
0709 13:02:27.259284 (+181413us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:27.259343 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.259445 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:27.606782 (+347337us) raft_consensus.cc:1685] finished
0709 13:02:27.606800 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.606963 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7138,"raft.queue_time_us":92503,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":14973,"thread_start_us":146993,"threads_started":1,"wal-append.queue_time_us":201425}
I20260709 13:02:27.632658  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1888) took 1321 ms. Trace:
I20260709 13:02:27.643038  1641 rpcz_store.cc:276] 0709 13:02:26.311113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.782537 (+471424us) service_pool.cc:224] Handling call
0709 13:02:26.786381 (+  3844us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.786802 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:26.786831 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:27.008102 (+221271us) log.cc:844] Serialized 26 byte log entry
0709 13:02:27.452470 (+444368us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:27.452510 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.454431 (+  1921us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:27.632451 (+178020us) raft_consensus.cc:1685] finished
0709 13:02:27.632471 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.632601 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2105,"raft.queue_time_us":6455,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":6163,"spinlock_wait_cycles":41088,"thread_start_us":313104,"threads_started":1,"wal-append.queue_time_us":531603}
I20260709 13:02:27.644718  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:27.647228  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:27.647848  2298 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:27.648752  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:27.649461  1715 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:27.640374  1638 rpcz_store.cc:276] 0709 13:02:26.176184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.525098 (+348914us) service_pool.cc:224] Handling call
0709 13:02:26.540718 (+ 15620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.754274 (+213556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.630517 (+876243us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54001950d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:27.630781 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":453511552}
I20260709 13:02:27.649811  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:27.651157  1681 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:27.653088  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:27.653914  2298 raft_consensus.cc:2393] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 9: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:27.659876  1609 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:27.669409  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:27.670140  1715 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:27.741134  1565 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:27.754889  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:27.639138  4505 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:27.749655  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 98578.8us
I20260709 13:02:27.763664  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:27.748404  1711 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:27.766155  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:27.639138  4738 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:27.633932  4814 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:27.698158  4489 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:27.781088  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:27.615828  4822 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.737324  4544 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:27.784709  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:27.803793  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 738) took 1523 ms. Trace:
I20260709 13:02:27.810397  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:27.810315  2297 rpcz_store.cc:276] 0709 13:02:26.249930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.250073 (+   143us) service_pool.cc:224] Handling call
0709 13:02:27.773296 (+1523223us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54002a44d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:27.773407 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":619229440}
I20260709 13:02:27.755940  4813 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:27.737716  4650 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:27.750190  4806 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:02:27.811203  1610 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.000s	sys 0.018s
W20260709 13:02:27.811473  1568 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.013s	sys 0.005s
I20260709 13:02:27.811686  1640 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:27.827134  1568 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.013s	sys 0.005s
I20260709 13:02:27.814759  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:27.813422  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:27.783936  4048 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.215s	user 0.008s	sys 0.005s
I20260709 13:02:27.739094  4713 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:02:27.830165  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:27.839586  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:27.842343  1635 raft_consensus.cc:2393] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:27.803190  4879 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:27.825059  1610 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.000s	sys 0.018s
W20260709 13:02:27.844352  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 80773.2us
I20260709 13:02:27.817137  4505 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.847972  1681 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:27.851814  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 616) took 1668 ms. Trace:
I20260709 13:02:27.855479  1639 rpcz_store.cc:276] 0709 13:02:26.183415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.525369 (+341954us) service_pool.cc:224] Handling call
0709 13:02:26.525675 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.578259 (+ 52584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.578286 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:26.578321 (+    35us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:26.578350 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.851561 (+1273211us) spinlock_profiling.cc:243] Waited 52.4 ms on lock 0x7b5400461ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:27.851604 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.851745 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120543616}
I20260709 13:02:27.864488  1640 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:27.817296  4650 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:27.871984  1562 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.067s	sys 0.139s
W20260709 13:02:27.872537  1562 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.068s	sys 0.139s
I20260709 13:02:27.827860  4831 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:27.870206  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:27.876185  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1886) took 1565 ms. Trace:
I20260709 13:02:27.876379  1681 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:27.876698  1643 rpcz_store.cc:276] 0709 13:02:26.310448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.764997 (+454549us) service_pool.cc:224] Handling call
0709 13:02:26.765683 (+   686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.766116 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:26.766704 (+   588us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.476879 (+710175us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.526320 (+ 49441us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:27.526829 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 13:02:27.636575 (+109746us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:27.636615 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.636675 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:27.875971 (+239296us) raft_consensus.cc:1685] finished
0709 13:02:27.875993 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:27.876129 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":182339,"prepare.queue_time_us":242875,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":1410,"raft.queue_time_us":65824,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":24763,"spinlock_wait_cycles":42900864,"thread_start_us":679964,"threads_started":2,"wal-append.queue_time_us":707887}
I20260709 13:02:27.857381  4721 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:27.866532  4806 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:27.844647  4750 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:02:27.817008  4544 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.889344  4726 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:27.874452  4650 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:27.845486  4048 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.882181  4815 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:27.876256  4813 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:02:27.897384  4879 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:27.904232  4822 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:27.940992  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.005s	sys 0.013s
W20260709 13:02:27.941306  1564 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.011s	sys 0.008s
W20260709 13:02:28.001609  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.009s	sys 0.013s
I20260709 13:02:27.873642  4505 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:27.910181  4774 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:27.982762  4048 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:27.998032  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 121742us
W20260709 13:02:28.004457  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.004647  1681 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:27.965191  4879 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:27.967197  4813 consensus_queue.cc:1243] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:02:27.950723  4757 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:27.991657  4889 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.073s	user 0.005s	sys 0.002s
I20260709 13:02:27.844071  4873 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
W20260709 13:02:28.001693  1564 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.012s	sys 0.009s
W20260709 13:02:27.913450  4878 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
W20260709 13:02:28.002172  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 731756us
I20260709 13:02:28.005806  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:27.976756  4737 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:28.006778  1681 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:28.013195  1681 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:28.003444  4635 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:28.014515  1681 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:28.008523  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 161245us
I20260709 13:02:28.017277  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:28.018136  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:28.020527  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:28.023643  1681 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:28.008590  4879 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:28.008996  4713 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:28.022508  4726 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:28.044677  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.053648  1568 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:28.044989  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:28.045675  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 179555us
I20260709 13:02:28.043521  2268 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:28.051863  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.045791  4314 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:28.056617  4314 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.025579  4815 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:28.061236  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:28.046554  1639 raft_consensus.cc:2955] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:28.067082  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.067822  1563 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:28.068676  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:28.069159  1563 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:28.044893  4813 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:28.042339  4635 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:28.045514  4721 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:28.045475  4737 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:28.059708  4468 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.051857  4661 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:02:28.066336  4815 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:28.058768  4314 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:28.070842  4887 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:28.095831  4512 raft_consensus.cc:3055] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:28.096390  4512 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:28.097325  4512 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.098482  4512 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:28.082399  1569 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:02:28.071566  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.105352  2268 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:28.090359  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.071724  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.112906  1568 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:28.114550  1568 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:28.076740  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.119771  1569 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:28.120951  2268 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:28.122298  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:28.123224  1716 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:28.123522  1563 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:28.127583  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.129561  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:28.130195  2298 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.134222  1715 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:28.135192  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:28.136162  2303 raft_consensus.cc:2410] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:02:28.136392  4661 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:28.076297  4721 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:28.082531  4740 consensus_meta.cc:287] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.785s	user 0.007s	sys 0.004s
W20260709 13:02:28.079896  4888 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:02:28.075682  4813 raft_consensus.cc:697] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:28.141863  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:28.067140  4757 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:28.076462  4544 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:28.124964  4689 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:28.087509  4650 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.148144  4689 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:28.148756  4689 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.149791  4650 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:28.077049  4737 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:28.151140  4737 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.159365  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.162844  2303 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:28.163821  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.166311  2299 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:28.174783  1716 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:28.179315  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.183627  4505 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:28.081377  4468 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.184963  1569 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:28.186033  4468 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:28.186395  1560 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:28.084754  4815 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:28.161149  4740 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.193107  4740 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:28.194339  4813 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.139256  4721 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:26.159060   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:28.174006  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:28.204267  2298 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:28.207691  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:28.212961  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:28.218891  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:28.223069  1534 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:28.206023  4468 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.228842  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:28.230507  1716 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.216225  4898 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:28.235606  4048 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.229076  4505 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:28.228998  4757 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:28.230827  4668 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:28.233682  4468 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:28.262033  4757 raft_consensus.cc:697] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:28.285689  4689 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.287482  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.290612  1560 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:28.273049  1716 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:28.299333  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.300561  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.295346  2268 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:28.302460  1717 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:28.303511  4770 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:28.303648  4538 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:28.304150  4770 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.305570  1560 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:28.307439  4770 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:28.304265  4538 raft_consensus.cc:697] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:28.307539  1568 pending_rounds.cc:85] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 2
I20260709 13:02:28.309665  1568 pending_rounds.cc:107] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:02:28.309942  1568 raft_consensus.cc:2889] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:28.306717  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.315889  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.316403  1564 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:02:28.322613  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.304268  4468 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:28.318709  4446 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.009s
W20260709 13:02:28.327431  4906 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:02:28.325410  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:28.329620  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.274856  4879 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:28.348673  4886 raft_consensus.cc:2955] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:28.331777  1564 raft_consensus.cc:2393] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:02:28.372149  4900 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:02:28.362272  4415 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:28.351076  4879 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:28.366695  1466 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45142 (request call id 348) took 27807 ms (client timeout 15000 ms). Trace:
W20260709 13:02:28.379614  1466 rpcz_store.cc:269] 0709 13:02:00.559118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.579470 (+ 20352us) service_pool.cc:224] Handling call
0709 13:02:28.366570 (+27787100us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:23.656204 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:23.911633 (+255429us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:23.914433 (+  2800us) write_op.cc:435] Acquired schema lock
0709 13:02:23.955488 (+ 41055us) tablet.cc:662] Decoding operations
0709 13:02:23.956136 (+   648us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:23.956194 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:23.956221 (+    27us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:02:23.956594 (+   373us) tablet.cc:701] Row locks acquired
0709 13:02:23.956610 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:02:23.956770 (+   160us) write_op.cc:270] Start()
0709 13:02:23.956881 (+   111us) write_op.cc:276] Timestamp: P: 1783602143956708 usec, L: 0
0709 13:02:23.956902 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:02:23.957536 (+   634us) log.cc:844] Serialized 1696 byte log entry
0709 13:02:25.209790 (+1252254us) op_driver.cc:464] REPLICATION: finished
0709 13:02:25.339182 (+129392us) write_op.cc:301] APPLY: starting
0709 13:02:25.343598 (+  4416us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:25.366997 (+ 23399us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:25.367147 (+   150us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:25.367774 (+   627us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:25.367976 (+   202us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:02:25.368031 (+    55us) write_op.cc:312] APPLY: finished
0709 13:02:25.375660 (+  7629us) log.cc:844] Serialized 41 byte log entry
0709 13:02:25.375834 (+   174us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:25.375937 (+   103us) write_op.cc:454] Released schema lock
0709 13:02:25.375979 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:25.376392 (+   413us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:25.376425 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2355420,"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":119818,"apply.run_cpu_time_us":2689,"apply.run_wall_time_us":41405,"num_ops":4,"prepare.queue_time_us":365401,"prepare.run_cpu_time_us":46987,"prepare.run_wall_time_us":1116139,"raft.queue_time_us":323485,"raft.run_cpu_time_us":9100,"raft.run_wall_time_us":80721,"replication_time_us":1252734,"thread_start_us":1213951,"threads_started":4,"wal-append.queue_time_us":798841,"wal-append.run_cpu_time_us":24594,"wal-append.run_wall_time_us":1614401}]]}
I20260709 13:02:28.334403  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.336078  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:28.387390  1560 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:28.355185  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.389057  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.362048  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.330540  1716 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:28.357682  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 538) took 1108 ms. Trace:
I20260709 13:02:28.354074  1563 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:02:28.387470  1644 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.340077  4757 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.395500  1562 rpcz_store.cc:276] 0709 13:02:27.249071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.249203 (+   132us) service_pool.cc:224] Handling call
0709 13:02:27.260099 (+ 10896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.304291 (+ 44192us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:27.353677 (+ 49386us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.132657 (+778980us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.132860 (+   203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:28.132907 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:28.132950 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.357369 (+224419us) spinlock_profiling.cc:243] Waited 43.9 ms on lock 0x7b54001cdc54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:28.357430 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.357606 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30155,"prepare.queue_time_us":424155,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":4187,"raft.queue_time_us":31199,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":5746,"spinlock_wait_cycles":183975680,"thread_start_us":807155,"threads_started":3,"wal-append.queue_time_us":653593}
I20260709 13:02:28.397029  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.337095  4538 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.353796  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.356119  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 1979) took 1006 ms. Trace:
I20260709 13:02:28.400025  1710 rpcz_store.cc:276] 0709 13:02:27.349834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.352531 (+  2697us) service_pool.cc:224] Handling call
0709 13:02:28.355890 (+1003359us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54003db3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:28.356045 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474582912}
I20260709 13:02:28.401770  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.352523  4869 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:28.392050  1560 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:02:28.392544  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.396328  1718 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:02:28.372862  4855 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 1.073s	user 0.006s	sys 0.000s
I20260709 13:02:28.404975  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 619) took 2220 ms. Trace:
I20260709 13:02:28.355337  4446 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:28.405457  1642 rpcz_store.cc:276] 0709 13:02:26.184390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.539557 (+355167us) service_pool.cc:224] Handling call
0709 13:02:26.555217 (+ 15660us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:26.640607 (+ 85390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:26.640644 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:26.647908 (+  7264us) log.cc:844] Serialized 26 byte log entry
0709 13:02:27.137496 (+489588us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:27.137535 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.137599 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.372349 (+1234750us) spinlock_profiling.cc:243] Waited 3.58 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:28.404693 (+ 32344us) raft_consensus.cc:1685] finished
0709 13:02:28.404715 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.404919 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23240,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":8393,"spinlock_wait_cycles":8236800,"thread_start_us":553540,"threads_started":2,"wal-append.queue_time_us":617861}
I20260709 13:02:28.407716  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.398475  1563 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:28.407886  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.409328  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.409919  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.397925  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.410889  1713 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:28.397351  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.411819  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:28.412456  1713 raft_consensus.cc:2376] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9. Re-sending same reply.
I20260709 13:02:28.413229  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.413879  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.414842  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:28.415155  2299 raft_consensus.cc:2410] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:02:28.415468  1716 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:28.418661  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.420033  4471 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:28.420742  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:28.421392  1642 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:28.422353  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:28.420111  2268 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:28.423767  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.404474  4869 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:28.424944  4770 consensus_queue.cc:1059] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:28.425035  4869 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:28.425840  4770 consensus_queue.cc:1243] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:02:28.426069  4869 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.426367  4770 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:28.419438  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.427067  1710 raft_consensus.cc:2410] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:28.427143  1715 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:28.418800  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.429455  4770 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:28.431030  1711 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:02:28.554397  1532 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:28.430611  4669 raft_consensus.cc:3055] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:28.434682  1644 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:28.570169  4669 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:02:28.571552  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.571460  4669 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.573531  4669 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:28.440752  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:28.556550  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.523275  4689 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:28.562552  1716 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:28.598217  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.599116  1716 raft_consensus.cc:2393] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:28.599886  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.600627  1716 raft_consensus.cc:2410] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:02:28.533988  4788 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:28.496528  4910 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:02:28.427584  4446 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.581005  1715 raft_consensus.cc:2393] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 9: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:28.582842  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.596649  2300 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:28.609206  1716 raft_consensus.cc:3055] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:28.609575  1716 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:28.612509  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 58199us
I20260709 13:02:28.613662  1644 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:28.576373  4491 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.576081  4649 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:28.610281  1716 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.618577  1716 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.622577  1532 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:28.612464  4048 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:28.616099  4668 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:28.641032  1563 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:28.645594  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 631) took 1003 ms. Trace:
I20260709 13:02:28.649914  1710 rpcz_store.cc:276] 0709 13:02:27.642652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.465778 (+823126us) service_pool.cc:224] Handling call
0709 13:02:28.557612 (+ 91834us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.557816 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:28.558345 (+   529us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.620986 (+ 62641us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:28.621051 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:28.621110 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.645368 (+ 24258us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.645536 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":153,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":86,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":1361,"thread_start_us":58291,"threads_started":1}
I20260709 13:02:28.633445  4743 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:28.620621  4721 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.627034  4610 consensus_queue.cc:1059] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:02:28.621587  4491 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:28.680698  1717 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:28.668424  4743 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:28.646956  4898 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:02:28.690761  4743 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:28.696447  4869 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:28.619704  4879 consensus_queue.cc:1175] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" responder_term: 8 status { last_received { term: 8 index: 9 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:28.644241  4770 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.699844  4869 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.710623  4869 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:28.712656  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.713662  4635 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:28.714509  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 543) took 1461 ms. Trace:
I20260709 13:02:28.714890  1565 rpcz_store.cc:276] 0709 13:02:27.252988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.253217 (+   229us) service_pool.cc:224] Handling call
0709 13:02:27.253560 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.753141 (+499581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.714284 (+961143us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b5400297cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:28.714450 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1120940288}
I20260709 13:02:28.716310  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.717367  1565 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:28.719884  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.720533  1569 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:28.721392  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.723940  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:28.726291  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.727274  2298 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:28.730646  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.731966  4491 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:28.732087  4461 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:28.732784  4446 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:28.735215  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 94109.8us
I20260709 13:02:28.735950  1532 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:28.737030  1565 raft_consensus.cc:1217] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[4.3-7.4]   Dedup: 7.4->[]
I20260709 13:02:28.738421  1565 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: [4.3-7.4]
I20260709 13:02:28.739637  1532 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:28.743249  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.743901  1564 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:28.747979  4721 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:28.749773  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:28.752486  4743 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:28.752844  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:28.756024  4880 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.706s	user 0.005s	sys 0.000s
I20260709 13:02:28.748718  4491 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:02:28.759332  4491 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.759760  4491 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:28.754698  4743 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:28.766896  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.749917  4886 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:28.815323  1716 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:02:28.815542  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 65833.7us
W20260709 13:02:28.816627  4812 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:02:28.817456  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:28.821223  2300 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:28.831106  1711 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:28.833595  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.834264  1711 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:28.839461  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:28.837387  4635 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:02:28.841763  4922 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:28.842504  1561 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:02:28.842324  4922 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.843994  1532 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:28.844468  4922 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:28.845011  4922 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:28.855468  1711 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:02:28.833333  4743 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:28.835867  4886 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:28.861600  1561 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:28.861786  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 622) took 1287 ms. Trace:
I20260709 13:02:28.863487  4886 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.865619  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1824) took 1478 ms. Trace:
I20260709 13:02:28.868928  1710 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:28.870321  1710 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:28.879469  1681 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:28.881691  1681 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:28.883787  4869 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:28.885035  1532 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:28.885280  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 623) took 2675 ms. Trace:
I20260709 13:02:28.885526  1681 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:28.903687  4879 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:28.904337  4879 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:28.904846  4879 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:28.887547  4761 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:28.907063  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 625) took 1331 ms. Trace:
I20260709 13:02:28.907310  4761 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.907775  4761 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.907476  1717 rpcz_store.cc:276] 0709 13:02:27.575574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.312634 (+737060us) service_pool.cc:224] Handling call
0709 13:02:28.312950 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.681296 (+368346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.906777 (+225481us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b5400339ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:28.906997 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":845214592}
I20260709 13:02:28.889739  4774 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:02:28.912426  2302 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:28.913904  4668 raft_consensus.cc:3055] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:28.914321  4668 raft_consensus.cc:740] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:28.915213  4668 consensus_queue.cc:260] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:28.866185  1714 rpcz_store.cc:276] 0709 13:02:27.388670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.389147 (+   477us) service_pool.cc:224] Handling call
0709 13:02:28.289722 (+900575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.290068 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:28.603010 (+312942us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.696372 (+ 93362us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:28.696442 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:28.696492 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.696809 (+   317us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.865377 (+168568us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b54000483d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:28.865540 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37003,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":31,"raft.queue_time_us":76,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":3582,"spinlock_wait_cycles":2069929728,"thread_start_us":379177,"threads_started":2,"wal-append.queue_time_us":116110}
I20260709 13:02:28.862385  1712 rpcz_store.cc:276] 0709 13:02:27.574045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.300320 (+726275us) service_pool.cc:224] Handling call
0709 13:02:28.371048 (+ 70728us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.371260 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.371291 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:28.371338 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:28.371385 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.371442 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:28.861531 (+490089us) spinlock_profiling.cc:243] Waited 70.4 ms on lock 0x7b54003db3d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:28.861705 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161935104}
I20260709 13:02:28.885612  1636 rpcz_store.cc:276] 0709 13:02:26.209749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.648268 (+438519us) service_pool.cc:224] Handling call
0709 13:02:26.648683 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.236332 (+587649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.885053 (+1648721us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b54001e9454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:28.885221 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1344626432}
I20260709 13:02:28.891131  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:28.911617  4887 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.924078  1713 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.913162  1716 raft_consensus.cc:2955] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:28.929273  4887 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.930341  4685 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:28.930900  4685 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.932384  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:28.932926  4685 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:28.933581  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:28.935231  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 632) took 2660 ms. Trace:
I20260709 13:02:28.935626  1637 rpcz_store.cc:276] 0709 13:02:26.274834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.708278 (+433444us) service_pool.cc:224] Handling call
0709 13:02:26.708555 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:27.310882 (+602327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.935003 (+1624121us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b5400155254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:28.935150 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1352541952}
I20260709 13:02:28.937072  1637 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:28.937878  1643 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:28.940402  1713 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
W20260709 13:02:28.940565  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.943616  1681 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:28.946615  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.947307  2298 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:28.954906  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.956234  1681 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:28.938840  4668 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:28.961300  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:28.961273  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.965742  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:28.966559  2302 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.970075  1711 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:28.973590  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:28.953406  4461 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:28.959285  4487 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:28.977535  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.979175  1567 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:28.979504  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:28.979924  2298 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEARNER]: Advancing to term 9
I20260709 13:02:28.980335  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:28.983332  2300 raft_consensus.cc:2376] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6. Re-sending same reply.
I20260709 13:02:28.964941  4812 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:28.985366  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:28.985832  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:28.985420  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.962714  4415 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:28.990397  1561 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:28.954268  4869 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:02:28.991961  1609 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:28.992295  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.993486  1712 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:28.971729  4491 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:28.997882  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:28.978539  4487 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.998854  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:28.987689  4786 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.037117  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:29.036767  1564 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 5: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:29.037667  2301 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:02:29.038388  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 46496.5us
I20260709 13:02:29.039655  4747 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:29.040133  4747 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:29.040510  4747 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:29.040920  4491 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.037221  4743 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:29.041960  4743 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.016835  4415 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.045143  4415 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.047773  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:29.049264  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:28.997874  4885 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.053795  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:29.055174  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:29.056994  4740 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:02:29.063686  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 64916.3us
I20260709 13:02:29.066215  1607 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:29.068002  2298 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:02:29.068589  2298 pending_rounds.cc:85] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 6
I20260709 13:02:29.068951  2298 pending_rounds.cc:107] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:02:29.069607  2298 raft_consensus.cc:2889] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:29.071194  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:29.027583  4487 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:28.987886  4628 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.073146  1709 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 5: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:29.074206  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:28.989017  4461 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.081351  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1893) took 1306 ms. Trace:
I20260709 13:02:29.081344  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:29.085278  1644 raft_consensus.cc:2368] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8.
I20260709 13:02:29.081845  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:29.057737  4792 raft_consensus.cc:3055] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:29.084667  1638 rpcz_store.cc:276] 0709 13:02:27.775293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.778349 (+  3056us) service_pool.cc:224] Handling call
0709 13:02:27.778757 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.803217 (+ 24460us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:27.830534 (+ 27317us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.918872 (+1088338us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:28.919369 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 13:02:28.919516 (+   147us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:28.919715 (+   199us) log.cc:844] Serialized 14 byte log entry
0709 13:02:28.919879 (+   164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.919945 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.081083 (+161138us) raft_consensus.cc:1685] finished
0709 13:02:29.081104 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.081280 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":903565,"prepare.queue_time_us":27698,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":13988,"raft.queue_time_us":899177,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":24894,"spinlock_wait_cycles":20451584,"thread_start_us":160205,"threads_started":1,"wal-append.queue_time_us":253069}
W20260709 13:02:29.091972  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 52651us
I20260709 13:02:28.989950  4812 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.097924  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:29.084178  2302 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:29.069432  4649 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:29.047068  4446 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:02:29.120641  1609 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:28.995502  4887 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:29.121706  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:29.124897  1638 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:29.125094  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:29.125675  1609 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:29.127696  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 537) took 1872 ms (client timeout 2236 ms). Trace:
W20260709 13:02:29.128206  1566 rpcz_store.cc:269] 0709 13:02:27.248701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.248826 (+   125us) service_pool.cc:224] Handling call
0709 13:02:29.120950 (+1872124us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54001ff254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:29.121066 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":700835584}
I20260709 13:02:29.053890  4786 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.133997  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:29.117971  4812 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:29.108247  4792 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:29.070434  4885 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.119910  4649 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.120455  4415 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:29.162151  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:29.080804  4538 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:02:29.128362  4403 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:29.128176  4733 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.129457  4471 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:29.137027  4333 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:29.167938  4538 raft_consensus.cc:3055] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:29.175383  4538 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:29.113960  4831 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:02:29.128747  4746 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:02:29.176316  4538 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.177354  4403 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:29.177623  4538 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:29.178887  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.113646  4685 consensus_queue.cc:1059] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:02:29.165038  4649 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:29.092798  4628 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.184827  4628 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.173552  4743 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.174435  4733 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.187605  4733 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.193606  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:29.195616  1637 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:29.197028  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:29.197834  1643 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:29.199080  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:29.137563  4860 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:29.207348  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:29.130108  4785 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 13:02:29.281682  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:29.282773  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 157197us
W20260709 13:02:29.283812  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 157335us
I20260709 13:02:29.288484  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1908) took 1271 ms. Trace:
I20260709 13:02:29.288887  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:29.238235  4744 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:02:29.293009  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 336874us
I20260709 13:02:29.300478  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:29.294322  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:29.298326  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:29.295897  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 216118us
I20260709 13:02:29.388010  2268 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:29.299779  4669 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:02:29.300503  4744 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:29.238068  4628 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:02:29.327183  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:29.331236  1609 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:29.289108  1639 rpcz_store.cc:276] 0709 13:02:28.016904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.025169 (+  8265us) service_pool.cc:224] Handling call
0709 13:02:28.046241 (+ 21072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.046445 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:28.143617 (+ 97172us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.138413 (+994796us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.138945 (+   532us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.286768 (+147823us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.286821 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:29.288207 (+  1386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.288287 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.288416 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":795973,"prepare.queue_time_us":306811,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":892406,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"spinlock_wait_cycles":7617792,"thread_start_us":319206,"threads_started":2}
W20260709 13:02:29.362012  4926 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:02:29.323992  1682 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:29.400653  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 76787.5us
I20260709 13:02:29.312153  4814 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:29.242431  4860 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:29.374320  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 73138.2us
W20260709 13:02:29.353876  4757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.005s	sys 0.011s
I20260709 13:02:29.309211  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:29.404991  2301 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:29.361795  4885 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.291651  1715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:02:29.284106  4792 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.242354  4831 raft_consensus.cc:3055] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:02:29.342489  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 133672us
W20260709 13:02:29.388337  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 94327.9us
I20260709 13:02:29.402465  4940 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:29.406618  4757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.005s	sys 0.014s
I20260709 13:02:29.407076  1715 raft_consensus.cc:606] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:02:29.436909  1715 raft_consensus.cc:614] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:29.393992  4689 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:29.414584  4649 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:29.415608  4744 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:29.441519  4744 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:29.400518  4937 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:02:29.447290  1644 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:29.419626  4925 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
W20260709 13:02:29.420421  4935 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:02:29.402472  4786 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.424350  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:29.449626  4757 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:29.450832  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:29.451763  4929 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:02:29.452883  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:29.453764  4650 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:29.454236  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:02:29.454434  4928 log.cc:927] Time spent T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 13:02:29.386445  4927 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:02:29.454573  4650 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:29.456073  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.430550  4792 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:29.405550  4942 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:29.463428  4689 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:29.464133  4689 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:29.464193  4942 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:29.465121  4942 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:29.467726  4930 log.cc:927] Time spent T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:02:29.468384  4942 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:29.469403  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 1989) took 1122 ms. Trace:
I20260709 13:02:29.470027  1718 rpcz_store.cc:276] 0709 13:02:28.346828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.772990 (+426162us) service_pool.cc:224] Handling call
0709 13:02:28.773544 (+   554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:28.774057 (+   513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.774102 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:28.795482 (+ 21380us) log.cc:844] Serialized 26 byte log entry
0709 13:02:28.918452 (+122970us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b18003a79e0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:02:29.122608 (+204156us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:29.122662 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.122750 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.469021 (+346271us) raft_consensus.cc:1685] finished
0709 13:02:29.469049 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.469326 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11363,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"spinlock_wait_cycles":65697024,"thread_start_us":203965,"threads_started":1,"wal-append.queue_time_us":443000}
W20260709 13:02:29.432324  4936 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:02:29.434026  4669 consensus_queue.cc:1059] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:29.433892  4860 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.476920  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 639) took 1381 ms. Trace:
I20260709 13:02:29.477341  1641 rpcz_store.cc:276] 0709 13:02:28.095158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.099788 (+  4630us) service_pool.cc:224] Handling call
0709 13:02:28.102855 (+  3067us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:28.934666 (+831811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.934697 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:28.935137 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 13:02:29.133305 (+198168us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:29.133343 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.133408 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.476690 (+343282us) raft_consensus.cc:1685] finished
0709 13:02:29.476715 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.476862 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":907494,"raft.queue_time_us":832164,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":12151,"thread_start_us":197895,"threads_started":1,"wal-append.queue_time_us":307535}
I20260709 13:02:29.412240  4945 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:02:29.420953  4471 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:29.480472  4471 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:29.406163  4747 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.440534  4649 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:29.474507  4879 consensus_meta.cc:287] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.000s
I20260709 13:02:29.483789  4879 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.485126  4747 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:29.399348  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 68230.3us
I20260709 13:02:29.510696  1609 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:29.491648  4812 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.496536  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:29.469237  4942 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:29.532008  1715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:02:29.532721  1715 raft_consensus.cc:606] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:02:29.533087  1715 raft_consensus.cc:614] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:29.538725  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:29.539769  2298 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:29.544404  1638 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:29.551079  4744 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:29.551417  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:29.551716  4744 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:29.553872  4744 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:29.555624  4785 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:29.556243  4785 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.556692  4785 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:29.512877  4650 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:29.518790  4786 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:29.567265  4786 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:29.567811  4786 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:29.570675  4812 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.571619  4333 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:29.572125  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:29.572258  4333 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.573423  4812 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.573989  4812 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:29.575318  4761 consensus_meta.cc:287] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.667s	user 0.010s	sys 0.010s
I20260709 13:02:29.575908  4761 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:29.577953  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:29.562690  4879 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.423461  4831 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:29.579687  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 83273us
I20260709 13:02:29.584707  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:29.585968  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:29.589298  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:29.581234  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:29.593449  2298 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:29.593218  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:29.601020  1682 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:29.602119  1609 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:29.603190  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:29.605166  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 213028us
I20260709 13:02:29.560070  4689 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.608026  1682 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:29.609220  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 206408us
I20260709 13:02:29.611699  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:29.612186  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:29.613135  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:29.589603  4939 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:29.618691  4938 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:29.619194  4938 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.619650  4938 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:29.619987  4740 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:29.620478  4740 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:29.618718  4747 consensus_queue.cc:1059] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:29.588515  4812 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.589051  4885 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:29.591988  4650 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:29.566614  4487 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:29.591833  4649 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:29.605195  4879 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:29.609997  4744 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:29.609427  4814 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:29.620901  4740 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:29.648141  4814 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:29.648784  4814 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.649874  1712 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:29.632135  4650 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:02:29.650352  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 216976us
I20260709 13:02:29.653873  2269 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:29.617802  4538 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:02:29.638715  4812 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:29.624337  4860 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:02:29.639350  4461 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.653151  4879 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:29.655534  4650 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:29.655164  4814 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:29.644448  4831 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.673866  4471 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:29.675007  4689 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:29.665715  4860 consensus_queue.cc:1243] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:02:29.675407  4940 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:29.698618  4955 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:29.699321  4955 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.684461  4761 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:29.699774  4955 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:29.723996  4955 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.621467  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:29.755949  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 134295us
I20260709 13:02:29.758819  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:29.764626  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:29.769208  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 147149us
W20260709 13:02:29.782043  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 168849us
I20260709 13:02:29.784257  1682 leader_election.cc:400] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8.
I20260709 13:02:29.784793  1682 leader_election.cc:403] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:29.784651  4314 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:29.786228  4314 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:29.789079  4314 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:29.789281  4048 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:29.791247  4491 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:29.791754  4487 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:29.792423  4491 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.793610  4415 consensus_queue.cc:1059] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:29.795279  4415 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:29.782684  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:29.796216  4415 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:29.798017  4471 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:29.799069  4471 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8.
I20260709 13:02:29.799096  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:29.800467  1567 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:29.790143  4314 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:29.802888  4314 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.805985  4314 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.801721  4048 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.807713  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:29.817687  1563 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:29.810660  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:29.789557  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:29.812583  4491 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.814998  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:29.838944  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 68211.1us
I20260709 13:02:29.806768  4314 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:29.707109  4937 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:02:29.863075  4685 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.864120  4685 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:29.876487  4955 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:29.880487  4048 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.868878  4415 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:29.887665  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 98480.9us
I20260709 13:02:29.893299  4962 raft_consensus.cc:3055] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:29.893451  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:29.893954  4937 raft_consensus.cc:3055] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:29.894095  4960 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:29.894534  4937 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:29.896082  4937 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.898350  4937 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:29.900781  4814 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:29.876336  4314 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:29.914713  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:29.920542  4737 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.886472  4403 raft_consensus.cc:3055] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:29.934078  1567 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:29.938416  1566 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:02:29.956802  1710 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:29.957484  4959 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.958592  4959 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:29.960013  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 266293us
I20260709 13:02:29.962009  4959 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:29.963411  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 309519us
I20260709 13:02:29.893988  4962 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:29.929757  4403 raft_consensus.cc:740] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:29.968478  4962 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.931946  4737 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:29.969210  4403 consensus_queue.cc:260] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:29.976047  4737 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:29.977077  4831 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:29.979976  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 89818.8us
I20260709 13:02:29.981117  1682 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:29.983276  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:29.949659  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 78084.6us
I20260709 13:02:29.975862  4403 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:02:29.997232  4792 consensus_meta.cc:287] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.538s	user 0.015s	sys 0.005s
I20260709 13:02:29.998986  2302 raft_consensus.cc:2410] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:29.999410  4955 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:30.003752  4963 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:30.005769  4946 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:29.938603  4314 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:30.023375  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 58374us
I20260709 13:02:30.026840  4333 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.026979  4938 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.027843  4689 raft_consensus.cc:3055] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:30.028327  4689 raft_consensus.cc:740] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:30.029651  4689 consensus_queue.cc:260] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.030447  4938 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.025094  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:30.034128  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.034533  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.036636  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:30.043179  4931 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.807s	user 0.005s	sys 0.002s
I20260709 13:02:30.041330  4814 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:30.047003  4685 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:30.048271  4610 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:30.049002  4689 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:30.053200  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.045091  1560 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:30.055298  1562 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:30.056277  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.056483  1718 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:30.057036  1560 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:30.057829  4797 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.059551  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.056303  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:30.061370  1563 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:30.058619  4797 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.065493  4797 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:30.059312  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.075481  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:30.077441  4785 consensus_meta.cc:287] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.520s	user 0.008s	sys 0.008s
I20260709 13:02:30.078622  4814 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.079413  4814 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:30.080559  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:30.081550  2303 raft_consensus.cc:2393] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:30.081832  4814 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:30.083267  4922 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:30.084291  4922 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:30.087535  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:30.066551  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1911) took 2005 ms. Trace:
I20260709 13:02:30.094697  1640 rpcz_store.cc:276] 0709 13:02:28.060622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.060780 (+   158us) service_pool.cc:224] Handling call
0709 13:02:28.061236 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:28.919468 (+858232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.919497 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:28.919853 (+   356us) log.cc:844] Serialized 26 byte log entry
0709 13:02:29.115753 (+195900us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:29.115809 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.115909 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.966495 (+850586us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:30.066025 (+ 99530us) raft_consensus.cc:1685] finished
0709 13:02:30.066047 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.066336 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":857774,"raft.queue_time_us":857949,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"spinlock_wait_cycles":7631616,"thread_start_us":195576,"threads_started":1,"wal-append.queue_time_us":235153}
I20260709 13:02:30.105044  2297 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:30.105788  1718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:02:30.106797  1718 raft_consensus.cc:606] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:30.107183  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:30.070320  4333 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.108251  4333 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.109587  1532 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:30.110843  4922 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:30.111418  4922 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:30.112326  4797 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:30.112775  4797 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.113166  4797 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:30.114394  4969 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.131s	user 0.002s	sys 0.000s
W20260709 13:02:30.115316  4786 consensus_meta.cc:287] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.547s	user 0.003s	sys 0.015s
I20260709 13:02:30.116194  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.116984  1560 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:30.115834  4786 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.118134  4962 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:30.107406  1718 raft_consensus.cc:614] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:30.024128  4959 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:30.132850  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:30.133637  1563 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:30.138799  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.139673  4879 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:30.140180   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:02:30.140516  4879 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.141040  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:30.141678  1710 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:30.110759  4333 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.049072  4610 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.148559  1569 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:30.150355  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:30.154342  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:30.155349  1640 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:30.094816  4685 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:30.156493  4685 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:30.159256  1644 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:30.164610  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:30.166730  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 183656us
I20260709 13:02:30.077939  4785 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.172391  4887 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:30.175000  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.158809  4685 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:30.183449  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 181253us
I20260709 13:02:30.162525  4737 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:30.173280  4785 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.198341  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:30.041757  4650 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:30.201973  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:30.175088  4922 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.173211  4887 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.173914  4733 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:30.196246  4879 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.234202  4610 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.234364  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.222860  2302 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:30.222566  4733 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:30.225517  4972 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:02:30.227934  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:30.229519  4650 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:30.214391  2268 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:30.222898  4740 consensus_meta.cc:287] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.528s	user 0.016s	sys 0.001s
I20260709 13:02:30.230748   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:30.231209  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.231573  1682 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:30.231909  4737 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.232718  1639 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:30.233067  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:30.222986  4744 consensus_meta.cc:287] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.515s	user 0.005s	sys 0.012s
I20260709 13:02:30.238360  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:30.248584  1563 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:30.196028  4938 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:02:30.252657  1609 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:30.257441  4446 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:30.258572  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:30.258430  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:30.259276  4740 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:30.263270  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 55200.8us
I20260709 13:02:30.264959  1560 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:30.267624  2304 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:30.267760  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:30.264012  4544 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.018s	sys 0.051s
I20260709 13:02:30.282941  1640 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:30.263669  4785 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:30.284214  4959 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:30.284428  4785 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.284943  4785 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:30.285122  4544 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.019s	sys 0.051s
I20260709 13:02:30.285727  4737 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.285873  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:30.287173  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.288178  1564 raft_consensus.cc:2410] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:02:30.283911  4740 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:30.287977  1681 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:30.265560  4446 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:30.301631  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.303903  4505 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:30.306003  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:30.311363  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.312041  1569 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
W20260709 13:02:30.313423  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 103199us
I20260709 13:02:30.314352  1607 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:30.303959  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:30.237753  4922 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:30.290364  4744 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:30.323103  4650 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:30.323802  4650 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:30.323973  4980 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:02:30.324319  4650 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:30.328377  1681 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:30.329953  1681 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:30.330803  4922 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:30.331071  4942 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:30.331871  4942 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:30.310446  4610 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:30.332366  4942 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:30.333561  4942 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:30.335044  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.307392  4959 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.316313  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:30.348323  2298 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:30.367388  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 63552.4us
I20260709 13:02:30.367830  4967 consensus_queue.cc:1059] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.130s
I20260709 13:02:30.369010  4967 raft_consensus.cc:3055] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:30.369585  4967 raft_consensus.cc:740] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:30.371153  4733 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.372298  4733 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:30.372762  4733 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:30.373258  4733 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:30.375794  4689 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:02:30.371248  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 55064.9us
I20260709 13:02:30.367394  4788 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:02:30.380941  4797 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:30.385958  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 154543us
W20260709 13:02:30.389606  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 104018us
I20260709 13:02:30.412287  1607 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:30.348846  4960 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:02:30.441464  4977 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:30.445904  1636 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:30.446405  1642 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:02:30.447715  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:30.467542  4544 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:30.459144  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:30.468895  4887 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:30.446760  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 116977us
I20260709 13:02:30.433523  4403 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:30.470633  1681 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:30.470685  4403 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.472038  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:30.472910  1638 raft_consensus.cc:2393] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:30.430084  4898 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:30.467397  1682 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:30.463320  1637 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:30.448256  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:30.430749  4950 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:30.467413  1534 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:30.484390  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 72207.9us
I20260709 13:02:30.427058  4959 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:30.469213  4786 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:30.447692  4983 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:02:30.379583  4967 consensus_queue.cc:260] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:30.485258  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:30.494853  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:30.495515  4976 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:30.498090  1607 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:30.478152  4937 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.555s	user 0.006s	sys 0.007s
I20260709 13:02:30.463575  4747 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:30.499364  4747 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.502648  4747 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.505496  4976 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:30.506012  4685 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:30.506415  4685 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.506768  4685 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:30.448698  4446 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.507328  4786 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:30.508183  4786 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:30.508721  4786 raft_consensus.cc:697] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:30.509814  4786 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.513177  4959 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:30.513922  4959 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:30.514532  4959 raft_consensus.cc:697] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:30.460270  4797 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.525144  2298 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:30.517410  1607 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:30.526767  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 78622.4us
I20260709 13:02:30.485129  4403 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:30.564829  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 48735.6us
I20260709 13:02:30.566427  4788 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:30.570250  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:30.571350  4668 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.572108  4645 consensus_queue.cc:1059] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:30.575695  4967 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:30.578362  1560 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:30.579003  4950 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.599404  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:30.600490  1635 raft_consensus.cc:2410] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:02:30.601133  4950 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:30.601663  4950 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:30.601835  1635 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:02:30.571384  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 138133us
I20260709 13:02:30.603688  4669 raft_consensus.cc:3055] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:30.604187  4669 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:30.605288  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:30.605340  4669 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.578567  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:30.607106  4403 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:30.607967  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.607834  4403 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:30.608794  1568 raft_consensus.cc:2393] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:30.609664  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:30.611132  4747 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:30.611653  4610 raft_consensus.cc:3055] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:30.612106  4610 raft_consensus.cc:740] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:30.611835  4747 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:30.613129  4610 consensus_queue.cc:260] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.579492  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:30.514853  4685 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:30.615388  1569 raft_consensus.cc:2393] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:30.616230  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.616470  4974 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:30.582819  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:30.617436  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:30.583194  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:30.618746  1642 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:30.584338  4959 consensus_queue.cc:237] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.579533  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:30.621302  1644 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:30.602309  4950 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.618577  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:30.596793  4546 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:30.597996  4977 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:30.599195  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:30.623706  4733 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:30.635696  1566 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:02:30.642470  1533 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:30.644621  4788 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:30.646611  4761 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:30.657339  1568 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: [7.8-8.9]
I20260709 13:02:30.625373  1569 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:30.610694  4403 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:30.614650  4747 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:30.673882  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 201833us
I20260709 13:02:30.656489  4831 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:30.683656  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:30.658881  4974 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:30.616482  4685 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:30.645547  4743 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:02:30.695267  1681 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:30.695698  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 89039.8us
W20260709 13:02:30.680254  2268 connection.cc:256] RPC call timeout handler was delayed by -1.1103579019999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
I20260709 13:02:30.628141  4668 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.714416  4960 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:30.714818  4950 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:30.691748  4974 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.667387  4785 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.697862  4669 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:30.656798  4977 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.693445  4610 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:30.675729  4761 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.719491  1534 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:30.720492  4974 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:30.720594  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:30.750377  1640 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:30.732740  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:30.752897  2304 raft_consensus.cc:2410] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:30.744339  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:30.723992  4898 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:02:30.741443  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.691733  4650 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:30.775916  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.776984  1635 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:02:30.753948  2303 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:30.789235  2298 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:30.794787  1563 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:30.799417  1638 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:30.800441  2301 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:02:30.800983  2301 pending_rounds.cc:85] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 6
I20260709 13:02:30.810441  1609 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:30.767867  4898 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:30.813831  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:30.822696  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:30.801528  2301 pending_rounds.cc:107] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:02:30.824715  2301 raft_consensus.cc:2889] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:30.780525  4403 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:30.828403  1609 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:30.830181  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:30.830869  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 135687us
I20260709 13:02:30.828900  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:30.843410  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:30.844259  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:30.851640  2297 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:30.859426  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:30.826226  4950 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:30.830195  4942 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:30.841892  4747 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:30.871528  2300 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:30.890049  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:30.890740  2268 leader_election.cc:400] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8.
I20260709 13:02:30.892359  2268 leader_election.cc:403] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:02:30.894258  4963 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8.
I20260709 13:02:30.891665  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:30.860422  2303 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
W20260709 13:02:30.901226  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 335654us
I20260709 13:02:30.876833  4942 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:30.900456  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 433209us
I20260709 13:02:30.895138  4996 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.925266  1569 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:30.927469  4761 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.886670  4403 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.928047  4963 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:30.943136  4403 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.953212  4997 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.959410  4668 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.963474  4650 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.963943  4939 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:30.965114  4995 log.cc:927] Time spent T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:02:30.991430  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 207856us
I20260709 13:02:30.881019  4950 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:30.966450  4999 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:02:30.967047  4546 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:30.906262  4942 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:30.895459  4685 consensus_queue.cc:1059] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:02:30.929415  4996 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:30.928388  2268 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:30.935568  4785 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:30.987437  1563 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:30.996150  4997 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:30.999255  4977 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:31.008505  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:31.010303  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:31.011134  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:31.011850  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:31.012547  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:31.013288  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:31.013944  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:31.060957  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 132685us
W20260709 13:02:31.054507  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 59584.2us
W20260709 13:02:31.053188  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 340043us
I20260709 13:02:31.061847  2268 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:31.063175  1641 raft_consensus.cc:2410] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:02:31.063156  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:31.064538  1642 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:31.060773  1639 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:31.026508  4505 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:31.076057  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:31.076924  1643 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:31.077883  4950 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:31.078294  5004 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.079288  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:31.079975  1565 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:31.082430  5001 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:31.083187  5001 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:31.085131  1534 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:31.085525  5001 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:31.086102  5001 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:31.025400  4546 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:31.087587  4831 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:31.088171  4831 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:31.091133  4446 consensus_queue.cc:1059] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:31.092265  4446 consensus_queue.cc:1059] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:31.093632  4962 raft_consensus.cc:3055] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:31.094061  4962 raft_consensus.cc:740] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:31.077926  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:31.058723  4939 raft_consensus.cc:697] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:31.074297  4977 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:31.070009  4669 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:31.139145  1715 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:31.139631  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 54679us
I20260709 13:02:31.129086  4962 consensus_queue.cc:260] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.140702  4886 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:31.141501  4997 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:31.123554  4937 raft_consensus.cc:3055] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:31.156054  4962 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:31.134228  5004 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.170037  1569 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:31.160145  4937 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:02:31.222410  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 369635us
W20260709 13:02:31.225890  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 158876us
I20260709 13:02:31.231992  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:31.232841  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:31.233541  1565 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:31.236658  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:31.239375  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:31.239784  1712 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:31.241355  4048 raft_consensus.cc:3055] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:31.241837  4048 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:31.242815  4048 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.243916  4048 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:31.248538  4939 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.253347  4785 raft_consensus.cc:3055] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:31.253868  4785 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:31.257052  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.256088  1681 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:31.269318  1681 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:31.276319  5001 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:31.277010  5001 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.277446  5001 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:31.236079  4761 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:02:31.281956  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:31.289731  4733 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:31.293020  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:31.293497  4974 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.543s	user 0.005s	sys 0.005s
I20260709 13:02:31.294255  5007 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:31.270943  4831 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.226150  4886 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:02:31.254787  4785 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.294008  4974 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.307426  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:31.297036  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:31.324996  1714 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:02:31.310484  4785 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:31.296868  5004 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:31.294894  5007 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:31.297830  4831 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:31.298933  4996 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:31.339610  4989 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.309458  4942 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.344370   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:31.335942  5007 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:31.299185  4937 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.326759  4898 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:31.361513  4831 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:31.385933  1609 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:31.392783  1643 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:31.375787  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 298426us
I20260709 13:02:31.394495  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:31.399092  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 129836us
I20260709 13:02:31.351380  4989 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.366415  4996 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:31.477047  1644 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:31.394170  4544 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:31.460562  1534 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.548s (SENT) [suppressed 9 similar messages]
W20260709 13:02:31.491420  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.494841  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.448529  4937 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:31.491662  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:31.490449  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:31.493057  1711 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:31.499965  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:31.493883  4544 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:31.500440  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:31.502542  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:31.493503  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 98985.7us
I20260709 13:02:31.518863  2302 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:31.510207  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:31.510637  1563 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:31.510545  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:31.511276  4998 raft_consensus.cc:3055] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:31.511634  5008 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:31.513037  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:31.513487  5010 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:31.515517  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.515861  4989 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:31.515944  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:31.518023  1716 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:31.521479  1717 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:31.550943  2300 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:31.390090  4831 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:31.487803  4959 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.033s	sys 0.018s
W20260709 13:02:31.561415  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.567945  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:31.592888  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.593447  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.594096  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:31.568287  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:31.567960  2299 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:31.595034  1563 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:31.568804  4938 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.569612  4942 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:31.570820  4898 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:31.570739  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:31.543449  4998 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:31.495604  4996 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:31.572072  5008 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.573117  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:31.573836  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:31.574224  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:31.558032  4048 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:31.575237  5010 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.576519  1565 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:31.576853  4733 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:31.579843  1568 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:31.580500  4989 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:31.581708  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1933) took 1006 ms. Trace:
I20260709 13:02:31.582772  4204 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:31.583663  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:31.584247  4974 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:31.585162  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:31.585228  4814 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:31.642529  2296 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:31.587387  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:31.591058  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:31.597040  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:31.621526  5014 log.cc:927] Time spent T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.281s	user 0.001s	sys 0.001s
W20260709 13:02:31.598624  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 80778.8us
I20260709 13:02:31.547397  4544 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:31.503588  5013 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:02:31.612573  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 152056us
I20260709 13:02:31.575068  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:31.564563  4967 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:31.596206  4959 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.907s	user 0.033s	sys 0.019s
I20260709 13:02:31.599050  4650 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.602532  4403 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.628422  1635 rpcz_store.cc:276] 0709 13:02:30.575711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.626322 (+ 50611us) service_pool.cc:224] Handling call
0709 13:02:30.668374 (+ 42052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.779572 (+111198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.581368 (+801796us) spinlock_profiling.cc:243] Waited 85.1 ms on lock 0x7b54000d07d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:31.581623 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195683200}
I20260709 13:02:31.670082  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:31.574772  5001 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:31.603209  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:31.648569  1563 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:31.655277  1643 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:02:31.680457  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:31.663561  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 64981.9us
I20260709 13:02:31.681270  1561 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:31.665366  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:31.610103  4938 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.676367  4204 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.677079  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:31.682111  1562 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:31.645561  4733 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.677333  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:31.677685  4650 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.678202  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:31.696153  2305 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:31.642096  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:31.676848  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:31.678541  4831 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:31.684437  1710 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:31.689903  1682 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:31.695341  2297 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:31.685281  4689 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:02:31.708122  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:31.708576  4204 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:31.709446  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:31.678546  4403 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.719091  1709 raft_consensus.cc:2368] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7.
I20260709 13:02:31.724329  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:31.685319  4963 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.731189  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:31.662304  4814 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.668708  4989 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.690902  4879 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:02:31.736814  1683 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:31.725929  4831 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:31.684619  4967 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:31.747975  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:31.684924  5013 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.754113  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:31.776132  2300 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:31.773669  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:31.779932  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 90175.8us
I20260709 13:02:31.777395  1717 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:31.752497  5017 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.783641  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:31.781340  1683 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:31.801635  1682 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:31.890612  1636 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:31.755486  5026 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:31.733898  4963 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:31.913489  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:31.747555  4204 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:02:31.919932  1711 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:31.932641  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:31.935937  5008 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:31.780658  4446 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:31.939177  2302 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:31.776924  5040 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:31.959707  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.965835  5025 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 13:02:31.901834  4939 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.731637  4998 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:31.891165  4960 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:31.967235  2300 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:31.966152  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:31.912144  5039 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.970750  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:31.910575  4831 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:31.983109  2304 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:02:31.915004  4887 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:31.940322  4942 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:32.031133  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:32.061992  1712 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:32.061736  4942 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:32.035008  5039 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.063103  1643 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:02:32.063776  4887 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.064714  1637 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:31.927464  4950 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.098s
W20260709 13:02:32.036303  5028 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
W20260709 13:02:32.038004  5034 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.258s	user 0.000s	sys 0.000s
W20260709 13:02:32.038825  5024 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.354s	user 0.000s	sys 0.001s
I20260709 13:02:32.039290  5010 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:32.069345  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:31.986370  5030 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.383s	user 0.001s	sys 0.000s
I20260709 13:02:32.045315  4403 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:32.043450  4996 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:32.051236  5003 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:02:31.915863  5017 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.015939  5026 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:32.016824  4939 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:31.981590  4446 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.057312  4938 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:32.057408  4831 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:32.060473  4963 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:32.060911  1638 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:02:31.974144  4733 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:32.069938  5027 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.293s	user 0.000s	sys 0.000s
I20260709 13:02:32.046156  4976 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:32.096071  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:32.072217  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:32.052858  4967 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:32.071568  4546 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:32.324817  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:32.343035  1638 pending_rounds.cc:85] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 3
I20260709 13:02:32.307304  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:32.343618  1638 pending_rounds.cc:107] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 4.4
W20260709 13:02:32.314070  1714 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.974s	user 0.004s	sys 0.004s
I20260709 13:02:32.344349  1714 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
W20260709 13:02:32.217337  2305 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.002s
I20260709 13:02:32.345234  2305 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:32.307729  1712 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:32.345970  1715 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
W20260709 13:02:32.141424  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 212590us
I20260709 13:02:32.354609  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1876) took 1062 ms. Trace:
I20260709 13:02:32.354681  1609 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:32.355394  1714 rpcz_store.cc:276] 0709 13:02:31.282186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.292364 (+ 10178us) service_pool.cc:224] Handling call
0709 13:02:31.296775 (+  4411us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b58001e4598. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:02:32.344852 (+1048077us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5236992}
W20260709 13:02:32.358080  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 813216us
I20260709 13:02:32.361423  1638 raft_consensus.cc:2889] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:32.213290  5025 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:02:32.362852  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:32.363023  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:32.369771  2302 raft_consensus.cc:2410] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:32.358704  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:32.372275  2300 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:32.307917  5039 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:32.364270  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:32.236641  4937 consensus_meta.cc:287] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.001s
I20260709 13:02:32.029701  5008 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:32.372246  1712 raft_consensus.cc:2376] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8. Re-sending same reply.
I20260709 13:02:32.353885  4967 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:32.341465  4998 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:32.374742  1607 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.006s	sys 0.014s
I20260709 13:02:32.343880  5010 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:32.386351  1607 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.007s	sys 0.014s
W20260709 13:02:32.377125  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 575628us
I20260709 13:02:32.379307  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:32.386749  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.000s	sys 0.022s
W20260709 13:02:32.380285  2301 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.014s	sys 0.009s
W20260709 13:02:32.387428  2301 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.014s	sys 0.009s
I20260709 13:02:32.369165  5001 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:32.383697  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 726974us
I20260709 13:02:32.363822  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:32.386888  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 792921us
W20260709 13:02:32.387292  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.000s	sys 0.022s
I20260709 13:02:32.350589  4887 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:32.387734  1682 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:32.387282  1709 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:32.397393  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:32.398263  1715 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:32.397049  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:32.384408  5007 consensus_meta.cc:287] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.011s	user 0.011s	sys 0.001s
W20260709 13:02:32.400929  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.359736  4938 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:02:32.388201  5039 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:32.368784  4737 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:02:32.413519  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:32.415405  1532 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:32.414497  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:32.418601  1607 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:32.419934  1607 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:32.405798  4785 consensus_meta.cc:287] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.070s	user 0.007s	sys 0.007s
I20260709 13:02:32.469868  4785 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:32.470490  4785 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:32.416976  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:32.427680  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:32.428277  4887 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:02:32.418022  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 59008.9us
I20260709 13:02:32.431828  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:32.478438  2298 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.036s	sys 0.101s
I20260709 13:02:32.432528  4669 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.433035  4989 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:32.433318  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:32.391670  4689 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:32.410169  4685 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:32.413159  4668 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:32.394425  2299 raft_consensus.cc:2955] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:32.436378  4733 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:32.437199  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 49556.1us
W20260709 13:02:32.493342  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 79919.8us
I20260709 13:02:32.493700  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:32.494550  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:32.494771  1682 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:32.439213  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:32.439399  4491 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:32.441273  4997 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:32.445943  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 652) took 1291 ms. Trace:
I20260709 13:02:32.400278  4967 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.449846  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2007) took 1084 ms. Trace:
I20260709 13:02:32.500478  1682 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:32.450500  4743 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:32.452394  4963 raft_consensus.cc:3055] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:32.454303  4938 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:32.460544  5013 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:32.461504  5039 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:32.464123  4446 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:32.465356  5020 raft_consensus.cc:3055] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:32.465881  4546 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:32.467474  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:32.468307  4544 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.797s	user 0.008s	sys 0.002s
I20260709 13:02:32.407590  5007 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.369330  5026 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:32.532945  2300 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
W20260709 13:02:32.478839  2298 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.036s	sys 0.101s
W20260709 13:02:32.488696  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 55498.8us
I20260709 13:02:32.490298  4689 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.498111  1710 rpcz_store.cc:276] 0709 13:02:31.155103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.155244 (+   141us) service_pool.cc:224] Handling call
0709 13:02:31.156657 (+  1413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.699336 (+542679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.445593 (+746257us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b54003377d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:32.445823 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1201717120}
I20260709 13:02:32.409659  5010 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.509137  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 651) took 1435 ms. Trace:
I20260709 13:02:32.512166  4963 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:32.505702  4668 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:32.491808  5038 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:32.476792  1568 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:32.531610  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:32.523520  2268 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:32.500567  4962 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:32.507414  4950 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:32.500324  1716 rpcz_store.cc:276] 0709 13:02:31.365480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.365662 (+   182us) service_pool.cc:224] Handling call
0709 13:02:31.376968 (+ 11306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.564998 (+188030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.449184 (+884186us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5400339fd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:32.449755 (+   571us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323917184}
W20260709 13:02:32.517202  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.543941  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:32.513217  1682 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:32.522365  4989 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:32.597159  4960 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:32.496707  4669 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.615531  4669 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:32.618014  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 661) took 1105 ms. Trace:
I20260709 13:02:32.624261  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:32.539799  4544 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.471616  4785 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.497782  4887 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.513386  4645 consensus_queue.cc:1059] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:02:32.531243  4997 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:32.526340  5001 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:32.644404  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 120991us
I20260709 13:02:32.532644  5020 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:32.522646  4946 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.045s	sys 0.085s
I20260709 13:02:32.505352  5052 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:02:32.546247  1718 rpcz_store.cc:276] 0709 13:02:31.074368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.087163 (+ 12795us) service_pool.cc:224] Handling call
0709 13:02:31.087603 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:31.088321 (+   718us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:31.224247 (+135926us) log.cc:844] Serialized 14 byte log entry
0709 13:02:31.388465 (+164218us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:31.465834 (+ 77369us) log.cc:844] Serialized 26 byte log entry
0709 13:02:31.465979 (+   145us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:31.466013 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.466088 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.508830 (+1042742us) raft_consensus.cc:1685] finished
0709 13:02:32.508882 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.509058 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":187797,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":43,"raft.queue_time_us":83873,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":40213,"thread_start_us":254815,"threads_started":2,"wal-append.queue_time_us":400539}
I20260709 13:02:32.552100  4743 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:32.586503  4668 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:32.586975  1464 catalog_manager.cc:5697] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:32.589946  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 565) took 1996 ms. Trace:
I20260709 13:02:32.592159  4967 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:32.594555  4650 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:32.595515  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51688.3us
W20260709 13:02:32.596293  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 83179us
I20260709 13:02:32.719960  1682 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:32.637559  4505 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:32.626137  1639 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:32.527746  4491 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:32.527088  4922 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 13:02:32.627885  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.628588  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:32.631762  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 211916us
I20260709 13:02:32.632642  1609 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:32.637998  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:32.618665  1717 rpcz_store.cc:276] 0709 13:02:31.512358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.616920 (+1104562us) service_pool.cc:224] Handling call
0709 13:02:32.617331 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.617598 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:32.617637 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:32.617694 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:32.617735 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.617805 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.617943 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.641933  1644 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:32.552472  4937 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:02:32.580804  4939 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:32.646992  1637 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:32.647854  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.648532  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:32.652809  1718 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:32.706908  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 61822.4us
I20260709 13:02:32.708326  4962 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:02:32.706171  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 62517.5us
I20260709 13:02:32.721218  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:32.624496  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:32.651162  4946 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.045s	sys 0.086s
I20260709 13:02:32.644021  4997 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.649108  4685 raft_consensus.cc:697] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:32.713521  1564 rpcz_store.cc:276] 0709 13:02:30.593418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.593588 (+   170us) service_pool.cc:224] Handling call
0709 13:02:30.631530 (+ 37942us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:30.977950 (+346420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:30.978012 (+    62us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:31.058493 (+ 80481us) log.cc:844] Serialized 52 byte log entry
0709 13:02:31.356922 (+298429us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:31.590375 (+233453us) spinlock_profiling.cc:243] Waited 35 ms on lock 0x7b1800065680. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb822a6 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b
0709 13:02:32.588731 (+998356us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b1800065680. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb1455 00007fd7a56993c7 00007fd7a1cab9a7 00007fd7a1cbb0bf
0709 13:02:32.588890 (+   159us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.589124 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.589619 (+   495us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400250154. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:32.589654 (+    35us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.589697 (+    43us) raft_consensus.cc:1685] finished
0709 13:02:32.589707 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.589864 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":235795,"prepare.queue_time_us":107428,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":265277,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":228,"spinlock_wait_cycles":322593664,"thread_start_us":394697,"threads_started":2,"wal-append.queue_time_us":441019,"wal-append.run_cpu_time_us":2137,"wal-append.run_wall_time_us":1111471}
W20260709 13:02:32.837080  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 204565us
I20260709 13:02:32.837646  1717 pending_rounds.cc:85] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 3
I20260709 13:02:32.853092  1717 pending_rounds.cc:107] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:02:32.853430  1717 raft_consensus.cc:2889] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:32.838877  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 119899us
I20260709 13:02:32.722707  4737 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.844290  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:32.648190  5043 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:32.845659  1643 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:32.651271  5017 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:32.859030  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:32.759896  2302 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:02:32.854848  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:32.862095  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 662) took 1307 ms. Trace:
I20260709 13:02:32.862339  2302 pending_rounds.cc:85] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 8
I20260709 13:02:32.863896  2302 pending_rounds.cc:107] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:02:32.864281  2302 raft_consensus.cc:2889] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:32.843166  1568 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:32.846988  5054 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.859112  4650 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:32.858968  5043 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.862689  1718 rpcz_store.cc:276] 0709 13:02:31.545991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.652241 (+1106250us) service_pool.cc:224] Handling call
0709 13:02:32.652636 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.847632 (+194996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.853550 (+  5918us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:32.855192  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 134058us
I20260709 13:02:32.866149  1534 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:32.769904  4967 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.870106  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:32.879038  1644 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:32.880380  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:32.848290  5004 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:32.854297  4997 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:32.866103  4610 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.867578  1464 catalog_manager.cc:5697] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 4 to 6, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 6 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:32.890463  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:32.907729  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:32.865548  5013 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:32.910449  1718 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 10: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:32.879958  5021 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:32.654192  5039 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.857357  4797 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:02:32.890419  5038 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:32.899824  4922 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:32.948079  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1875) took 1454 ms. Trace:
I20260709 13:02:32.911626  4610 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:32.962340  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 92335.7us
I20260709 13:02:32.960227  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2017) took 1391 ms. Trace:
I20260709 13:02:32.967703  1718 rpcz_store.cc:276] 0709 13:02:31.569004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.948282 (+1379278us) service_pool.cc:224] Handling call
0709 13:02:32.948641 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.959869 (+ 11228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:32.959901 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:32.959947 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:32.959986 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.960047 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.960167 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.969918  5067 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.970636  5067 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.962122  1464 catalog_manager.cc:5697] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 4 to 5, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 5 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:32.959522  4962 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.981137  4743 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:32.945276  5013 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:32.865324  4737 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.983748  5013 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:32.984747  4737 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:32.962576  5007 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:32.987043  4685 consensus_queue.cc:237] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:32.963822  1713 rpcz_store.cc:276] 0709 13:02:31.281794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.287089 (+  5295us) service_pool.cc:224] Handling call
0709 13:02:31.517790 (+230701us) spinlock_profiling.cc:243] Waited 7.16 ms on lock 0x7b58001e4598. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:02:31.517987 (+   197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.655719 (+137732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.655751 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:31.655796 (+    45us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:31.655827 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.722400 (+1066573us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54001351d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:32.722455 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.735727 (+ 13272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307264640}
I20260709 13:02:32.994692  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:32.894661  4967 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:32.957254  4446 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:33.005967  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:33.007347  1643 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:32.989316  1464 catalog_manager.cc:5697] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 3 to 6, term changed from 4 to 6, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 6 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:33.014405  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:32.961812  5049 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:33.010849  2269 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:32.991271  5055 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.911111  5004 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:33.004963  5015 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:33.009298  5018 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:02:32.993126  5046 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:33.006301  4939 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:33.039672  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:33.004288  5010 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.027846  4922 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:33.030223  5061 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
W20260709 13:02:33.030827  5057 log.cc:927] Time spent T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:02:33.030272  5058 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:33.018321  4937 consensus_queue.cc:1059] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:02:33.057966  4937 raft_consensus.cc:3055] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:32.987097  4491 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.058537  4937 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:33.024912  4887 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:32.868419  5020 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.015695  4544 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.066247  4922 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.067974  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:33.069677  4922 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.070266  5008 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:33.032354  4946 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:33.033682  5055 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.005788  5026 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:33.009349  4446 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:33.045849  1464 catalog_manager.cc:5697] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 5 to 7, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 7 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:33.040198  4939 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:33.047948  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 53382.5us
I20260709 13:02:33.048767  1713 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:33.086545  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 663) took 1516 ms. Trace:
I20260709 13:02:33.087002  1713 rpcz_store.cc:276] 0709 13:02:31.570035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.019512 (+1449477us) service_pool.cc:224] Handling call
0709 13:02:33.041348 (+ 21836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.086220 (+ 44872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.086482 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.891456  5054 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:33.051237  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 207059us
I20260709 13:02:33.091544  2268 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:33.051959  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 172473us
I20260709 13:02:33.070469  5015 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:33.072512  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:33.074589  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 1947) took 1510 ms. Trace:
W20260709 13:02:33.031920  5060 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:02:33.047827  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:33.097033  1682 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:33.083039  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 944) took 1413 ms. Trace:
I20260709 13:02:33.098109  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 666) took 1368 ms. Trace:
I20260709 13:02:33.098596  1713 rpcz_store.cc:276] 0709 13:02:31.729766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.088096 (+1358330us) service_pool.cc:224] Handling call
0709 13:02:33.090554 (+  2458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.097749 (+  7195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.097779 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.097821 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:33.097855 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.097917 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.098048 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.095249  1644 rpcz_store.cc:276] 0709 13:02:31.563883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.564164 (+   281us) service_pool.cc:224] Handling call
0709 13:02:33.074312 (+1510148us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400124b54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:33.074470 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2412469504}
I20260709 13:02:33.101240  1718 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:33.101086  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:33.059787  4937 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.078239  5008 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:33.081810  4946 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.115125  5069 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
I20260709 13:02:33.116467  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:33.117161  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:33.115593  4946 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:33.117808  1564 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:33.118069  5081 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.080868  5001 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:33.102085  2304 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:33.122843  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:33.110567  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:33.123788  1643 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:33.089320  4446 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:33.089149  4939 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:33.112063  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:33.129899  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:33.098245  5052 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:33.098562  2298 rpcz_store.cc:276] 0709 13:02:31.669656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.936414 (+266758us) service_pool.cc:224] Handling call
0709 13:02:31.936736 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:32.543216 (+606480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:32.543265 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:32.576396 (+ 33131us) log.cc:844] Serialized 26 byte log entry
0709 13:02:32.863543 (+287147us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:33.054067 (+190524us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.082623 (+ 28556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.082717 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.082761 (+    44us) raft_consensus.cc:1685] finished
0709 13:02:33.082776 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.082952 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22375,"prepare.queue_time_us":188303,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":8501,"raft.queue_time_us":309805,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":7497,"thread_start_us":995212,"threads_started":3,"wal-append.queue_time_us":317669}
W20260709 13:02:33.115592  5042 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.444s	user 0.001s	sys 0.000s
I20260709 13:02:33.119822  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2013) took 1560 ms. Trace:
W20260709 13:02:33.108137  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.067507  4963 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.121363  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:33.121738  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:33.135824  1644 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:33.130421  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:33.130661  1639 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:33.137358  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:33.131891  1717 rpcz_store.cc:276] 0709 13:02:31.559751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.736136 (+1176385us) service_pool.cc:224] Handling call
0709 13:02:32.837373 (+101237us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:32.882034 (+ 44661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:32.882070 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:32.911821 (+ 29751us) log.cc:844] Serialized 52 byte log entry
0709 13:02:33.080547 (+168726us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:33.080593 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.080676 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.119566 (+ 38890us) raft_consensus.cc:1685] finished
0709 13:02:33.119592 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.119757 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3777,"raft.queue_time_us":3945,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":12176,"thread_start_us":79731,"threads_started":1,"wal-append.queue_time_us":141869}
W20260709 13:02:33.140074  4733 consensus_meta.cc:287] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.640s	user 0.007s	sys 0.004s
I20260709 13:02:33.140162  1639 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:33.140688  4733 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:33.119453  5008 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:33.142235  1635 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
W20260709 13:02:33.143064  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.052593  5046 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.143505  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.130491  4403 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:33.109436  5026 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:02:33.131453  4544 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.131956  5052 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:33.198139  1563 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:33.200408  1569 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:33.203550  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.208628  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:33.209818  1607 connection.cc:256] RPC call timeout handler was delayed by -1.4923905020000348s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20790000000000003s for a response
I20260709 13:02:33.209302  1569 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:02:33.210525  1569 pending_rounds.cc:85] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 6
I20260709 13:02:33.118854  5081 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.211232  1464 catalog_manager.cc:5697] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:33.212083  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.210829  1569 pending_rounds.cc:107] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:02:33.213353  1569 raft_consensus.cc:2889] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:33.213022  1563 raft_consensus.cc:2410] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:33.194204  5007 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:33.217789  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.223248  1564 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:33.223698  1565 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:33.216293  1464 catalog_manager.cc:5697] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 4 to 5, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:33.205292  4937 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:33.204672  5052 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:33.214265  4998 consensus_meta.cc:287] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.823s	user 0.008s	sys 0.005s
I20260709 13:02:33.251919  1567 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:33.256000  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.257886  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.258754  1568 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:33.259922  4668 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.260025  5082 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:02:33.260655  4668 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.261116  1568 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:02:33.263028  5064 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:02:33.264521  4668 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.265849  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:33.266799  1569 raft_consensus.cc:2368] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8.
I20260709 13:02:33.243893  4879 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.267560  4879 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:33.258046  5079 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:02:33.219779  4610 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.270972  4879 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.210268  4544 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.275493  5008 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.277804  4544 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.278759  5008 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:33.281297  5029 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 1.764s	user 0.010s	sys 0.004s
I20260709 13:02:33.287014  5080 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:33.287384  5071 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.287600  5080 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.287982  5080 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:33.287940  5071 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:33.289285  5045 consensus_queue.cc:1059] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.102s
W20260709 13:02:33.290086  5063 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
W20260709 13:02:33.292177  5072 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:02:33.215569  4963 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:33.218750  5004 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:33.251576  5038 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.318692  1637 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:33.259071  4967 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:33.400192  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 255268us
I20260709 13:02:33.256407  5007 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:33.418107  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 207664us
W20260709 13:02:33.419528  1607 connection.cc:256] RPC call timeout handler was delayed by -1.4184263220000162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
I20260709 13:02:33.397876  5021 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.425396  1464 catalog_manager.cc:5697] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 7 to 8, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 8 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:33.427693  1464 catalog_manager.cc:5697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:33.435686  4939 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.438228  4939 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.438328  4886 raft_consensus.cc:3055] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:33.439132  1641 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:33.440157  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.447177  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:33.449651  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:33.451987  1607 connection.cc:256] RPC call timeout handler was delayed by -1.1026294140000346s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21810000000000002s for a response
W20260709 13:02:33.452667  1607 connection.cc:256] RPC call timeout handler was delayed by -1.0900449190000359s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
W20260709 13:02:33.453074  1607 connection.cc:256] RPC call timeout handler was delayed by -1.0106678780000493s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
I20260709 13:02:33.454133  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:33.430083  1464 catalog_manager.cc:5697] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:33.462234  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:33.464248  4733 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:33.444094  4886 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:33.459168  4403 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.467479  4403 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:33.467417  4886 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.468313  4788 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:33.468819  4886 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:33.469025  5008 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:33.469924  4403 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:33.459489  4544 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.470906  4544 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.463222  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:33.471220  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:33.475675  1639 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:33.459838  5015 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.468842  4788 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:33.496814  5015 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:33.497462  4788 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.491336  4879 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.499769  4879 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:33.501386  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2012) took 2008 ms. Trace:
I20260709 13:02:33.505314  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:33.505563  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 119019us
I20260709 13:02:33.505995  2304 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:33.501788  1464 catalog_manager.cc:5697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 4 to 7, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:33.473670  1682 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:33.465426  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:33.493239  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:33.466210  4733 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:33.509368  1682 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:33.413234  5038 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:33.574831  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 175782us
W20260709 13:02:33.567878  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.614254  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 148916us
I20260709 13:02:33.510073  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:33.617295  2305 raft_consensus.cc:2410] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:02:33.618206  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 108980us
I20260709 13:02:33.619341  1682 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:33.550201  1464 catalog_manager.cc:5697] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 5 to 8, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:33.473578  5039 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:33.625025  1607 leader_election.cc:400] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7.
I20260709 13:02:33.501926  1716 rpcz_store.cc:276] 0709 13:02:31.492572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.581575 (+1089003us) service_pool.cc:224] Handling call
0709 13:02:32.631494 (+ 49919us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:32.736003 (+104509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:32.736035 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:32.777244 (+ 41209us) log.cc:844] Serialized 26 byte log entry
0709 13:02:33.004692 (+227448us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:33.195145 (+190453us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.196291 (+  1146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.500376 (+304085us) spinlock_profiling.cc:243] Waited 6.06 ms on lock 0x7b18003837a0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb822a6 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419
0709 13:02:33.501058 (+   682us) spinlock_profiling.cc:243] Waited 95.3 ms on lock 0x7b5400268ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:33.501115 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.501158 (+    43us) raft_consensus.cc:1685] finished
0709 13:02:33.501170 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.501307 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31009,"prepare.queue_time_us":42704,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":9426,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":26373,"spinlock_wait_cycles":233178112,"thread_start_us":253579,"threads_started":2,"wal-append.queue_time_us":208020}
I20260709 13:02:33.491645  5008 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:33.626777  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:33.498836  4788 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:33.429884  5081 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.445397  5021 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.497566  5071 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:33.650679  1562 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:33.539723  4998 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:33.651368  1464 catalog_manager.cc:5697] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 1 to 5, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:33.640587  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:33.656881  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:33.644899  5043 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:33.652688  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 238246us
I20260709 13:02:33.668452  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.653509  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:33.657446  1716 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:33.577704  4668 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.627245  4733 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:33.650552  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.630488  5008 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:33.623757  4785 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:33.627008  4204 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:33.687799  1464 catalog_manager.cc:5697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 2 to 5, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:33.630606  4959 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:33.672139  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:33.694046  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.687242  1566 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:33.699185  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.684346  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:33.701057  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.701620  5039 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:33.701807  1568 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
W20260709 13:02:33.703166  4950 consensus_meta.cc:287] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.102s	user 0.012s	sys 0.007s
I20260709 13:02:33.703812  4950 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:33.654999  4998 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:33.707278  4998 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:33.707835  5055 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.710522  4814 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.711179  4645 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:33.659603  5026 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.713114  5026 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.676105  4761 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.662246  5071 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:33.715318  5071 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:33.715726  5071 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:33.716490  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1886) took 2226 ms. Trace:
I20260709 13:02:33.716895  1715 rpcz_store.cc:276] 0709 13:02:31.489850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.426904 (+937054us) service_pool.cc:224] Handling call
0709 13:02:32.427231 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.427422 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:32.630278 (+202856us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.715895 (+1085617us) spinlock_profiling.cc:243] Waited 5.57 ms on lock 0x7b18004371a0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a
0709 13:02:33.716134 (+   239us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.716183 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:33.716224 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.716289 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.716428 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":343314,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":3824,"spinlock_wait_cycles":12820864,"thread_start_us":447225,"threads_started":2,"wal-append.queue_time_us":381374}
I20260709 13:02:33.719183  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:33.719875  1717 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:33.720717  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:33.721763  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:33.718779  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:33.724536  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.725188  1565 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:33.726495  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:33.727502  1715 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
I20260709 13:02:33.541119  4946 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.712353  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:33.736680  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.742277  1566 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:02:33.749377  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:33.767228  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 74663.4us
I20260709 13:02:33.722788  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:33.768793  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:33.770653  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 86427.3us
I20260709 13:02:33.772745  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:33.793932  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:33.798130  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:33.774029  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:33.680809  5008 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:33.779397  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:33.712275  4544 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.780016  1714 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:33.799588  2302 raft_consensus.cc:2410] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:02:33.801092  4942 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:33.802608  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:33.805668  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:33.806221  5067 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:33.806960  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:33.750702  5046 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.811146  4785 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:33.811640  4785 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:33.812037  4785 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:33.814507  5015 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:33.780318  5054 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.816289  4946 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:33.782496  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:33.792001  1711 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 660) took 2267 ms (client timeout 1735 ms). Trace:
I20260709 13:02:33.785320  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:33.740458  4950 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.795439  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:33.792431  4879 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:33.752192  5039 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:33.818567  2305 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:33.686862  4204 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:33.825506  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:33.818102  1711 rpcz_store.cc:269] 0709 13:02:31.512062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.547236 (+ 35174us) service_pool.cc:224] Handling call
0709 13:02:33.779090 (+2231854us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54003dc7d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:33.779217 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":635045632}
I20260709 13:02:33.799633  2303 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
W20260709 13:02:33.790697  4937 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.000s
I20260709 13:02:33.686862  4668 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.829344  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:33.830084  4887 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:33.830232  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:33.847558  2305 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:33.792196  4645 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:33.831477  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:33.821190  5045 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:33.845822  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:33.859898  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.853322  2302 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:33.750650  4761 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:33.857332  1566 raft_consensus.cc:1240] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:33.824503  4950 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.867293  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.829186  5038 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:33.868858  1565 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:33.828727  4204 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:33.865545  4963 consensus_meta.cc:287] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.549s	user 0.005s	sys 0.009s
I20260709 13:02:33.875500  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.877075  1560 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:33.840431  4946 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:33.878201  1711 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:33.879679  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 50453.7us
I20260709 13:02:33.889580  1534 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:33.870870  5045 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.883064  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:33.872555  1564 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:33.893285  2298 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:33.890813  2303 raft_consensus.cc:2410] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:02:33.880223  4446 consensus_meta.cc:287] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.746s	user 0.012s	sys 0.000s
I20260709 13:02:33.879901  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:33.868811  5021 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.895648  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:33.883889  5081 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.902016  5081 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:33.885635  4610 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:33.902729  4650 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:33.903246  4650 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:33.903203  4610 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:33.904268  5081 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:33.904861  5081 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:33.905762  4610 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:33.897354  5038 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:02:33.908556  5086 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.328s	user 0.003s	sys 0.000s
I20260709 13:02:33.909368  4761 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:33.911754  4668 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.912595  1567 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:33.913770  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2020) took 2343 ms. Trace:
I20260709 13:02:33.914158  1718 rpcz_store.cc:276] 0709 13:02:31.569978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.968739 (+1398761us) service_pool.cc:224] Handling call
0709 13:02:33.003657 (+ 34918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.101792 (+ 98135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.913531 (+811739us) spinlock_profiling.cc:243] Waited 97.2 ms on lock 0x7b5400148cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:33.913710 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223618432}
I20260709 13:02:33.882371  4645 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:33.884295  5078 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:33.916867  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:33.920356  1565 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:33.922637  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 295962us
I20260709 13:02:33.890043  4946 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:33.925191  2304 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:33.894858  5045 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:33.931757  1569 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:33.933610  2301 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:33.898015  5026 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:33.936882  1711 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:02:33.928349  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 46019.7us
I20260709 13:02:33.899847  5092 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.921545  5021 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:33.945303  4645 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.947890  5043 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.004s
W20260709 13:02:33.952730  5052 consensus_meta.cc:287] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.694s	user 0.008s	sys 0.007s
I20260709 13:02:33.953231  5052 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.953423  5092 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:33.956247  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:33.961403  4998 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:33.964713  4785 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:33.967298  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:33.968111  1711 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:33.969409  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:33.971930  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.934891  4946 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:34.027508  1717 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:34.048334  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:34.027647  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 83016.8us
W20260709 13:02:34.049072  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 65992.4us
I20260709 13:02:34.034876  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:34.039121  1533 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:33.900082  4544 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:02:33.935122  5001 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:33.901162  4446 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.058123  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:34.057682  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:34.059442  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 163904us
I20260709 13:02:34.049599  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:33.958362  4650 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:34.062170  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.059589  2302 raft_consensus.cc:2368] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8.
I20260709 13:02:34.053231  1565 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
W20260709 13:02:34.061513  4997 consensus_meta.cc:287] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.114s	user 0.008s	sys 0.007s
I20260709 13:02:34.060163  4505 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:34.077312  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:34.064203  4546 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.518s	user 0.009s	sys 0.010s
I20260709 13:02:34.078055  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:34.081074  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:34.081847  2304 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:34.083303  2304 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:34.053436  5015 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:34.115760  5015 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.085002  4997 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.066255  5001 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:34.067164  4785 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:34.087107  4544 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:34.086993  1607 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:34.089563  4733 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:34.092484  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 792) took 1005 ms. Trace:
I20260709 13:02:34.065466  4831 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:34.080286  4505 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.100488  4446 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:34.071096  5039 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:34.068575  5106 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:02:34.107601  4950 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:34.109594  5071 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:34.081380  5043 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:34.110829  4048 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.112380  4998 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:34.118844  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1885) took 2629 ms. Trace:
I20260709 13:02:34.141906  1709 rpcz_store.cc:276] 0709 13:02:31.489581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.417570 (+927989us) service_pool.cc:224] Handling call
0709 13:02:32.426885 (+  9315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.574330 (+147445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:32.574362 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:32.574426 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:32.574477 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.118636 (+1544159us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400239954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:34.118682 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.118792 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":292802304}
I20260709 13:02:34.143261  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:34.144385  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:34.118667  5015 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:34.119593  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2380) took 1496 ms. Trace:
I20260709 13:02:34.147512  1561 rpcz_store.cc:276] 0709 13:02:32.623459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.626471 (+  3012us) service_pool.cc:224] Handling call
0709 13:02:32.626837 (+   366us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:32.647528 (+ 20691us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:32.865805 (+218277us) log.cc:844] Serialized 14 byte log entry
0709 13:02:32.973878 (+108073us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:33.000164 (+ 26286us) log.cc:844] Serialized 52 byte log entry
0709 13:02:33.000301 (+   137us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:33.000504 (+   203us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.008578 (+  8074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.008700 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:33.265222 (+256522us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b1800190c60. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:33.290194 (+ 24972us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b1800190c60. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb4522 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:34.115983 (+825789us) spinlock_profiling.cc:243] Waited 6.34 ms on lock 0x7b1800190c60. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:34.119380 (+  3397us) raft_consensus.cc:1685] finished
0709 13:02:34.119401 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.119538 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":137530,"prepare.queue_time_us":251963,"prepare.run_cpu_time_us":139,"prepare.run_wall_time_us":1145,"raft.queue_time_us":50367,"raft.run_cpu_time_us":484,"raft.run_wall_time_us":74163,"spinlock_wait_cycles":42737664,"thread_start_us":197341,"threads_started":2,"wal-append.queue_time_us":123822}
I20260709 13:02:34.150403  2298 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:34.150537  4669 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.113592  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 943) took 2444 ms. Trace:
I20260709 13:02:34.115253  1636 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:34.151659  2299 rpcz_store.cc:276] 0709 13:02:31.669420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.934933 (+265513us) service_pool.cc:224] Handling call
0709 13:02:31.935315 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.371256 (+435941us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:32.596573 (+225317us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.013659 (+417086us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.018656 (+  4997us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.028134 (+  9478us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.028204 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:33.028275 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.113262 (+1084987us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b54003a2d54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:34.113308 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.113515 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19653,"prepare.queue_time_us":206738,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":115719,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":13642,"spinlock_wait_cycles":1002130176,"thread_start_us":381454,"threads_started":3,"wal-append.queue_time_us":286509}
I20260709 13:02:34.115042  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:34.114462  4650 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:34.155194  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 68325.4us
I20260709 13:02:34.156381  1607 leader_election.cc:400] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8.
I20260709 13:02:34.156701  1607 leader_election.cc:403] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:02:34.122058  1710 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:34.123870  5001 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:34.149060  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.157459  4544 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:34.157941  4544 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:34.158313  4544 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:34.158416  5007 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:34.125473  2296 rpcz_store.cc:276] 0709 13:02:33.087389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.087539 (+   150us) service_pool.cc:224] Handling call
0709 13:02:33.097711 (+ 10172us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.098074 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.098110 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:33.713038 (+614928us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b2c00047a38. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:02:33.713457 (+   419us) log.cc:844] Serialized 26 byte log entry
0709 13:02:33.879256 (+165799us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:33.879308 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.879382 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.092154 (+212772us) raft_consensus.cc:1685] finished
0709 13:02:34.092175 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.092410 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29799,"raft.queue_time_us":4371,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":9250,"spinlock_wait_cycles":27967360,"thread_start_us":165156,"threads_started":1,"wal-append.queue_time_us":228864}
W20260709 13:02:34.161123  5085 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.568s	user 0.001s	sys 0.000s
I20260709 13:02:34.161537  4785 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:34.124186  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:34.162740  5106 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8.
I20260709 13:02:34.163069  4650 consensus_queue.cc:1059] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:34.163275  4785 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:34.164178  4669 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.164723  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 687) took 1131 ms. Trace:
I20260709 13:02:34.126746  4831 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:34.126966  4505 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:34.200192  1681 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:34.127538  5097 log.cc:927] Time spent T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
W20260709 13:02:34.128168  5099 log.cc:927] Time spent T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:02:34.129813  5052 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:34.131108  5008 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.196769  5007 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:34.133950  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 689) took 1099 ms. Trace:
I20260709 13:02:34.133965  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.168433  4491 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:34.134248  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:34.168989  5020 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:34.226509  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.228106  1644 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:34.169375  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:34.170082  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:34.170231  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:34.170872  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:34.172817  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:34.173645  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:34.173893  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:34.205601  5004 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:34.139372  5043 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.174691  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:34.236449  1568 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:34.139451  1711 raft_consensus.cc:2393] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:34.140112  1534 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:34.175973  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:34.176146  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:34.239234  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.239755  2302 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:34.239881  1644 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:34.140547  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1883) took 2651 ms. Trace:
I20260709 13:02:34.240547  5008 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:34.240728  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:34.240892  1714 rpcz_store.cc:276] 0709 13:02:31.489024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.393607 (+904583us) service_pool.cc:224] Handling call
0709 13:02:33.779817 (+1386210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.800017 (+ 20200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.140371 (+340354us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54003dc7d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:34.140494 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3156030336}
I20260709 13:02:34.141054  4048 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:34.243216  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 667) took 2499 ms. Trace:
I20260709 13:02:34.243624  1713 rpcz_store.cc:276] 0709 13:02:31.743756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.120569 (+1376813us) service_pool.cc:224] Handling call
0709 13:02:33.132056 (+ 11487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.144118 (+ 12062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.144159 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:33.186423 (+ 42264us) log.cc:844] Serialized 26 byte log entry
0709 13:02:33.658151 (+471728us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:33.726394 (+ 68243us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.726631 (+   237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.726703 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.242994 (+516291us) raft_consensus.cc:1685] finished
0709 13:02:34.243015 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.243157 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11549,"prepare.queue_time_us":81303,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":18398,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":18195,"thread_start_us":438962,"threads_started":1,"wal-append.queue_time_us":673446}
I20260709 13:02:34.245388  4048 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:34.245975  4048 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:34.246313  4048 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:34.246647  4048 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:34.165225  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.180241  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1890) took 1310 ms. Trace:
I20260709 13:02:34.253042  1717 rpcz_store.cc:276] 0709 13:02:32.870683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.807895 (+937212us) service_pool.cc:224] Handling call
0709 13:02:33.808221 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.053180 (+244959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.179989 (+126809us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54001478d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:34.180175 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373557632}
I20260709 13:02:34.126658  4733 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:34.123660  5101 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:02:34.182407  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:34.182709  5081 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:34.098613  5045 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.186993  1718 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:34.283465  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 658) took 2790 ms. Trace:
I20260709 13:02:34.237790  5043 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:34.187290  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 296707us
I20260709 13:02:34.187747  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:34.188517  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.286222  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:34.190403  1532 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:34.198184  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.197830  1637 rpcz_store.cc:276] 0709 13:02:33.033594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.044029 (+ 10435us) service_pool.cc:224] Handling call
0709 13:02:33.044432 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.329920 (+285488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.164457 (+834537us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54000d0054. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:34.164665 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":630079360}
I20260709 13:02:34.201785  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:34.287524  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 657) took 2797 ms. Trace:
W20260709 13:02:34.156405  5091 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.529s	user 0.002s	sys 0.000s
I20260709 13:02:34.295148  2268 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:34.297436  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:34.298481  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:34.299232  1567 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:34.121886  4997 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:34.283947  1712 rpcz_store.cc:276] 0709 13:02:31.511348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.597858 (+1086510us) service_pool.cc:224] Handling call
0709 13:02:32.598319 (+   461us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:32.647770 (+ 49451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:32.647800 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:32.716038 (+ 68238us) log.cc:844] Serialized 52 byte log entry
0709 13:02:32.999450 (+283412us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:33.017140 (+ 17690us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.017331 (+   191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.017399 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.207928 (+1190529us) spinlock_profiling.cc:243] Waited 3.2 ms on lock 0x7b1800437500. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:34.283013 (+ 75085us) raft_consensus.cc:1685] finished
0709 13:02:34.283137 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.283372 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43114,"prepare.queue_time_us":45298,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":53348,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":59981,"spinlock_wait_cycles":7354240,"thread_start_us":137473,"threads_started":1,"wal-append.queue_time_us":268832}
I20260709 13:02:34.301110  2268 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:34.135519  4689 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:34.212177  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:34.132843  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:34.310707  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 177932us
I20260709 13:02:34.214223  1681 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:34.215564  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:34.215252  1641 rpcz_store.cc:276] 0709 13:02:33.034233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.038715 (+  4482us) service_pool.cc:224] Handling call
0709 13:02:33.039153 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.460050 (+420897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.133752 (+673702us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b540032cdd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:34.133897 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":919332608}
I20260709 13:02:34.128429  4204 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:34.225557  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:34.226816  1638 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:34.200502  4960 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:34.231443  1569 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:34.126415  5092 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:34.207455  5001 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.233062  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:34.233558  2298 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:34.328840  1560 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:34.241442  1644 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:34.140504  5071 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:34.158442  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:34.248126  1564 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
W20260709 13:02:34.251025  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 110991us
I20260709 13:02:34.240728  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:34.206040  4946 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.255616  5004 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.230904  5026 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:34.137799  5039 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.357594  5039 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:34.363759  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.364537  1639 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:34.365762  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.267012  4831 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:34.367103  4546 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:02:34.237555  5080 consensus_meta.cc:287] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.947s	user 0.017s	sys 0.000s
I20260709 13:02:34.268603  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:34.272208  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 210795us
I20260709 13:02:34.371855  1609 leader_election.cc:400] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8.
I20260709 13:02:34.277664  4048 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.353557  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:34.288017  1710 rpcz_store.cc:276] 0709 13:02:31.490208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.580011 (+1089803us) service_pool.cc:224] Handling call
0709 13:02:32.587060 (+  7049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.157160 (+1570100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.287191 (+130031us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b540033a9d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:34.287432 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3530053888}
I20260709 13:02:34.291971  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:34.226491  5007 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:34.309407  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 119075us
I20260709 13:02:34.316619  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.310556  1561 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:34.317080  2304 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:34.379921  1635 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:34.322685  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:34.330005  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.380899  1712 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:34.330847  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 172480us
I20260709 13:02:34.345351  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:34.250350  4491 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.350803  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:34.346362  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:34.388192  1638 raft_consensus.cc:2410] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:02:34.328317  5015 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:34.345381  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:34.321552  5081 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:34.320405  5069 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:34.320878  4963 consensus_queue.cc:1059] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:02:34.374812  1642 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:34.375128  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:34.376049  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:34.264485  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:34.347934  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 133810us
I20260709 13:02:34.390563  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:34.368438  5080 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.324401  4879 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:34.387167  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:34.328052  4997 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:02:34.396982  5110 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:02:34.651109  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:34.345017  4733 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:34.342175  4446 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:34.345577  5001 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:34.573742  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 56045.5us
I20260709 13:02:34.479833  4491 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:34.520246  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.341799  4689 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:34.396888  4048 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:34.565109  1710 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:34.519217  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.669579  1637 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
W20260709 13:02:34.616078  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 351736us
I20260709 13:02:34.671087  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:34.369710  5008 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:34.636515  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:34.327215  4960 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.625747  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.674793  4960 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:34.674561  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.387702  4788 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:34.675580  1635 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:34.685462  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.451675  5100 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:34.657150  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 191450us
I20260709 13:02:34.695298  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.699605  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:34.657994  1712 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:34.658279  1716 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 9: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:34.397279  4785 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.273s
I20260709 13:02:34.623243  5069 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.605195  5082 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:34.632084  4998 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:34.378307  5026 raft_consensus.cc:697] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:34.398860  1644 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:34.654109  5052 consensus_queue.cc:1059] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:02:34.652130  4733 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:34.466800  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:34.652688  4997 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:34.451845  4939 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.213s
I20260709 13:02:34.734457  1644 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:34.704766  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:34.705818  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.711117  1710 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:34.676527  5106 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:34.724705   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:34.707708  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:34.757429  1635 raft_consensus.cc:2376] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8. Re-sending same reply.
I20260709 13:02:34.763813  1641 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:34.675613  4788 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.736720  1712 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:34.740375  1637 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:34.767546  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:34.727160  4204 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:34.711573  5045 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:34.659250  4879 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.709722  5069 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:34.771200  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.732843  4733 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:34.709504  4998 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.703075  4960 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:34.808331  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:34.812575  1644 raft_consensus.cc:2410] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:02:34.835114  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:34.810491  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:34.836128  1710 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:34.837288  1712 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:34.728274  5082 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.840945  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.841972  1644 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:34.842901  1637 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:34.844519  1637 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:34.702993  5100 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:34.784222  4544 consensus_meta.cc:287] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.003s
I20260709 13:02:34.849300  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:34.850199  1718 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:34.703332  4974 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
W20260709 13:02:34.863368  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.794065  4788 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:34.859447  4204 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:34.883203  1565 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.008s	sys 0.020s
I20260709 13:02:34.830082  4946 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:34.832598  4998 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:34.841928  5082 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:34.886571  1565 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.008s	sys 0.020s
I20260709 13:02:34.833045  5004 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:34.901724  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:34.902698  1710 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:34.860930  5007 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:34.907334  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:34.883113  5078 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.020s	sys 0.007s
W20260709 13:02:34.910337  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 523267us
I20260709 13:02:34.885030  5008 raft_consensus.cc:697] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:34.901042  4204 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:34.899956  5015 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:34.869076  4544 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.918210  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:34.928464  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 151190us
I20260709 13:02:34.925858  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:34.918183  5078 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.021s	sys 0.008s
I20260709 13:02:34.924037  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2370) took 1477 ms. Trace:
I20260709 13:02:34.926980  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.932147  1560 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:02:34.937031  1560 pending_rounds.cc:85] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 7
I20260709 13:02:34.937839  1560 pending_rounds.cc:107] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:02:34.938079  1560 raft_consensus.cc:2889] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:34.937026  1637 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
W20260709 13:02:34.922977  5043 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.004s
W20260709 13:02:34.917956  5114 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.336s	user 0.001s	sys 0.000s
W20260709 13:02:34.935127  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 264152us
I20260709 13:02:34.935261  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:34.942965  1563 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:34.939386  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 270572us
I20260709 13:02:34.939383  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:34.944175  1637 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:34.930078  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:34.941540  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 676) took 1245 ms. Trace:
W20260709 13:02:34.933096  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.003649  1714 rpcz_store.cc:276] 0709 13:02:33.696026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.242055 (+546029us) service_pool.cc:224] Handling call
0709 13:02:34.242385 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.242600 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:34.242857 (+   257us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.941193 (+698336us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.941259 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:34.941308 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.941369 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.941489 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4771,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":50101,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":15938,"thread_start_us":697892,"threads_started":2,"wal-append.queue_time_us":663625}
I20260709 13:02:35.004961  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:35.005870  1533 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.006570  5081 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:35.007032  5081 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.007328  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:34.989270  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 703) took 1005 ms. Trace:
I20260709 13:02:34.981178  4048 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:34.956415  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:35.009848  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:34.945966  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:34.936484  1643 rpcz_store.cc:276] 0709 13:02:33.446400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.448201 (+  1801us) service_pool.cc:224] Handling call
0709 13:02:33.448517 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.455582 (+  7065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.455625 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:33.459962 (+  4337us) log.cc:844] Serialized 26 byte log entry
0709 13:02:34.120154 (+660192us) spinlock_profiling.cc:243] Waited 6.95 ms on lock 0x7b180037d5c0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:02:34.249920 (+129766us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:34.647612 (+397692us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.923649 (+276037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.923724 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.923769 (+    45us) raft_consensus.cc:1685] finished
0709 13:02:34.923859 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.923987 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6645,"prepare.queue_time_us":459813,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":4945,"raft.queue_time_us":133668,"raft.run_cpu_time_us":6417,"raft.run_wall_time_us":64443,"spinlock_wait_cycles":15994240,"thread_start_us":782453,"threads_started":3,"wal-append.queue_time_us":852606}
I20260709 13:02:34.981178  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:34.938115  5080 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:34.926389  4204 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:35.023569  4544 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:34.944914  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 293879us
I20260709 13:02:35.027052  5127 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:35.027956  4505 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.813s	user 0.008s	sys 0.004s
I20260709 13:02:34.918474  5026 consensus_queue.cc:237] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:35.028569  1681 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:35.028429  4505 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:34.980242  5048 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:02:35.031651  5048 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:35.031890  1681 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.032130  5048 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:35.032784  5137 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:35.033224  5137 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.032956  5048 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.033701  5137 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.034281  5048 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:35.034436  1642 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:34.948802  4785 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:35.035159  1641 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:35.035405  4997 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:34.998095  5045 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:02:35.037071  5045 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:02:35.037243  5118 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:02:35.037499  5045 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:35.007421  5081 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:34.948189  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:35.008908  1640 rpcz_store.cc:276] 0709 13:02:33.983854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.034188 (+ 50334us) service_pool.cc:224] Handling call
0709 13:02:34.060380 (+ 26192us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:34.060627 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.060666 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:34.061235 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:02:34.295058 (+233823us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:34.295107 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.295188 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:34.989038 (+693850us) raft_consensus.cc:1685] finished
0709 13:02:34.989058 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.989200 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":193924,"threads_started":1,"wal-append.queue_time_us":308718}
I20260709 13:02:35.016727  5082 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.040491  5082 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.017838  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:35.019203  5088 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:34.943172  1682 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:35.050464  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 107377us
I20260709 13:02:35.051514  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:35.027628  5127 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.031569  5080 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:35.033119  1681 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:34.950014  5001 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:35.036127  4974 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.940344  5122 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:34.943547  4446 raft_consensus.cc:697] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:35.045845  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:35.051888  1643 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:02:35.051993  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 106109us
I20260709 13:02:35.185375  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:34.956377  1639 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:35.056181  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:34.952008  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:34.946429  5043 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.062691  5004 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:35.072160  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:35.057129  5082 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:35.086108  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:35.192588  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:35.062878  5088 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.038575  5045 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:35.143770  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 110707us
I20260709 13:02:35.179209  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:35.061568  5015 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.062080  5008 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:35.179416  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 111582us
I20260709 13:02:35.206444  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:35.206588  1682 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:35.179579  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:35.003412  4946 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:35.071986  5128 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:35.057265  4785 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:02:35.180275  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2033) took 1731 ms (client timeout 1966 ms). Trace:
I20260709 13:02:35.180406  1717 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:35.188894  5051 raft_consensus.cc:3055] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:35.182510  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 677) took 1453 ms. Trace:
I20260709 13:02:35.231786  1713 rpcz_store.cc:276] 0709 13:02:33.729387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.245247 (+515860us) service_pool.cc:224] Handling call
0709 13:02:34.245521 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.438411 (+192890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:34.438456 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.438531 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:34.438584 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.088935 (+650351us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54001c1454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:35.182287 (+ 93352us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.182445 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443071872}
I20260709 13:02:35.183262  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:35.236965  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:35.197724  4403 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:35.189059  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.198331  5071 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:35.057359  5080 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.193059  2298 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:35.217377  4048 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:35.223191  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:35.186784  1637 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:35.183975  5122 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.258198  4505 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:35.258527  5122 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:02:35.225232  1716 rpcz_store.cc:269] 0709 13:02:33.448388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.679846 (+231458us) service_pool.cc:224] Handling call
0709 13:02:35.097606 (+1417760us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b5400339fd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:35.180210 (+ 82604us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2111068928}
W20260709 13:02:35.203083  5121 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.262s	user 0.006s	sys 0.003s
W20260709 13:02:35.203861  5020 consensus_meta.cc:287] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.968s	user 0.008s	sys 0.007s
I20260709 13:02:35.225397  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:35.198513  5004 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.189536  1568 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:35.238839  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:35.284292  2299 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:35.284740  1565 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.198881  5124 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:35.242511  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 53557us
I20260709 13:02:35.216616  5104 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:35.037267  4204 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:35.181073  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:35.194056  4974 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:35.206014  4937 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:35.255182  4446 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.188313  4886 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:02:35.257069  4879 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:35.229583  5051 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:35.242252  5071 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.259356  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:35.317472  2302 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:35.260869  4998 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:35.263271  5047 raft_consensus.cc:3055] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:35.263784  4788 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.266768  4669 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:35.267271  4785 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:35.320329  5047 raft_consensus.cc:740] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:35.225288  4946 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.321957  4788 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:35.180517  5090 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:35.322609  1709 raft_consensus.cc:2368] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10.
I20260709 13:02:35.322803  5090 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:35.267599  5080 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:35.324870  1569 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:35.322640  4946 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:35.323800  4998 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:35.239029  4997 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:35.281610  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 56280.4us
I20260709 13:02:35.327320  4997 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:35.328265  4997 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:35.330724  4998 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.331343  4998 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.332324  5071 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:35.332662  5026 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:35.333242  5026 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.334231  4998 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:35.280643  1638 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.033s	sys 0.038s
W20260709 13:02:35.334903  1638 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.033s	sys 0.038s
I20260709 13:02:35.335529  5026 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:35.336148  5100 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:35.339275  5127 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:35.335582  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 712) took 1153 ms. Trace:
I20260709 13:02:35.340598  1638 rpcz_store.cc:276] 0709 13:02:34.182221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.451283 (+269062us) service_pool.cc:224] Handling call
0709 13:02:34.451624 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.451830 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:34.651790 (+199960us) log.cc:844] Serialized 14 byte log entry
0709 13:02:35.335234 (+683444us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:35.335300 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:35.335350 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.335418 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.335527 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":291232,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":186362,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":850933,"threads_started":2}
I20260709 13:02:35.341972  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:35.343446  5137 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.347298  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:35.348866  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:35.347301  1714 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:35.348428  5069 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.293175  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:35.352234  2299 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:35.344522  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:35.293826  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:35.258256  4733 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.299197  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:35.299428  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:35.365157  1563 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:35.275172  5045 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:35.289774  5004 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:35.292640  5001 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:35.301544  4544 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:02:35.351496  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.295068  5021 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:35.354247  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 60521.8us
I20260709 13:02:35.297349  1715 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
W20260709 13:02:35.295493  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 287056us
I20260709 13:02:35.409627  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.410142  1533 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.412250  1562 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:35.357590  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:35.415491  1683 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.416874  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:35.417665  1718 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:35.419809  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.418794  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:35.291635  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:35.290859  5104 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:35.424976  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:35.427675  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:35.428975  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:35.429188  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.432894  1607 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:35.358976  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:35.370522  1565 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:35.374363  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:35.372977  2302 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:35.472046  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:35.375684  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2389) took 1187 ms. Trace:
I20260709 13:02:35.474747  1564 rpcz_store.cc:276] 0709 13:02:34.173027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.173331 (+   304us) service_pool.cc:224] Handling call
0709 13:02:35.359945 (+1186614us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b54002977d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:35.360057 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36881920}
I20260709 13:02:35.376322  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:35.376719  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:35.376322  5141 log.cc:927] Time spent T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:02:35.377064  5043 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:35.503381  4505 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:35.379276  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:35.504491  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:35.380059  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:35.505334  2296 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:02:35.505560  1642 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:35.505860  2296 pending_rounds.cc:85] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 8
I20260709 13:02:35.505945  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:35.506486  2296 pending_rounds.cc:107] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:02:35.506778  1562 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:02:35.506942  2296 raft_consensus.cc:2889] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:35.298591  4403 consensus_queue.cc:1059] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:02:35.510015  4403 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:35.381861  5088 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:35.314769  5084 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 1.728s	user 0.002s	sys 0.015s
I20260709 13:02:35.383172  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:35.517436  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 987) took 2445 ms. Trace:
I20260709 13:02:35.500458  1533 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:35.384181  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:35.413642  4788 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:35.385941  1532 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.387202  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:35.374477  5127 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:35.555634  1715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1895) took 1915 ms (client timeout 2099 ms). Trace:
I20260709 13:02:35.496150  1681 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:35.503274  5007 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:35.503512  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:35.503940  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.566418  1564 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:35.303763  4937 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.384671  4733 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:35.496127  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 80762.7us
I20260709 13:02:35.574400  1683 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.517905  2297 rpcz_store.cc:276] 0709 13:02:33.071744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.073065 (+  1321us) service_pool.cc:224] Handling call
0709 13:02:33.073474 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.073732 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.073760 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:33.074252 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:02:33.203670 (+129418us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:33.203710 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.203783 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:35.097983 (+1894200us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:35.517104 (+419121us) raft_consensus.cc:1685] finished
0709 13:02:35.517130 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:35.517365 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23681792,"thread_start_us":129032,"threads_started":1,"wal-append.queue_time_us":255852}
W20260709 13:02:35.556388  1715 rpcz_store.cc:269] 0709 13:02:33.640060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.774335 (+134275us) service_pool.cc:224] Handling call
0709 13:02:35.555393 (+1781058us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54000b0254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:35.555545 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3395758848}
I20260709 13:02:35.532999  1718 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:35.558182  1642 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:35.519537  4505 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:35.409078  5040 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:35.564561  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 68515.7us
I20260709 13:02:35.568181  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:35.581821  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.411209  5001 raft_consensus.cc:697] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:35.571499  5129 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:35.589937  1713 raft_consensus.cc:1240] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:35.504490  5043 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:35.582269  2296 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:35.594053  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:35.573344  1562 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:35.422812  4785 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:35.520709  4939 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:35.522400  5039 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:35.386124  5126 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.194s
W20260709 13:02:35.643568  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.569031  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:35.578171  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:35.645390  2305 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
W20260709 13:02:35.645526  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 67519.3us
I20260709 13:02:35.586426  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:35.587213  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.521699  4998 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:35.585974  5134 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:35.653071  1714 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:35.415817  5021 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.598518  5137 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:35.522612  4546 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:02:35.602408  1715 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:35.603521  5069 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:35.604259  5020 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:35.607357  5104 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:35.321604  5047 consensus_queue.cc:260] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:35.323792  5090 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.619211  5146 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:35.620093  5130 raft_consensus.cc:3055] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:35.688983  1561 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
W20260709 13:02:35.648115  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 61049.6us
I20260709 13:02:35.692412  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:35.598176  5133 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:02:35.702752  1683 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.654256  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:35.655301  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 330660us
I20260709 13:02:35.521541  4610 consensus_queue.cc:1059] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 13:02:35.323140  5051 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.663125  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:35.666074  5135 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:35.580446  4505 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.623653  5129 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:35.637665  4939 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.689096  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:35.722735  2301 raft_consensus.cc:2393] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:35.689981  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:35.555161  5127 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.577919  5007 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:35.694538  5130 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:35.695431  4546 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:35.697381  4974 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:35.656767  4960 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:35.578758  5080 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.707949  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:35.710974  4998 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:35.686578  5146 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.676934  5043 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:02:35.740039  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:35.741504  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:35.739068  5130 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:35.742262  4610 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:35.742703  4610 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:35.742897  5130 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:35.743597  4610 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:35.744863  4610 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:35.747285  4546 raft_consensus.cc:3055] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:35.747952  4546 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:35.748857  5007 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:35.728374  5127 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:35.706229  5047 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:35.693344  5039 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:35.639220  5040 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.709829  5020 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:35.730906  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:35.708902  1683 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.728206  5051 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:35.734390  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.719578  4505 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:35.734397  4446 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:35.726500  4939 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:35.753471  2297 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:35.723896  1565 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:35.720860  5129 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:35.754456  1712 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 10. Ops: []
W20260709 13:02:35.793653  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 360869us
I20260709 13:02:35.758445  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:35.760423  4403 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:35.760778  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:35.749228  4546 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:35.763716  2298 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:02:35.771680  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 62825.7us
I20260709 13:02:35.802557  1683 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.773414  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 1963) took 1193 ms. Trace:
I20260709 13:02:35.740274  5146 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:35.796077  5136 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:02:35.787590  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.731866  4204 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:35.756371  2268 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:35.803297  1641 rpcz_store.cc:276] 0709 13:02:34.580184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.799167 (+218983us) service_pool.cc:224] Handling call
0709 13:02:35.035000 (+235833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.035577 (+   577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.773193 (+737616us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b5400125050. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:35.773347 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":450523520}
I20260709 13:02:35.691237  5001 consensus_queue.cc:237] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:35.752437  4960 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:35.869180  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.807615  2297 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:35.757527  5069 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:35.796802  4879 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:35.831082  4937 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:35.832684  5149 log.cc:927] Time spent T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:02:35.846297  5015 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:35.868420  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 112144us
I20260709 13:02:35.874866  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:35.806718  1683 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.809471  5021 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:35.876329  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 69708.6us
I20260709 13:02:35.814091  5007 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:35.792107  5039 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:35.878592  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 52202.4us
I20260709 13:02:35.886330  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.879410  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:35.774773  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 254833us
I20260709 13:02:35.884840  1642 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:02:35.887578  1642 pending_rounds.cc:85] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 3
I20260709 13:02:35.887943  1642 pending_rounds.cc:107] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:02:35.888252  1642 raft_consensus.cc:2889] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:35.870891  4546 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:35.892992  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.887004  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:35.900301  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.900408  2268 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:35.901832  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 515979us
I20260709 13:02:35.872180  5137 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:35.905718  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:35.904953  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:35.902586  1532 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:35.888302  1716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.025s	sys 0.050s
I20260709 13:02:35.911722  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:35.912101  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 351042us
I20260709 13:02:35.912304  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:35.912495  1644 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
W20260709 13:02:35.911690  1716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.025s	sys 0.050s
I20260709 13:02:35.913750  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:35.914103  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:35.915259  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:35.915845  1638 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:35.912942  1635 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:35.889024  4937 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:35.922937  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:35.923450  5007 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:35.924556  4491 consensus_queue.cc:1059] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:35.925684  4491 raft_consensus.cc:3055] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:35.926033  4491 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:35.929044  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:36.079591  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 150647us
I20260709 13:02:35.885058  5015 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:35.892757  5047 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
W20260709 13:02:35.917584  5081 consensus_meta.cc:287] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.878s	user 0.013s	sys 0.000s
W20260709 13:02:36.018946  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.908697  5128 consensus_meta.cc:287] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.662s	user 0.006s	sys 0.004s
I20260709 13:02:35.981304  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:36.074522  1715 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:35.992594  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 90093.7us
I20260709 13:02:35.891997  4403 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:36.074882  1534 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:36.146365  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 71560.8us
I20260709 13:02:35.896788  4997 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:35.890337  4204 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:35.926723  4974 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.129s
I20260709 13:02:35.892426  4669 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 13:02:35.915498  5008 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:35.908212  4785 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:35.915700  5051 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:35.915843  4887 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:35.930203  4544 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:36.097095  2301 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
W20260709 13:02:36.123493  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.133227  5080 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:35.928009  5137 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.079218  5092 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:36.144554  1532 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:36.119571  5045 consensus_meta.cc:287] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.731s	user 0.009s	sys 0.005s
I20260709 13:02:36.109331  4446 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:36.121798  5021 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:36.150616  2304 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.021s	sys 0.023s
I20260709 13:02:36.166250  1534 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:36.214434  2304 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.021s	sys 0.024s
I20260709 13:02:36.148499  5128 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:36.197422  2301 pending_rounds.cc:85] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 7
I20260709 13:02:36.216712  2301 pending_rounds.cc:107] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:02:36.217260  2301 raft_consensus.cc:2889] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:36.217628  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51528.7us
I20260709 13:02:36.081043  4960 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:36.141207  5081 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:36.150098  5159 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:36.094740  4491 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:36.237253  4491 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:36.235710  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 687) took 1052 ms. Trace:
I20260709 13:02:36.239106  1712 rpcz_store.cc:276] 0709 13:02:35.183238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.183940 (+   702us) service_pool.cc:224] Handling call
0709 13:02:35.184227 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.794141 (+609914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.235363 (+441222us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54003395d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:36.235621 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1311104384}
I20260709 13:02:36.240271  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:36.240435  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:36.241086  5107 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:36.229916  1681 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:36.241684  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:36.155041  5158 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:36.167511  4204 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:36.243641  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:36.243930  2305 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:36.243744  4204 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.244675  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:36.245210  1712 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:36.245901  4204 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:36.246465  4204 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:36.235658  5156 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:02:36.235911  5159 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:36.261224  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 116935us
I20260709 13:02:36.243273  5158 raft_consensus.cc:697] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:36.168802  5051 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:36.231978  1535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.019s	sys 0.028s
W20260709 13:02:36.273026  1535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.019s	sys 0.028s
W20260709 13:02:36.256470  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.276130  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:36.277999  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:36.278908  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:36.279431  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:36.213846  5021 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.225983  4610 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:36.217556  4689 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:36.279526  1561 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:02:36.279860  1564 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:02:36.222011  4887 raft_consensus.cc:697] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:36.282614  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:36.234419  5126 consensus_queue.cc:1059] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:36.286279  1640 raft_consensus.cc:2393] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:36.278256  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:36.168531  4785 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.290333  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:36.230427  5092 raft_consensus.cc:697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:36.262645  1532 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:36.294740  1567 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:02:36.303184  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.225728  5160 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:36.311064  1533 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:36.310166  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:36.279073  5124 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.215039  4544 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.314399  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:36.256402  5131 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:36.255805  5069 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:36.280354  5051 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:36.323781  1564 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:36.255702  5080 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:36.269011  4937 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:02:36.324301  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 46187.5us
I20260709 13:02:36.358847  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:36.327800  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:36.331614  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:36.335887  1712 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:36.293569  5021 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:36.342700  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 119371us
I20260709 13:02:36.288445  4610 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.331614  4733 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:36.382381  5175 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:36.383333  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:36.383450  5175 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:36.350948  4403 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:02:36.392086  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.392082  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 129564us
I20260709 13:02:36.397528  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:36.343538  5160 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:36.403120  4048 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:02:36.405244  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.384790  4610 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:36.407709  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:36.407915  5187 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.408393  5015 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:36.408584  5187 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:36.411649  5004 consensus_meta.cc:287] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.916s	user 0.005s	sys 0.006s
I20260709 13:02:36.412137  5004 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:36.415346  5153 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
W20260709 13:02:36.421746  5152 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 13:02:36.384657  4204 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
W20260709 13:02:36.423900  5162 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:02:36.426436  5169 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:02:36.427268  5122 consensus_meta.cc:287] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.143s	user 0.012s	sys 0.000s
I20260709 13:02:36.429396  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 963) took 1373 ms. Trace:
I20260709 13:02:36.429911  2300 rpcz_store.cc:276] 0709 13:02:35.055957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.056071 (+   114us) service_pool.cc:224] Handling call
0709 13:02:35.179813 (+123742us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:35.195926 (+ 16113us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:35.195956 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:35.251330 (+ 55374us) log.cc:844] Serialized 52 byte log entry
0709 13:02:35.657567 (+406237us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:35.657624 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.657725 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:36.429186 (+771461us) raft_consensus.cc:1685] finished
0709 13:02:36.429205 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.429328 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4510,"raft.queue_time_us":18961,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":16795,"thread_start_us":339842,"threads_started":1,"wal-append.queue_time_us":641887}
I20260709 13:02:36.437846  4048 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:36.440483  4048 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:36.442595  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 811) took 1017 ms. Trace:
W20260709 13:02:36.395617  5127 consensus_meta.cc:287] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.641s	user 0.011s	sys 0.000s
W20260709 13:02:36.455054  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.465978  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.443150  2302 rpcz_store.cc:276] 0709 13:02:35.425653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.473375 (+ 47722us) service_pool.cc:224] Handling call
0709 13:02:35.473805 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:35.497444 (+ 23639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:35.497486 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:35.497948 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 13:02:35.849323 (+351375us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b180036ece0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:02:36.147372 (+298049us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:36.147416 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.147490 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:36.429114 (+281624us) raft_consensus.cc:1685] finished
0709 13:02:36.429137 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.442510 (+ 13373us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27350,"raft.queue_time_us":23914,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":10613632,"thread_start_us":297858,"threads_started":1,"wal-append.queue_time_us":767504}
W20260709 13:02:36.465934  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 69601.3us
I20260709 13:02:36.343283  5124 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.475255  1567 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:02:36.460839  1638 consensus_meta.cc:287] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.000s
I20260709 13:02:36.376132  5166 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:36.477294  1638 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
W20260709 13:02:36.435014  5071 consensus_meta.cc:287] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.102s	user 0.012s	sys 0.000s
I20260709 13:02:36.478899  1565 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:36.476655  1532 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:36.350778  5131 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:36.494683  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:36.495568  1564 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 6: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:36.356648  5069 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.414754  5004 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:36.364791  5080 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:36.488886  1565 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:36.499755  1564 raft_consensus.cc:1240] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:36.496870  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 809) took 1071 ms. Trace:
I20260709 13:02:36.367961  4937 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:36.477025  5130 consensus_meta.cc:287] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.733s	user 0.011s	sys 0.003s
I20260709 13:02:36.516561  4048 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.519523  4048 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:36.500478  2299 rpcz_store.cc:276] 0709 13:02:35.425029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.520996 (+ 95967us) service_pool.cc:224] Handling call
0709 13:02:35.568279 (+ 47283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.568538 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:36.007362 (+438824us) log.cc:844] Serialized 14 byte log entry
0709 13:02:36.496368 (+489006us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:36.496433 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:36.496487 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.496569 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.496795 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":660299,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":55697,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":61,"spinlock_wait_cycles":19200,"thread_start_us":799359,"threads_started":3}
I20260709 13:02:36.521636  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:36.400763  4733 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.525451  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:36.527335  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:36.427706  4887 consensus_queue.cc:237] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.528439  2301 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:36.427752  5122 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:36.535991  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:36.536598  1566 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:36.532059  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:36.538616  1564 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
W20260709 13:02:36.508545  4505 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.701s	user 0.010s	sys 0.001s
I20260709 13:02:36.500725  5126 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:36.526800  1565 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:36.546610  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 165033us
I20260709 13:02:36.540540  1681 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:36.531376  4939 consensus_meta.cc:287] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.725s	user 0.011s	sys 0.000s
I20260709 13:02:36.575771  1566 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:36.571321  1718 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:36.577597  4879 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:36.580745  5122 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:36.575306  4939 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.564862  1681 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:36.477648  5127 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:36.543663  5048 consensus_meta.cc:287] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.509s	user 0.013s	sys 0.003s
I20260709 13:02:36.596510  1565 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:36.542619  4204 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:36.501327  5071 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:36.605283  5122 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:36.616214  5124 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:36.616087  5122 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:36.620536  4610 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:36.472771  5092 consensus_queue.cc:237] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.449216  5158 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.560271  5137 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:36.582476  5081 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:36.628363  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 102656us
I20260709 13:02:36.618633  5122 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:36.652303  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 175731us
W20260709 13:02:36.635212  5146 consensus_meta.cc:287] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.811s	user 0.005s	sys 0.006s
I20260709 13:02:36.562490  4505 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:36.616230  4937 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:36.634514  5181 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.665879  1561 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:36.670840  4048 raft_consensus.cc:2764] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:02:36.672472  1565 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:36.639849  5195 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:36.675889  4505 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:36.629093  4204 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:36.658066  4785 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:36.642207  5159 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:36.667414  4544 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:36.704771  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.705016  5090 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:36.705466  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.707628  5157 raft_consensus.cc:3055] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:02:36.707919  5180 log.cc:927] Time spent T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:02:36.708154  5157 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:02:36.709061  5186 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:02:36.710587  5198 consensus_queue.cc:1059] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:02:36.711902  5198 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:36.712284  5198 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:36.713467  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 709) took 2549 ms. Trace:
I20260709 13:02:36.713153  5198 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:36.713901  1639 rpcz_store.cc:276] 0709 13:02:34.163913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.371087 (+207174us) service_pool.cc:224] Handling call
0709 13:02:34.435701 (+ 64614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.186086 (+750385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.713138 (+1527052us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b54002d8a54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:36.713368 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1184236800}
I20260709 13:02:36.714797  5196 consensus_queue.cc:1059] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:36.716216  4505 raft_consensus.cc:3055] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:36.716624  4505 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:36.717087  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:36.717491  4505 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:36.718755  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:36.720151  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:36.594678  5160 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.720289  5039 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:36.726593  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:36.702584  4831 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:36.693579  5182 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:02:36.731477  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:36.668286  5146 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.732041  2301 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
W20260709 13:02:36.736145  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 91563.1us
I20260709 13:02:36.738998  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2045) took 1797 ms. Trace:
I20260709 13:02:36.728348  5198 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:36.725824  4610 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:36.751318  4733 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:36.751977  4939 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:36.753159  2268 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:36.754998  5187 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:36.756647  4204 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:36.786969  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:36.739425  1569 rpcz_store.cc:276] 0709 13:02:34.941230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.941391 (+   161us) service_pool.cc:224] Handling call
0709 13:02:34.941815 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:35.325316 (+383501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.738709 (+1413393us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b54003e7454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:36.738902 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":880439168}
I20260709 13:02:36.686971  5195 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:36.763378  5126 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:36.764519  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:36.791631  2304 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:36.766595  5080 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:36.781257  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 127478us
I20260709 13:02:36.784505  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1900) took 2080 ms. Trace:
I20260709 13:02:36.757820  5069 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:36.673522  5181 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.729713  5188 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:36.788166  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:36.803730  1710 rpcz_store.cc:276] 0709 13:02:34.704051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.903830 (+199779us) service_pool.cc:224] Handling call
0709 13:02:34.914349 (+ 10519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.266432 (+352083us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:35.266462 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:35.266504 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:35.266533 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.768048 (+1501515us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b54001473d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:36.784272 (+ 16224us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.784426 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":809262976}
W20260709 13:02:36.788980  5182 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:36.806874  1712 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:36.807017  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:36.792797  5129 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.984s	user 0.008s	sys 0.003s
I20260709 13:02:36.811892  5127 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:36.815351  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2397) took 1167 ms. Trace:
I20260709 13:02:36.794889  5126 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:36.817906  5131 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:36.819036  4797 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:02:36.820026  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:36.820845  5135 consensus_meta.cc:287] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.106s	user 0.004s	sys 0.008s
I20260709 13:02:36.821270  5135 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:36.821516  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:02:36.822141  5176 log.cc:927] Time spent T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:02:36.812147  5071 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:36.791796  5189 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:36.823923  5189 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:36.822371  1710 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:36.824329  5189 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:36.829286  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2046) took 1887 ms. Trace:
I20260709 13:02:36.832185  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:36.833043  1636 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 6: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:36.786193  4939 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:36.833914  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:36.794466  5052 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:36.834488  1637 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:36.815831  1642 rpcz_store.cc:276] 0709 13:02:35.647655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.647851 (+   196us) service_pool.cc:224] Handling call
0709 13:02:35.648236 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.888802 (+240566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.815117 (+926315us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b5400154854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:36.815288 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":528211200}
I20260709 13:02:36.838364  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:36.839221  1640 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:36.841190  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:36.829748  1560 rpcz_store.cc:276] 0709 13:02:34.942331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.947237 (+  4906us) service_pool.cc:224] Handling call
0709 13:02:34.947637 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:34.947992 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:35.769557 (+821565us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b1800381280. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb1455 00007fd7a56993c7 00007fd7a1cab9a7 00007fd7a1cbb0bf 00007fd7a1cbb038
0709 13:02:36.156467 (+386910us) log.cc:844] Serialized 14 byte log entry
0709 13:02:36.479662 (+323195us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:36.495014 (+ 15352us) log.cc:844] Serialized 26 byte log entry
0709 13:02:36.495202 (+   188us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:36.495250 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.495343 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:36.828879 (+333536us) raft_consensus.cc:1685] finished
0709 13:02:36.829079 (+   200us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.829223 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12429,"prepare.queue_time_us":874814,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":3606,"raft.queue_time_us":30567,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":31634,"spinlock_wait_cycles":24136960,"thread_start_us":636345,"threads_started":3,"wal-append.queue_time_us":346565}
I20260709 13:02:36.844635  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:36.845633  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 678) took 3091 ms. Trace:
I20260709 13:02:36.846138  1717 rpcz_store.cc:276] 0709 13:02:33.749395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.292736 (+543341us) service_pool.cc:224] Handling call
0709 13:02:35.180162 (+887426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:35.224914 (+ 44752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.840329 (+1615415us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b5400339fd0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:36.840480 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1980751232}
I20260709 13:02:36.848272  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:36.825843  5181 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:36.856014  1532 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:36.857208  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 685) took 2529 ms. Trace:
I20260709 13:02:36.789418  5021 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.858997  1711 rpcz_store.cc:276] 0709 13:02:34.327359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.469178 (+141819us) service_pool.cc:224] Handling call
0709 13:02:34.570239 (+101061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.675187 (+104948us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:34.675525 (+   338us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.979198 (+303673us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.979273 (+    75us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:34.979334 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.856966 (+1877632us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b540020ba54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:36.857019 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.857145 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21920,"prepare.queue_time_us":9580,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":23943,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":13427,"spinlock_wait_cycles":240823040,"thread_start_us":280950,"threads_started":1,"wal-append.queue_time_us":845799,"wal-append.run_cpu_time_us":1635,"wal-append.run_wall_time_us":1042595}
I20260709 13:02:36.860970  1532 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:36.844264  5199 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.861357  1641 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:02:36.867220  4491 consensus_meta.cc:287] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.629s	user 0.012s	sys 0.000s
I20260709 13:02:36.862576  4974 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:36.875756  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:36.891358  1717 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 9: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:36.840818  4939 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:36.894155  1534 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:36.906683  1607 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:36.908136  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:36.908793  1569 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:36.908658  1607 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:36.909395  1563 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:02:36.908720  1560 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:02:36.911034  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:36.811132  5188 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.924790  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:36.925590  1563 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:36.909997  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:36.924244  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:36.912534  2268 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:36.915115  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:36.923341  5127 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:36.930686  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:36.927088  1560 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 9: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:36.903273  4886 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:36.876762  5199 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:36.932044  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1912) took 1568 ms. Trace:
I20260709 13:02:36.937848  5202 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:36.939288  5199 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:36.939417  4403 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:36.940404  4403 raft_consensus.cc:3055] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:36.933811  1643 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:36.944553  4403 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:36.944751  5155 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.838590  5129 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:02:36.934935  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1902) took 2227 ms (client timeout 1593 ms). Trace:
W20260709 13:02:36.946437  1709 rpcz_store.cc:269] 0709 13:02:34.708016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.731064 (+ 23048us) service_pool.cc:224] Handling call
0709 13:02:36.934713 (+2203649us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54001365d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:36.934849 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1265511296}
I20260709 13:02:36.925177  1534 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:36.947675  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:36.920087  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.937872  1713 rpcz_store.cc:276] 0709 13:02:35.363936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.591013 (+227077us) service_pool.cc:224] Handling call
0709 13:02:35.591348 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.751003 (+159655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:35.751050 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:35.751108 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:35.751149 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.931804 (+1180655us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54001487d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:36.931856 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:36.931984 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":366522112}
I20260709 13:02:36.873608  4610 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:36.942152  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:36.942874  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:36.950064  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.931610  5166 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:36.931428  5175 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:36.930738  4733 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:36.963645  1565 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:02:36.965006  1532 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.593s (SENT) [suppressed 6 similar messages]
I20260709 13:02:36.945209  5155 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:36.967756  1710 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:36.966459  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.999095  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:37.015538  4733 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:37.018369  4733 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.019500  5129 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:37.020378  5136 consensus_queue.cc:1059] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:37.021245  5069 raft_consensus.cc:3055] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:37.021291  5187 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:37.021740  5069 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:37.022655  5069 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:37.023536  5051 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.001s
I20260709 13:02:37.024259  5051 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.955147  4886 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:37.023649  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.967304  5201 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:37.029515  5039 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.030224  5039 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:37.030534  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:36.965827  5175 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.031744  5175 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.032521  5039 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:36.943358  5202 raft_consensus.cc:697] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:37.027838  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:37.037959  5188 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:37.015774  4546 consensus_meta.cc:287] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.068s	user 0.012s	sys 0.001s
I20260709 13:02:37.046571  1640 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:36.946322  5195 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:37.049695  1637 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:02:37.023705  5069 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:37.029418  5201 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.056294  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:37.028518  4886 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.028705  5021 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.029609  4831 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:37.060639  4886 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.061053  4831 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.043900  5127 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:37.058336  5201 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:36.945804  4403 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:37.059978  5208 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:02:37.064975  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:37.065748  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:37.068351  1642 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:37.063252  4831 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.065925  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:37.034024  5202 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:36.967545  5166 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.216900  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1011) took 1875 ms. Trace:
W20260709 13:02:37.142940  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 177944us
I20260709 13:02:37.225662  2303 rpcz_store.cc:276] 0709 13:02:35.341285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.341431 (+   146us) service_pool.cc:224] Handling call
0709 13:02:35.341789 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:35.638763 (+296974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:35.638800 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:35.638850 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:35.638880 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.216564 (+1577684us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b54004e5f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:37.216629 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.216813 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":682438016}
I20260709 13:02:37.176512  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:37.069759  1640 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:02:37.193223  5051 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:37.098203  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.330157  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:37.249270  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.336546  1644 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:37.336716  1568 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.225360  1636 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 5.
W20260709 13:02:37.298956  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 734173us
W20260709 13:02:37.299785  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 584131us
I20260709 13:02:37.254222  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:37.058729  5195 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.361214  1532 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:37.339375  1638 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:02:37.251464  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 186941us
I20260709 13:02:37.350901  1710 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:02:37.340253  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:37.369285  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:37.370126  1711 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:37.369995  1713 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:37.346985  1681 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:37.329679  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 220790us
W20260709 13:02:37.276541  5140 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 1.652s	user 0.001s	sys 0.005s
I20260709 13:02:37.372170  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:37.370493  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:37.413885  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.422628  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.226130  4403 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:37.089485  5187 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:37.383208  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:37.383761  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:37.444548  1711 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:37.338789  5039 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:37.227887  5153 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:37.385008  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:37.244184  5144 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:37.394171  1532 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:37.463088  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.254899  4546 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:37.369038  5199 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:37.381982  1609 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:37.322412  5129 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:37.222633  5192 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.109824  4831 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.443202  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 99699.6us
I20260709 13:02:37.317797  4886 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.509483  1681 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:37.446405  1715 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:37.364148  4785 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.436946  5126 consensus_meta.cc:287] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.618s	user 0.011s	sys 0.000s
I20260709 13:02:37.512639  5104 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:37.512542  4785 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.512964  5126 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:37.513690  4831 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:37.515064  4785 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.496135  5129 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.515316  5126 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:37.063220  5127 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:37.516409  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2390) took 2141 ms (client timeout 1649 ms). Trace:
I20260709 13:02:37.516870  1681 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:37.517072  5129 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.517341  5127 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.372280  5131 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:37.517793  5170 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:37.518091  5131 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.518496  5170 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:37.372824  5051 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.480890  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.478236  4689 consensus_queue.cc:1059] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:02:37.519203  5051 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.519928  4689 raft_consensus.cc:3055] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:37.491950  1568 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:37.299958  5161 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.159s
I20260709 13:02:37.521764  5051 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:37.448222  5189 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.622s	user 0.011s	sys 0.002s
I20260709 13:02:37.522480  5189 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.523644  5161 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:37.524224  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1911) took 2160 ms. Trace:
I20260709 13:02:37.524847  2302 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:37.524386  5175 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.525023  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1017) took 1163 ms. Trace:
I20260709 13:02:37.525457  2300 rpcz_store.cc:276] 0709 13:02:36.361702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.437896 (+ 76194us) service_pool.cc:224] Handling call
0709 13:02:36.481484 (+ 43588us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.539205 (+ 57721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:36.539235 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:36.539277 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:36.539311 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.524820 (+985509us) spinlock_profiling.cc:243] Waited 57.5 ms on lock 0x7b5400710f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:37.524868 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.524972 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132204800}
I20260709 13:02:37.515666  4785 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:37.299284  5004 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:37.527143  5181 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:37.527570  5146 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:37.528656  5210 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
W20260709 13:02:37.528964  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.259442  4939 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:37.511756  5001 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:37.495683  4546 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.511365  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:37.533878  4546 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.534348  5155 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:37.535930  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:37.536376  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2049) took 1641 ms. Trace:
I20260709 13:02:37.537158  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:37.537472  5209 log.cc:927] Time spent T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:02:37.537945  1565 raft_consensus.cc:2368] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11.
I20260709 13:02:37.540334  1563 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:37.459863  5015 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:37.527770  5212 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:02:37.541170  1637 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:37.541834  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:37.482322  5144 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:37.509621  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 115564us
I20260709 13:02:37.459355  5187 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.524590  1718 rpcz_store.cc:276] 0709 13:02:35.363686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.580042 (+216356us) service_pool.cc:224] Handling call
0709 13:02:35.580492 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.576496 (+996004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.523975 (+947479us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b54001351d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:37.524168 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2278392064}
I20260709 13:02:37.520287  4689 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:02:37.516808  1643 rpcz_store.cc:269] 0709 13:02:35.375394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.375896 (+   502us) service_pool.cc:224] Handling call
0709 13:02:37.516203 (+2140307us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540032c8d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:37.516312 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3352104320}
I20260709 13:02:37.369275  5195 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:37.536718  1714 rpcz_store.cc:276] 0709 13:02:35.894672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.894884 (+   212us) service_pool.cc:224] Handling call
0709 13:02:35.895365 (+   481us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:35.895759 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:35.895784 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:35.928932 (+ 33148us) log.cc:844] Serialized 52 byte log entry
0709 13:02:36.255320 (+326388us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:36.518719 (+263399us) log.cc:844] Serialized 14 byte log entry
0709 13:02:36.545014 (+ 26295us) log.cc:844] Serialized 14 byte log entry
0709 13:02:36.545258 (+   244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:36.545337 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.298252 (+752915us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b180036be00. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb2ec0a 00007fd7a826e219 00007fd7a5b7558f 00007fd79ebb4571 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd
0709 13:02:37.533849 (+235597us) spinlock_profiling.cc:243] Waited 32.3 ms on lock 0x7b180036be00. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:37.536181 (+  2332us) raft_consensus.cc:1685] finished
0709 13:02:37.536202 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.536326 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":749107,"prepare.queue_time_us":193698,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":113,"raft.queue_time_us":43844,"raft.run_cpu_time_us":461,"raft.run_wall_time_us":53766,"spinlock_wait_cycles":92929792,"thread_start_us":370557,"threads_started":2,"wal-append.queue_time_us":401107}
I20260709 13:02:37.562597  1532 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:37.585750  1681 leader_election.cc:400] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11.
I20260709 13:02:37.586131  1681 leader_election.cc:403] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:02:37.589274  1532 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:37.575474  4887 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.029s	sys 0.070s
W20260709 13:02:37.593484  4887 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.029s	sys 0.070s
I20260709 13:02:37.594436  5187 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:37.595185  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:37.595836  1715 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
W20260709 13:02:37.596516  5213 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:02:37.597469  5198 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:37.597860  5198 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:37.598203  5198 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.601078  4886 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.512686  5043 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:37.606240  5166 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.461948  5039 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.608961  5039 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.606731  5043 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:37.568531  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.575901  1560 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 9: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:37.583243  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:37.577126  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:37.553834  1713 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:37.614151  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:37.614929  1565 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:37.615626  1566 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:37.527638  5181 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:37.629976  1568 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:37.526718  5175 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.621685  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:37.626055  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.524510  5189 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:37.566895  5015 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.633504  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:37.635986  2269 connection.cc:256] RPC call timeout handler was delayed by -1.2454161790000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
W20260709 13:02:37.636660  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.637490  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:37.508876  5192 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.637322  5015 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:37.647356  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:37.556885  5071 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:02:37.612031  5107 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:37.595758  5040 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.616811  4960 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11.
I20260709 13:02:37.651415  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:37.632858  5181 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:37.631515  4886 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:02:37.675062  5129 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:37.675602  5129 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:37.676013  5129 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:37.655046  4689 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.661085  4831 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:37.661741  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:37.684572  1565 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:37.662348  5043 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:37.652491  5045 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:37.662986  1567 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:37.686095  5045 raft_consensus.cc:697] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:37.663820  5195 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:37.671002  5136 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:37.686302  5021 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.687438  5136 raft_consensus.cc:3055] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:37.687134  5045 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.687827  5136 raft_consensus.cc:740] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:37.688650  5136 consensus_queue.cc:260] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.657269  4937 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.690694  4937 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:37.695958  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 314062us
I20260709 13:02:37.714349  2303 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:37.724560  1609 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:37.701232  5090 consensus_meta.cc:287] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.987s	user 0.012s	sys 0.002s
I20260709 13:02:37.730201  1609 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:37.703194  4689 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.760422  2298 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:37.773074  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:37.687134  5021 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:37.755648  4939 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:37.760211  5175 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.765120  5155 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:37.817267  5131 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.817364  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:37.818374  1641 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:37.819341  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:37.820045  1639 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
W20260709 13:02:37.820407  5224 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:02:37.821123  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1918) took 1005 ms. Trace:
I20260709 13:02:37.821460  5166 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:37.822100  5166 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.822806  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:37.826339  1643 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 6: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:37.827632  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:37.828212  1643 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:37.821494  1712 rpcz_store.cc:276] 0709 13:02:36.816064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.823843 (+  7779us) service_pool.cc:224] Handling call
0709 13:02:36.824188 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:37.319184 (+494996us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:37.319215 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:37.322706 (+  3491us) log.cc:844] Serialized 26 byte log entry
0709 13:02:37.764496 (+441790us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:37.764546 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.764629 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:37.820925 (+ 56296us) raft_consensus.cc:1685] finished
0709 13:02:37.820943 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.821070 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":494484,"raft.queue_time_us":494663,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":3832,"thread_start_us":284646,"threads_started":1,"wal-append.queue_time_us":357359}
I20260709 13:02:37.831601  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:37.831717  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:37.832543  1643 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:37.823635  2298 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:37.784085  4939 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:37.834463  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:37.835005  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:37.835769  2302 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:37.832964  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:37.834093  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:37.839888  1609 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:37.842219  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:37.843007  1639 raft_consensus.cc:2393] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:37.837119  2305 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:37.835052  1560 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:37.847193  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:37.848579  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:37.848999  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 191230us
I20260709 13:02:37.849113  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:37.843721  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:37.838066  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:37.854909  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:37.855382  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:37.853837  1609 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:37.859365  1641 raft_consensus.cc:2368] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11.
I20260709 13:02:37.861946  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:37.860107  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:37.862998  1560 raft_consensus.cc:2393] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 9: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:37.825773  5195 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:37.793942  5175 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:37.865032  1639 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:37.845278  4939 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:37.853892  2269 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:37.868263  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 722) took 1949 ms. Trace:
I20260709 13:02:37.844796  5144 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:37.868714  1635 rpcz_store.cc:276] 0709 13:02:35.918968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.919090 (+   122us) service_pool.cc:224] Handling call
0709 13:02:35.919366 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:36.348593 (+429227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:36.348635 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:36.348695 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:36.348740 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.868039 (+1519299us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54002d8554. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:37.868088 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:37.868209 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":986675456}
I20260709 13:02:37.838804  5197 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.854694  5155 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:37.869438  2269 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:37.855067  5131 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:37.864909  5080 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:37.872713  5090 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:37.864408  4831 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:37.872830  4937 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:37.876024  5187 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:37.877820  1643 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:02:37.878671  2269 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:37.873411  5007 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:02:37.879493  5080 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:37.881776  5235 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:02:37.884296  5090 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:37.884900  5131 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.862792  5040 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:02:37.876991  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:37.864749  5146 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:37.899729  2269 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:37.875327  5155 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:37.868736  5195 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.892879  4831 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:37.900759  5052 consensus_queue.cc:1059] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:37.862382  5181 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:37.901413  4937 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:37.877203  5197 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.903679  5175 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:37.908874  5021 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:37.911456  5187 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.908326  5201 consensus_meta.cc:287] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.844s	user 0.012s	sys 0.001s
I20260709 13:02:37.912173  1566 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:02:37.913276  4505 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:37.927934  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2421) took 1444 ms. Trace:
W20260709 13:02:37.924538  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 68771.2us
I20260709 13:02:37.955327  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:37.956245  1682 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:37.957021  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:37.958001  1560 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:37.940078  4689 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:37.920778  5080 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:37.953716  5199 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:37.952724  4797 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:02:37.962996  5199 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.965396  5199 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.923352  5090 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.971299  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:37.946952  1533 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:37.972596  1533 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:37.973459  5131 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.974061  5131 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:37.974376  5131 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:37.932272  5155 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:37.976156  5181 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:37.976912  5181 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:37.977576  5181 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:37.978078  5181 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:37.977720  5015 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:37.932888  4886 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.926014  5192 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:37.939618  4831 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:37.954061  1567 rpcz_store.cc:276] 0709 13:02:36.483922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.484091 (+   169us) service_pool.cc:224] Handling call
0709 13:02:36.484781 (+   690us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.686108 (+1201327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:37.927766 (+241658us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400179dd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:37.927890 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2709022080}
I20260709 13:02:37.940675  5146 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:37.953205  5230 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:02:37.980232  5195 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:37.959561  4689 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:37.959805  5187 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.943169  5232 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:02:37.980265  1639 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:37.983264  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:38.007897  1641 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:37.983421  5144 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.984757  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:37.985284  4446 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.890812  5198 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:37.987118  1637 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:37.987171  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:38.017485  1638 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:37.987625  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:37.989554  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:37.988584  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:37.991379  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:37.992401  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:37.994205  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:38.019135  1640 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 5.
I20260709 13:02:37.974721  5131 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.949424  4669 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:37.944092  5126 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.981206  5001 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:37.961872  4997 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:38.005229  4689 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.027920  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:38.012729  5187 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:38.018963  1636 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:38.022119  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:38.028584  2304 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:38.022969  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:38.023447  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 95947.6us
I20260709 13:02:37.996456  4886 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:37.959409  5201 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.001123  5166 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.012600  5146 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.012493  5144 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.021775  5021 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:38.045742  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:38.021981  5189 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:38.001320  5129 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.056775  2300 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 9: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:38.024839  4446 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.016757  5195 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:02:38.027822  4997 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:38.063758  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:38.027462  4669 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.044677  5166 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.051276  4797 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:38.000079  5181 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:38.079939  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:38.083825  1636 raft_consensus.cc:2393] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:38.083709  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.084309  1717 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:38.079141  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:38.087292  1644 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:38.087397  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:38.088225  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:38.088950  1644 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
W20260709 13:02:38.089280  5246 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:02:38.086159  5090 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.089537  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:38.080514  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:38.081195  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:38.091486  1640 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:38.081441  1710 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:38.027616  5126 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:38.093897  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.094388  5126 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:38.094596  1718 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:38.060699  5199 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:38.095620  5199 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:38.096100  5199 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:38.080199  1561 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:38.082057  1638 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:38.092868  5040 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.104900  5040 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.107863  5198 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.113360  2269 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:38.114326  5069 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:38.114711  5069 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:38.114907  5129 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.115159  5069 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:38.116590  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.117233  1718 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:38.118417  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.119141  1718 raft_consensus.cc:2368] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10.
I20260709 13:02:38.120410  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:38.120998  1718 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.124806  2269 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:38.051626  5189 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:38.130789  2299 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:38.167940  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 145071us
I20260709 13:02:38.162524  1718 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:38.174858  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:38.175088  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 218117us
I20260709 13:02:38.175647  1718 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:38.175874  1682 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:38.176915  1682 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:38.072283  4997 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.123626  5069 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:38.135983  5235 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:02:38.179322  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.063699  4939 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.178447  1682 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:38.129201  5222 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:38.082970  4797 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.193895  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.136554  5189 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:38.190414  4887 consensus_queue.cc:1059] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:02:38.179147  5129 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:38.181324  5040 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.198171  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:38.199538  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.177798  5043 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.226688  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:38.227679  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:38.139966  5107 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.203661  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:38.176707  5144 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.164561  5192 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:38.186637  5071 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:38.205359  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.203239  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:38.250612  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 47484.3us
I20260709 13:02:38.206091  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:38.225072  4733 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.126298  5166 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:38.207321  5247 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:38.276091  4733 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.277319  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:38.277966  1560 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:38.278077  5166 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:38.210893  5155 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.196570  5222 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:38.279223  5222 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:38.253067  5071 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:38.280176  5071 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:38.280541  5155 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.217007  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.281900  1641 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:38.239449  1682 leader_election.cc:400] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11.
I20260709 13:02:38.283142  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:38.240319  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:38.244542  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.247221  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.248594  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:38.248369  1607 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:38.287338  1564 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 5.
I20260709 13:02:38.194790  4669 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:38.245138  5040 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:38.253816  5198 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:38.251675  1533 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:38.251753  5107 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.322897  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:38.323930  4689 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.324786  5107 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.326357  1711 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:38.328087  5157 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.246258  5131 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.309389  5198 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.331415  5131 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.331647  5198 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.245828  5192 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:38.259927  5069 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:38.261495  5080 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.263958  4831 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.232908  5001 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:38.265319  4446 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:38.224009  4546 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:02:38.267443  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:38.267570  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:38.349294  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.212112  5199 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.273224  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2058) took 1358 ms. Trace:
I20260709 13:02:38.350033  1709 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:38.350090  1561 rpcz_store.cc:276] 0709 13:02:36.914689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.914894 (+   205us) service_pool.cc:224] Handling call
0709 13:02:38.273061 (+1358167us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400216454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:38.273165 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2638579968}
I20260709 13:02:38.275246  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.351419  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:38.351987  1561 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:38.278355  4733 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.198582  5242 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.200362  4797 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:38.289119  1560 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:02:38.306288  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 58073.6us
I20260709 13:02:38.281965  5197 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:38.309816  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 58273.5us
I20260709 13:02:38.308813  5155 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:38.310760  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:38.319244  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:38.244246  5187 raft_consensus.cc:3055] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:38.256026  4939 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.332379  5192 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.319001  5222 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.336113  1566 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:38.336620  1532 leader_election.cc:400] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10.
I20260709 13:02:38.385676  1532 leader_election.cc:403] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:38.341013  1711 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
W20260709 13:02:38.342178  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 102955us
I20260709 13:02:38.345664  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:38.345875  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:38.349550  1638 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:38.230818  5043 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:38.356048  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:38.310581  4048 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:38.358554  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:38.357479  5040 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:02:38.364178  5242 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.365312  4689 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:38.368086  2303 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:38.367833  1718 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.368845  1642 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:38.367789  1534 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:38.347208  5146 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:38.372495  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.425140  1713 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:38.372787  1561 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:38.380666  1560 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:38.439718  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:38.381243  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:38.386540  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 50084.6us
I20260709 13:02:38.413312  4048 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:38.440951  4048 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.382575  2299 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:38.414891  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:38.448510  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 67410.4us
I20260709 13:02:38.412238  5181 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.415969  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:38.416860  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 58385.1us
I20260709 13:02:38.455823  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:38.456048  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:38.379765  4446 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:38.418081  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 62139us
W20260709 13:02:38.414445  5211 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 13:02:38.422811  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 55124.4us
I20260709 13:02:38.426738  4939 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:38.429733  5043 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:38.431180  5146 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:38.424674  5021 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:38.437171  1564 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:38.447962  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:38.451620  1532 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:38.450014  5155 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.452909  5219 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:38.456941  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:38.370282  5187 raft_consensus.cc:740] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:38.456525  1637 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:38.460314  1681 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:02:38.513660  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 53433.4us
I20260709 13:02:38.514398  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:38.460965  1638 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:38.377640  4204 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:02:38.516083  5004 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:38.422132  5126 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:38.516597  5004 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:38.516688  5126 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.517132  5126 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.517520  5004 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:38.463195  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:38.519690  1643 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: [6.7-7.8]
W20260709 13:02:38.519835  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 56908.8us
I20260709 13:02:38.520792  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:38.521585  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:38.453254  1714 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:38.416190  5107 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:02:38.486032  1718 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:38.489614  4048 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:38.492364  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:38.528146  1716 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:38.491415  4939 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.450225  5225 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.505028  5146 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:38.506666  4937 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:38.507301  5021 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:38.450191  5124 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:38.483573  4446 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.509279  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:38.524044  5219 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:38.511835  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:38.564859  1564 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 9: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:38.415308  1682 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:38.565670  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 150463us
I20260709 13:02:38.533553  1533 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:38.535431  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:38.537580  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 88089.9us
I20260709 13:02:38.551828  4048 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:38.452741  5232 consensus_queue.cc:1059] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:02:38.570114  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.571508  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.572780  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.574222  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.576761  1638 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:38.562016  1560 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:38.511142  1635 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:38.529377  4204 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:02:38.530444  5043 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:38.568320  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:38.473801  5181 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.530148  4403 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:38.554776  4939 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:38.581719  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 60203.1us
W20260709 13:02:38.559027  5227 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:02:38.590291  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:38.581991  2269 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:38.591539  1715 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:38.583058  1713 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:38.584919  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.570396  5040 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:38.592455  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:38.593297  1715 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 9: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:38.590390  1710 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:02:38.530341  4733 consensus_queue.cc:1059] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:02:38.528429  5155 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.561601  5080 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.559471  4937 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:38.579005  5219 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.563426  5146 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:02:38.587901  4403 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.592550  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.598011  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.608026  1718 raft_consensus.cc:2393] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 10: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:38.595052  5092 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:38.589603  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:38.594734  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:38.570267  5021 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:02:38.602851  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 151316us
I20260709 13:02:38.555233  5198 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:38.559770  5225 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:38.598133  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:38.677919  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 88436.1us
I20260709 13:02:38.606093  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.578295  5124 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.591300  4204 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:38.640959  5092 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.641189  5043 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.641860  1710 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:38.643973  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.720497  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.721776  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.722852  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.647258  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.640806  1636 raft_consensus.cc:2410] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:38.648754  5090 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:38.661526  2297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1014) took 2741 ms (client timeout 1653 ms). Trace:
W20260709 13:02:38.730923  2297 rpcz_store.cc:269] 0709 13:02:35.919981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.920131 (+   150us) service_pool.cc:224] Handling call
0709 13:02:38.661227 (+2741096us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b5400686ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:38.661401 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5181301632}
I20260709 13:02:38.731841  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:38.733217  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:38.663568  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.582847  5237 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:02:38.677246  1533 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:38.680904  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2063) took 1680 ms. Trace:
I20260709 13:02:38.734706  1711 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:38.693535  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:38.693883  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:38.594427  4505 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:38.710068  1532 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:38.610280  4403 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:38.606683  5146 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:38.738867  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 61762.4us
I20260709 13:02:38.609269  4831 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.734859  1563 rpcz_store.cc:276] 0709 13:02:37.000622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.002173 (+  1551us) service_pool.cc:224] Handling call
0709 13:02:37.002576 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:37.540741 (+538165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.678837 (+1138096us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b54002aab54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:38.680815 (+  1978us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1236327168}
I20260709 13:02:38.741134  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:38.745857  2296 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:38.742206  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 48755.5us
I20260709 13:02:38.747046  5202 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:38.747722  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:38.606057  5199 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.743670  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:38.678138  5261 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:38.694852  5197 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:38.729992  5092 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:38.783519  2268 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:38.743748  1532 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:38.783036  1635 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
W20260709 13:02:38.803303  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 59721us
I20260709 13:02:38.787673  1609 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:38.677294  5052 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:02:38.807658  1609 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:38.783064  5071 consensus_meta.cc:287] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.000s
I20260709 13:02:38.742954  4505 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.812193  5242 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:38.737771  5222 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:38.842885  5040 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.748631  5232 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:38.800602  4689 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:38.782423  5161 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:38.799860  5197 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.813199  5267 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:38.839272  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 96117us
W20260709 13:02:38.819113  5255 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.347s	user 0.000s	sys 0.001s
I20260709 13:02:38.736472  5043 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:38.828079  5258 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:02:38.744380  4937 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.814087  4887 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:38.832070  4204 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:38.829466  5256 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
W20260709 13:02:38.852021  5189 consensus_meta.cc:287] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.628s	user 0.004s	sys 0.008s
I20260709 13:02:38.814596  5259 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:38.873790  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 71510.8us
I20260709 13:02:38.824954  5198 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:38.746210  5146 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:38.747921  5202 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:38.877933  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 289533us
I20260709 13:02:38.834486  5262 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:38.843176  5051 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:02:38.827522  5021 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:38.799166  5260 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:38.974697  4997 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.795s	user 0.011s	sys 0.000s
I20260709 13:02:38.880615  4505 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:38.891412  5161 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.886725  5197 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:38.891641  5267 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.826365  5268 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:38.905298  1564 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:38.874256  4886 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:38.877658  5261 raft_consensus.cc:697] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:38.887833  5232 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.941172  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:38.996234  1565 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:38.883937  5071 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:38.891222  4689 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.826297  5225 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:38.913954  5196 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:39.004802  5196 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.962313  5181 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.967461  5124 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.968700  5202 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:38.972750  5043 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:38.973536  5219 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:38.957625  4446 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:38.957284  5195 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.937425  5280 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:38.976658  5278 consensus_queue.cc:1059] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:38.977473  5269 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:38.977741  4887 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:38.889181  5040 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:38.979007  5281 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.978997  5155 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:39.003481  5268 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:38.888204  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 80660.8us
W20260709 13:02:39.010143  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 69048.6us
I20260709 13:02:38.964746  5259 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.035138  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:38.976215  5126 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:39.011729  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 53705.9us
I20260709 13:02:38.995285  5260 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.041059  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:38.947161  5189 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.017742  5015 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:39.020350  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2402) took 1692 ms. Trace:
I20260709 13:02:39.045979  1642 rpcz_store.cc:276] 0709 13:02:37.327682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.328031 (+   349us) service_pool.cc:224] Handling call
0709 13:02:37.985919 (+657888us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.422427 (+436508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.019458 (+597031us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b54004628d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:39.020141 (+   683us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54004628d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:39.020286 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2392328320}
I20260709 13:02:39.024031  4689 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.003609  4997 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:39.004097  5232 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:39.049896  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2424) took 1519 ms (client timeout 1644 ms). Trace:
I20260709 13:02:39.038803  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:39.040028  1609 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:39.052969  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.028450  5192 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.003s
I20260709 13:02:39.014381  5052 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:39.054221  2296 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:39.043745  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 238666us
I20260709 13:02:39.045076  5281 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:39.057684  5040 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:39.058521  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:39.059492  1562 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 13:02:39.050520  1568 rpcz_store.cc:269] 0709 13:02:37.529970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.565594 (+ 35624us) service_pool.cc:224] Handling call
0709 13:02:39.049635 (+1484041us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b54002c5954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:39.049807 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41795328}
I20260709 13:02:39.059522  2297 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:39.055794  1532 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:39.068168  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:39.070055  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:39.071679  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:39.044895  5222 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:39.032045  5195 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.073503  5222 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.075780  5195 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:39.060276  5155 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:02:39.077667  5189 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.081423  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:39.062000  5192 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.082295  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:39.083050  1709 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:39.083658  5192 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:39.083074  1565 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:39.084157  5219 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.085129  4403 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:39.085601  5107 consensus_meta.cc:287] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.555s	user 0.009s	sys 0.001s
I20260709 13:02:39.086043  5107 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10.
I20260709 13:02:39.086102  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:39.087713  1711 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:02:39.087086  1561 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:39.087939  2302 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:39.091845  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:39.088341  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:39.096771  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:39.099092  1534 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:39.099113  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:39.100059  1715 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:39.061444  1609 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:39.086759  1569 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:39.042239  5278 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:39.112959  1713 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:39.113184  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:39.039120  5268 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:39.101514  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:39.115444  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:39.109481  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 48113.3us
I20260709 13:02:39.123900  1568 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:39.117340  4505 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:39.047585  5260 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:39.086799  5107 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:39.128674  1563 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:39.145668  1639 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:39.153978  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:39.154518  2298 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1018) took 2791 ms (client timeout 2776 ms). Trace:
I20260709 13:02:39.130574  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:39.158581  2298 rpcz_store.cc:269] 0709 13:02:36.363491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:36.391046 (+ 27555us) service_pool.cc:224] Handling call
0709 13:02:39.154330 (+2763284us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400708354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:39.154448 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3102355200}
I20260709 13:02:39.159566  2268 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:39.138502  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:39.160014  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:39.156376  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:39.161928  2303 raft_consensus.cc:2410] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:02:39.110414  4797 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.731s	user 0.011s	sys 0.000s
I20260709 13:02:39.171831  4997 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:39.171782  4797 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:39.038838  4887 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.114495  5181 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:39.129812  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:39.164984  1638 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:02:39.188814  5181 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:39.164556  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:39.189293  5181 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:39.190054  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:39.095448  5269 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:39.191099  5269 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:39.191247  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:39.191526  5269 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:39.167358  5071 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:39.169734  5146 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:39.192786  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:39.040941  5259 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:39.193645  2303 raft_consensus.cc:2410] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:02:39.114586  5287 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:02:39.089501  5092 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.073515  5136 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:39.075773  5222 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:39.173317  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:39.174010  4505 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.135946  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:39.183611  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:39.200212  1568 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
W20260709 13:02:39.184222  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 93568.4us
I20260709 13:02:39.037787  5285 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 13:02:39.103070  5043 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:39.196687  4939 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.000s
I20260709 13:02:39.191992  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:39.192353  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:39.164563  5280 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:39.101056  5272 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:39.154217  5189 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:39.173512  5219 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:39.199690  1639 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:39.201197  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:39.201491  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 197376us
I20260709 13:02:39.202492  1532 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:39.287732  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 86544.9us
I20260709 13:02:39.288676  1532 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:39.062774  5129 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.113s
I20260709 13:02:39.262838  1637 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 6: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:02:39.263528  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 71592.9us
I20260709 13:02:39.264228  2304 raft_consensus.cc:2410] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:02:39.180691  5052 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:02:39.271713  5267 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:39.110631  5225 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:02:39.172693  5107 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.268201  5043 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:39.286515  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:39.114039  5261 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.311292  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:39.311980  5283 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.292361  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:39.292898  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 226953us
I20260709 13:02:39.313370  1640 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:39.314298  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:39.314625  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:39.283650  5280 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:39.315938  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:39.200872  5202 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.086s
W20260709 13:02:39.295864  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.295933  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:39.198120  4446 consensus_queue.cc:1059] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:02:39.275350  4689 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:39.296489  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 172013us
I20260709 13:02:39.296679  5195 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:39.301440  2303 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:39.304687  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:39.312024  1568 raft_consensus.cc:2368] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10.
I20260709 13:02:39.294763  5281 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.299326  4544 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.300284  5198 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:39.269044  4939 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:39.322338  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:39.291376  5189 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.297256  5232 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.455710  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:39.475392  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:39.378266  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:39.402213  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 106378us
I20260709 13:02:39.312587  5283 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.483991  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:39.474750  1639 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:39.470831  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:39.490048  5181 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.411780  5281 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:39.402045  5288 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 13:02:39.454124  5286 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:39.456562  5157 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:39.491529  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:39.404254  5004 consensus_queue.cc:1059] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 13:02:39.397117  5040 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.320989  4689 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.495705  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:39.517204  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:39.496199  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 734) took 1410 ms. Trace:
I20260709 13:02:39.517903  1641 rpcz_store.cc:276] 0709 13:02:38.086041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.087089 (+  1048us) service_pool.cc:224] Handling call
0709 13:02:39.496039 (+1408950us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b5400247f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:39.496142 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":445606656}
I20260709 13:02:39.499341  5092 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.497819  2304 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:39.384177  5273 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:39.482228  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 159968us
I20260709 13:02:39.500844  1638 raft_consensus.cc:2368] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10.
I20260709 13:02:39.375710  4886 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:39.505033  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:39.525566  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:39.480710  4797 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.530292  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:39.506551  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 192023us
I20260709 13:02:39.507357  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:39.507916  2303 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:39.466184  5259 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.396241  5195 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:39.430562  5129 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:39.487887  5198 raft_consensus.cc:697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:39.512569  2301 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:39.487504  5043 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.514892  5040 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:39.515477  5291 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:02:39.515259  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:39.496136  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 174278us
I20260709 13:02:39.546185  1641 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:39.481278  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:39.509009  4887 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.526688  5155 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:39.423043  5146 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.510303  5286 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:39.487708  4544 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.534168  1681 leader_election.cc:400] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10.
I20260709 13:02:39.584342  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:39.584657  1681 leader_election.cc:403] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:02:39.585124  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 51138.5us
I20260709 13:02:39.585430  5201 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:39.585655  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:39.586258  1641 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:39.587005  5161 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:39.587615  5161 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:39.587714  1639 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:39.520479  5281 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:39.519920  4689 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:39.401154  5243 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.116s
I20260709 13:02:39.541328  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:39.547401  1640 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:39.528839  5287 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:02:39.542356  5294 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:39.540886  5288 raft_consensus.cc:3055] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:39.551587  5274 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:39.551964  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:39.621873  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 69990.8us
I20260709 13:02:39.527025  4886 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:39.557502  5232 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.558808  4939 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.559758  5181 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:39.561151  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:39.627755  1569 raft_consensus.cc:2393] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:39.628700  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:39.563192  5269 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:39.455693  5260 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.527225  5273 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.583310  2303 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:39.472226  5280 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.537151  5129 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:39.593667  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:39.622771  1682 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:39.623426  1568 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
W20260709 13:02:39.577282  5197 consensus_meta.cc:287] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.000s
I20260709 13:02:39.623673  5277 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.624202  5272 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.666610  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:02:39.631177  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 89951.1us
I20260709 13:02:39.667917  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:39.596619  5267 raft_consensus.cc:3055] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:39.586511  4544 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:39.597157  5192 raft_consensus.cc:3055] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:39.589855  5268 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:39.667124  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:39.668581  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:39.669433  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:39.592794  5198 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:39.624615  5288 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:39.672288  1718 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:39.630587  5004 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:39.673230  1716 raft_consensus.cc:2393] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 10: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:39.680526  1714 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:39.689538  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:39.666342  5131 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:39.673491  2297 raft_consensus.cc:2368] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7.
I20260709 13:02:39.674117  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:39.598855  5283 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.675117  1639 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:39.633018  4887 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:02:39.661737  5129 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:39.676383  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:39.706568  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.661252  5274 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.738152  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:39.670279  4939 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:39.724109  1640 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:39.665434  5197 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.747635  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:39.748378  1640 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:39.749315  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:39.750276  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:39.671411  5267 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:39.724256  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:39.734896  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:39.756532  1714 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 9: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:02:39.735738  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 46328.2us
I20260709 13:02:39.743085  1564 raft_consensus.cc:2368] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10.
I20260709 13:02:39.727165  5004 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.744338  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:39.641824  5287 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:39.751910  2269 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:39.752331  1640 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:39.716831  4886 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.739663  5274 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:39.722843  4544 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:39.673036  5192 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:39.770802  5192 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.772022  5192 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:39.676647  5301 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:39.756165  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:39.776796  1716 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:39.760104  2303 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 8. Ops: []
I20260709 13:02:39.741833  5161 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10.
I20260709 13:02:39.768460  2269 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:39.768745  1639 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:39.736389  5189 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.739804  5225 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:39.782684  5301 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:39.768333  5004 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:39.767685  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:39.781848  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:39.784333  2269 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:39.784135  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:39.797129  1640 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:39.797334  1639 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:39.797510  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:39.802402  5269 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.805131  1640 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:39.805286  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:39.676457  5195 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.786494  5283 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:39.775280  5302 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:39.786320  5294 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.785285  5267 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:39.807335  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:39.673264  5272 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:39.673772  5277 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.822566  5266 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:39.674216  5286 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.793664  4886 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:39.823625  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:39.805140  5281 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:39.828634  1639 raft_consensus.cc:2410] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:02:39.820173  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:39.807665  5225 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:39.837118  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:39.838387  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:39.839860  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:39.816460  5302 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:39.840647  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.839998  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:39.843358  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:39.817404  5201 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:39.848250  1718 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:39.824038  5155 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:39.850261  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:39.853036  1716 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:39.825461  5131 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:39.852152  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:39.855053  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:39.835278  5040 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:39.819386  5278 consensus_queue.cc:1059] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:02:39.842729  4939 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:39.845845  5267 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:39.845336  5286 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:39.850811  5302 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:39.858685  5131 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:39.868628  5304 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:02:39.915011  1569 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:39.866978  5198 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:39.825538  5294 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.841013  4886 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.853988  5281 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:39.856261  5269 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:39.952474  5308 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 13:02:39.956399  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.857765  5201 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:39.856678  5280 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.859658  5266 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:39.873129  5260 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:39.966010  5310 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:02:39.948038  5197 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.966508  4886 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:39.875739  4939 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.919858  5278 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:39.932482  5198 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.808525  5288 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:39.965626  4689 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.938539  5131 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:39.955509  5268 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:39.968743  5273 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:39.979151  2304 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:39.952262  5269 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.974710  4939 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:39.971403  5266 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.968895  5301 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:39.992062  5277 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:39.974625  4544 consensus_queue.cc:1059] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:02:39.977412  5278 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:39.988507  5269 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:39.985586  4689 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:39.989881  5288 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:39.988408  5266 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:39.975219  5280 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:40.005255  5201 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:40.019395  5311 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:02:40.026266  5198 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:40.029604  4831 raft_consensus.cc:3055] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:40.035291  4831 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:40.051720  5278 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:40.081131  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 82919.3us
I20260709 13:02:40.089191  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:39.987483  1567 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:40.116637  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2011) took 1660 ms. Trace:
I20260709 13:02:40.117283  1636 rpcz_store.cc:276] 0709 13:02:38.456003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.456610 (+   607us) service_pool.cc:224] Handling call
0709 13:02:40.116025 (+1659415us) spinlock_profiling.cc:243] Waited 98.6 ms on lock 0x7b54003efb54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:40.116368 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226763008}
I20260709 13:02:40.120016  1636 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:40.123813  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.127980  5104 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.130432  1456 maintenance_manager.cc:419] P 6ffeec09a12e4bd6997354c8ecdff02d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033393
I20260709 13:02:40.130493  4960 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:40.131328  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2077) took 1063 ms. Trace:
I20260709 13:02:40.131995  4960 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.129360  5104 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.135571  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:40.136986  1565 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:40.131820  1564 rpcz_store.cc:276] 0709 13:02:39.067804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.067939 (+   135us) service_pool.cc:224] Handling call
0709 13:02:40.130790 (+1062851us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54001ce654. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:40.131111 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1550204288}
I20260709 13:02:40.139488  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:40.049458  5274 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:40.124193  5004 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:40.142613  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:40.157845  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2013) took 1624 ms. Trace:
I20260709 13:02:40.192220  4960 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:40.192616  5052 raft_consensus.cc:3055] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:40.193526  5052 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:40.156018  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2008) took 1700 ms. Trace:
I20260709 13:02:40.137025  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1033) took 1068 ms. Trace:
I20260709 13:02:40.201375  1710 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:40.147957  5249 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.202442  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1027) took 1135 ms. Trace:
I20260709 13:02:40.202355  5249 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.203018  2297 rpcz_store.cc:276] 0709 13:02:39.066416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.071453 (+  5037us) service_pool.cc:224] Handling call
0709 13:02:40.201882 (+1130429us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b5400642154. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:40.202236 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376637312}
W20260709 13:02:40.207070  1713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2065) took 1562 ms (client timeout 1680 ms). Trace:
I20260709 13:02:40.207808  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:40.210098  5292 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:40.189221  1637 rpcz_store.cc:276] 0709 13:02:38.533701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.535131 (+  1430us) service_pool.cc:224] Handling call
0709 13:02:40.157216 (+1622085us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b5400195ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:40.157651 (+   435us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1598985088}
W20260709 13:02:40.041086  5136 consensus_meta.cc:287] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.836s	user 0.013s	sys 0.000s
I20260709 13:02:40.197309  2298 rpcz_store.cc:276] 0709 13:02:39.068063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.203884 (+135821us) service_pool.cc:224] Handling call
0709 13:02:39.258451 (+ 54567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.501873 (+243422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.501921 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.501984 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:39.502015 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.136237 (+634222us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54003a2354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:40.136321 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.136796 (+   475us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559250176}
I20260709 13:02:40.210443  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:40.223102  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 62777.3us
I20260709 13:02:40.196945  1643 rpcz_store.cc:276] 0709 13:02:38.455070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.457268 (+  2198us) service_pool.cc:224] Handling call
0709 13:02:38.519430 (+ 62162us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:38.520097 (+   667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.155408 (+1635311us) spinlock_profiling.cc:243] Waited 61.7 ms on lock 0x7b5400155250. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:40.155781 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141910400}
I20260709 13:02:40.146692  5146 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:40.210821  5107 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:40.207698  1713 rpcz_store.cc:269] 0709 13:02:38.644503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.646790 (+  2287us) service_pool.cc:224] Handling call
0709 13:02:40.206470 (+1559680us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400268ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:40.206838 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":895325568}
I20260709 13:02:40.230899  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.232218  1713 raft_consensus.cc:2410] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:02:40.215801  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.218780  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.235941  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:40.051837  5302 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.237113  1714 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:40.238464  5092 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.233549  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:40.238269  5187 consensus_queue.cc:260] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.240579  5092 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.242650  2300 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:40.143493  5129 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:40.245373  1718 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:40.247251  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.249171  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.251169  5175 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.252589  5196 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:40.252332  5175 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.215144  5280 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:40.219648  5052 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.256875  5175 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:40.258158  5052 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:40.259379  1644 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:40.259968  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 997) took 1206 ms. Trace:
I20260709 13:02:40.260394  2302 rpcz_store.cc:276] 0709 13:02:39.053269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.053543 (+   274us) service_pool.cc:224] Handling call
0709 13:02:39.059946 (+  6403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.092598 (+ 32652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.259474 (+1166876us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b5400583ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:40.259766 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63957888}
W20260709 13:02:40.261319  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.261914  1709 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:40.263211  5007 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.263496  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:40.264081  5007 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.265111  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:40.265458  4960 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.266407  4960 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:40.271142  5107 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.259326  5288 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.272856  1709 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:40.266598  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:40.238273  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:40.274278  1718 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:40.242578  5187 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:40.270939  5299 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:02:40.276593  4048 consensus_queue.cc:1059] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:40.277571  4048 consensus_queue.cc:1243] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:40.278173  4048 raft_consensus.cc:3055] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:40.278566  4048 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:40.280077  4048 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.281870  4048 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:40.283260  5131 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.142439  4831 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:40.287573  4831 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:40.288363  5104 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:40.287915  5107 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:40.284108  5131 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.289907  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2442) took 1384 ms. Trace:
I20260709 13:02:40.291589  1564 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 10: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:40.292332  5288 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:40.293929  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 995) took 1254 ms. Trace:
I20260709 13:02:40.296145  4403 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:40.297331  1713 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:40.298861  5302 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:40.299608  1714 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:40.312013  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.302383  1560 rpcz_store.cc:276] 0709 13:02:38.904893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.905688 (+   795us) service_pool.cc:224] Handling call
0709 13:02:38.906030 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.959254 (+ 53224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:38.959304 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:38.959362 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:38.959410 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.289365 (+1329955us) spinlock_profiling.cc:243] Waited 52.9 ms on lock 0x7b5400277254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:40.289454 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.289703 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121718656}
I20260709 13:02:40.305286  2296 rpcz_store.cc:276] 0709 13:02:39.039862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.039978 (+   116us) service_pool.cc:224] Handling call
0709 13:02:39.041649 (+  1671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.054627 (+ 12978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.293212 (+1238585us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b54001aa754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:40.293701 (+   489us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28448512}
I20260709 13:02:40.148485  5198 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.160s
I20260709 13:02:40.316964  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:40.318145  1564 raft_consensus.cc:2410] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:40.320003  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:40.320830  1564 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:40.321532  5126 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:40.322014  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:40.211184  5292 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.333196  5292 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:40.324942  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2440) took 1422 ms. Trace:
I20260709 13:02:40.334158  1562 rpcz_store.cc:276] 0709 13:02:38.902798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.903678 (+   880us) service_pool.cc:224] Handling call
0709 13:02:38.904074 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.059945 (+155871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.324414 (+1264469us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400179dd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:40.324777 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":356923648}
I20260709 13:02:40.334892  5107 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:40.336895  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2081) took 1251 ms. Trace:
I20260709 13:02:40.337314  1562 rpcz_store.cc:276] 0709 13:02:39.085376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.335485 (+1250109us) service_pool.cc:224] Handling call
0709 13:02:40.336132 (+   647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.336412 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.336436 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:40.336494 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:40.336524 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.336575 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.336817 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:40.325907  5249 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:40.323632  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2443) took 1418 ms. Trace:
I20260709 13:02:40.339479  1566 rpcz_store.cc:276] 0709 13:02:38.905170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.905327 (+   157us) service_pool.cc:224] Handling call
0709 13:02:38.938029 (+ 32702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:38.983069 (+ 45040us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:38.983120 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:38.983185 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:38.983236 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.323124 (+1339888us) spinlock_profiling.cc:243] Waited 44.8 ms on lock 0x7b54000f6254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:40.323191 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.323442 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102995328}
I20260709 13:02:40.322885  1564 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:40.343700  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.345355  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2425) took 1047 ms. Trace:
I20260709 13:02:40.345875  1644 rpcz_store.cc:276] 0709 13:02:39.298104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.298262 (+   158us) service_pool.cc:224] Handling call
0709 13:02:40.158109 (+859847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.260087 (+101978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.260780 (+   693us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400195ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:40.345100 (+ 84320us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b5400195ad0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:40.345269 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2208315008}
I20260709 13:02:40.346441  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.349107  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2427) took 1043 ms. Trace:
I20260709 13:02:40.349606  1716 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:40.350498  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2417) took 1220 ms. Trace:
I20260709 13:02:40.349152  5198 raft_consensus.cc:3055] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:40.350901  1635 rpcz_store.cc:276] 0709 13:02:39.129913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.130312 (+   399us) service_pool.cc:224] Handling call
0709 13:02:40.350094 (+1219782us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b54001c7354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:40.350321 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1523158912}
I20260709 13:02:40.351663  5198 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:40.352500  1714 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:40.353224  5198 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:40.354029  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.354157  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:40.349651  1638 rpcz_store.cc:276] 0709 13:02:39.305046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.309024 (+  3978us) service_pool.cc:224] Handling call
0709 13:02:40.348646 (+1039622us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b540032c8d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:40.348866 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423286656}
I20260709 13:02:40.362339  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.363298  2302 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:40.363945  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.365484  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 620) took 2140 ms. Trace:
I20260709 13:02:40.365985  1567 rpcz_store.cc:276] 0709 13:02:38.225130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.227465 (+  2335us) service_pool.cc:224] Handling call
0709 13:02:40.365136 (+2137671us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54001cfa54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:40.365313 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4045916928}
I20260709 13:02:40.367197  2298 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:40.368559  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:40.369280  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:40.370335  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.371884  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:40.372493  5300 raft_consensus.cc:3055] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:40.373023  5300 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:40.375675  1717 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.376895  5288 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:40.383297  1562 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:40.380954  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:40.392650  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 631) took 1282 ms. Trace:
I20260709 13:02:40.393559  1562 rpcz_store.cc:276] 0709 13:02:39.110046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.338276 (+1228230us) service_pool.cc:224] Handling call
0709 13:02:40.338800 (+   524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.383740 (+ 44940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.392567 (+  8827us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:40.395474  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:40.399495  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:40.406111  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.408046  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:40.411162  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:40.380892  5272 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:40.416590  1567 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:40.374589  5300 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.380936  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1007) took 1082 ms. Trace:
I20260709 13:02:40.380978  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 639) took 1187 ms. Trace:
I20260709 13:02:40.423717  1560 rpcz_store.cc:276] 0709 13:02:39.191423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.378274 (+1186851us) service_pool.cc:224] Handling call
0709 13:02:40.378667 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.378968 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.379002 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:40.379061 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:40.379107 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.379167 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.379279 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:40.427114  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:40.431488  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 88760.5us
I20260709 13:02:40.422686  2303 rpcz_store.cc:276] 0709 13:02:39.298752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.670469 (+371717us) service_pool.cc:224] Handling call
0709 13:02:39.735434 (+ 64965us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.778002 (+ 42568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.380464 (+602462us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b54006077d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:40.380757 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56323840}
I20260709 13:02:40.387812  5198 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:40.433686  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:40.436224  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:40.437394  5092 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:40.438768  5080 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:02:40.444270  5004 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:40.446797  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:40.451582  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.447578  1715 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:40.458228  1717 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:40.462618  1568 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:40.466349  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:40.466218  5266 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:40.467378  1712 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
W20260709 13:02:40.468056  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.469125  1712 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:40.470315  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:40.470595  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.475903  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2439) took 1573 ms. Trace:
I20260709 13:02:40.476425  1563 rpcz_store.cc:276] 0709 13:02:38.902496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.902636 (+   140us) service_pool.cc:224] Handling call
0709 13:02:38.903011 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.157898 (+254887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.475343 (+1317445us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b54003e8354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:40.475678 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518389120}
I20260709 13:02:40.477867  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:40.478754  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:40.479853  1712 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:40.478674  1563 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:40.480893  5004 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:40.481899  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:40.483155  1635 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:40.483310  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:40.483902  5129 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:40.484335  1563 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:40.485422  1563 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:40.486203  4960 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:40.494895  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.495848  1714 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:40.496798  1607 leader_election.cc:400] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10.
I20260709 13:02:40.497143  1607 leader_election.cc:403] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:40.498487  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:40.490597  4939 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.500918  1563 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:02:40.472496  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 640) took 1159 ms. Trace:
I20260709 13:02:40.513361  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:40.514719  1568 rpcz_store.cc:276] 0709 13:02:39.306087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.391096 (+1085009us) service_pool.cc:224] Handling call
0709 13:02:40.458655 (+ 67559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.463073 (+  4418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.465020 (+  1947us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:40.517979  1638 raft_consensus.cc:2393] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:40.520231  1682 leader_election.cc:400] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10.
I20260709 13:02:40.520466  4939 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:40.521843  5269 consensus_meta.cc:287] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.507s	user 0.001s	sys 0.015s
I20260709 13:02:40.522446  5269 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.522724  5288 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:40.524241  5288 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:40.524693  5288 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:40.525898  5288 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:40.527328  1644 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 10: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:40.529390  5269 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:40.532379  5267 consensus_meta.cc:287] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.619s	user 0.005s	sys 0.018s
I20260709 13:02:40.534157  1560 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:40.534798  4403 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:40.535300  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:40.536180  1533 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:40.537145  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:40.537472  1565 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
W20260709 13:02:40.539114  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.540115  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:40.537137  5131 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:40.546612  5004 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:40.547060  5272 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:40.549850  2300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 839) took 2243 ms (client timeout 2769 ms). Trace:
I20260709 13:02:40.550362  1532 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:40.551133  4544 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:02:40.551630  1561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 627) took 2162 ms (client timeout 1666 ms). Trace:
I20260709 13:02:40.512801  1713 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:40.552891  2269 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:40.564590  1715 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
W20260709 13:02:40.565574  2300 rpcz_store.cc:269] 0709 13:02:38.306936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.307135 (+   199us) service_pool.cc:224] Handling call
0709 13:02:40.549591 (+2242456us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b5400367ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:40.549747 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4310474112}
I20260709 13:02:40.581336  1710 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:40.580420  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:40.587069  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:40.550654  5248 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
W20260709 13:02:40.566407  1561 rpcz_store.cc:269] 0709 13:02:38.389349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.439507 (+ 50158us) service_pool.cc:224] Handling call
0709 13:02:40.551201 (+2111694us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b5400325854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:40.551388 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1487414016}
I20260709 13:02:40.589526  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:40.571306  5080 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:40.596467  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:40.596783  5080 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.569414  4939 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:40.590669  4048 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:40.590317  5288 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:40.601325  1640 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:02:40.601588  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 49240.3us
W20260709 13:02:40.606619  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.607437  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 57246.5us
I20260709 13:02:40.603578  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:40.611304  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:40.608594  1637 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:40.594561  5131 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.620975  1563 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:40.623216  2268 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:40.627241  5069 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:40.627915  5069 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.628208  1565 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:40.628445  5069 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:40.629901  1565 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:40.630221  5045 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.630867  5045 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:40.632300  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 97261.8us
I20260709 13:02:40.632910  5272 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:40.632929  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 97138.6us
I20260709 13:02:40.639851  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:40.645169  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:40.680431  1713 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:40.632359  5241 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:40.631505  4048 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:40.682536  4048 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.683094  4048 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:40.678431  1716 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:40.679005  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:40.632032  5004 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:40.679807  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 58307us
I20260709 13:02:40.684566  1636 raft_consensus.cc:2368] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9.
W20260709 13:02:40.685487  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 53650.1us
I20260709 13:02:40.686390  1714 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:40.687608  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.696305  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:40.696296  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:40.701730  2296 raft_consensus.cc:2410] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
W20260709 13:02:40.704774  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.712144  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:40.715236  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:40.715971  1714 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:40.708853  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:40.717613  1713 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 10: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:40.721625  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.730691  1561 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:40.726996  1636 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:40.725312  1635 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 10: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:40.735705  1638 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:40.700188  5092 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:40.740304  1607 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:40.723327  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 77737.8us
I20260709 13:02:40.723080  1637 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:40.714416  1640 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:40.743708  1714 raft_consensus.cc:2410] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:02:40.685670  5269 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.748670  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:40.743911  1532 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:40.725879  5237 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.756178  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:40.759275  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 141984us
I20260709 13:02:40.749404  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:40.759996  1716 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:40.710489  5131 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:40.720275  5004 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.730299  5274 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:40.753770  5267 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:40.774008  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:40.797933  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:40.798943  1716 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:40.802498  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 90677us
W20260709 13:02:40.823848  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.825440  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 104718us
I20260709 13:02:40.811653  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:40.831559  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 628) took 1746 ms. Trace:
I20260709 13:02:41.018205  1714 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:02:40.744565  5241 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:40.764472  5269 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:40.864257  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.818315  5237 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:40.785408  5080 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:40.867697  5317 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:02:40.768858  5187 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.093s
W20260709 13:02:40.907325  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.927369  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.776440  5292 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 13:02:41.036007  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 205663us
I20260709 13:02:40.825363  5278 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:40.788175  5248 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:40.786760  5267 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:41.020458  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 251727us
I20260709 13:02:41.026052  1716 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:41.018627  1569 rpcz_store.cc:276] 0709 13:02:39.084833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.630483 (+545650us) service_pool.cc:224] Handling call
0709 13:02:39.634412 (+  3929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.929471 (+295059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.763997 (+834526us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b5400250154. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:40.831345 (+ 67348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644846336}
I20260709 13:02:41.028156  1640 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:02:41.040374  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 279876us
I20260709 13:02:41.062179  4048 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:41.063508  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:41.064509  1565 raft_consensus.cc:2368] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9.
I20260709 13:02:41.064604  4048 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:41.065706  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:41.066226  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.066329  1565 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:41.066872  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.067759  5272 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.068368  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:41.068578  5272 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:41.069813  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:41.070050  5272 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:41.070464  5272 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:41.070749  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:41.072005  5321 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.327s	user 0.000s	sys 0.009s
I20260709 13:02:41.073800  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 763) took 1267 ms. Trace:
I20260709 13:02:41.074293  1641 rpcz_store.cc:276] 0709 13:02:39.806016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.096692 (+290676us) service_pool.cc:224] Handling call
0709 13:02:40.195536 (+ 98844us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.202770 (+  7234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.202806 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.203508 (+   702us) log.cc:844] Serialized 26 byte log entry
0709 13:02:40.310388 (+106880us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:40.310445 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.310540 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.073404 (+762864us) raft_consensus.cc:1685] finished
0709 13:02:41.073429 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.073591 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":662,"raft.queue_time_us":9130,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":216,"thread_start_us":103568,"threads_started":1,"wal-append.queue_time_us":418776}
I20260709 13:02:41.027104  5052 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.076869  5052 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.077409  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2451) took 1538 ms. Trace:
I20260709 13:02:41.065725  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 634) took 1911 ms. Trace:
I20260709 13:02:41.079123  4403 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:41.079648  4403 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:41.080099  4403 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.055517  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:41.081075  1561 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:41.070464  1569 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:41.071873  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:41.080300  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.073437  5278 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.078259  1564 rpcz_store.cc:276] 0709 13:02:39.154272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.382665 (+1228393us) service_pool.cc:224] Handling call
0709 13:02:40.461781 (+ 79116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.487115 (+ 25334us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:40.681987 (+194872us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.065313 (+383326us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.065376 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:41.065426 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.065484 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.065585 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24469,"prepare.queue_time_us":288739,"prepare.run_cpu_time_us":8464,"prepare.run_wall_time_us":219341,"raft.queue_time_us":58376,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":462024,"threads_started":2}
I20260709 13:02:41.089383  5004 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.089936  5278 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:41.077836  1567 rpcz_store.cc:276] 0709 13:02:39.538577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.417707 (+879130us) service_pool.cc:224] Handling call
0709 13:02:40.418341 (+   634us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.441057 (+ 22716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.441104 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.441916 (+   812us) log.cc:844] Serialized 26 byte log entry
0709 13:02:40.748531 (+306615us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:40.748575 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.748663 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.077184 (+328521us) raft_consensus.cc:1685] finished
0709 13:02:41.077204 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.077345 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22087,"raft.queue_time_us":24000,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":240,"thread_start_us":302730,"threads_started":1,"wal-append.queue_time_us":632586}
I20260709 13:02:41.087570  5052 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:41.091976  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:41.036082  5274 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.093513  5274 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.096051  5274 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.096570  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:41.056016  5267 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:41.097329  5266 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:41.089818  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:41.103744  5266 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:41.104316  5266 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.104805  5266 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:41.105054  5241 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:41.107532  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2450) took 1569 ms. Trace:
I20260709 13:02:41.103606  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:41.097290  1569 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
W20260709 13:02:41.112484  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 81414.4us
W20260709 13:02:41.115077  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 78148.8us
I20260709 13:02:41.097426  1639 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:41.078047  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 632) took 1953 ms. Trace:
I20260709 13:02:41.155097  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:41.158833  5187 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:41.159341  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 55825.5us
W20260709 13:02:41.162110  5288 consensus_meta.cc:287] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.540s	user 0.016s	sys 0.004s
I20260709 13:02:41.170974  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2087) took 1335 ms. Trace:
I20260709 13:02:41.184043  5187 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.186537  5187 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:41.108081  1562 rpcz_store.cc:276] 0709 13:02:39.538294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.434419 (+896125us) service_pool.cc:224] Handling call
0709 13:02:40.515570 (+ 81151us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.560711 (+ 45141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.560747 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.578410 (+ 17663us) log.cc:844] Serialized 26 byte log entry
0709 13:02:40.764849 (+186439us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:40.764905 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.764997 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.107280 (+342283us) raft_consensus.cc:1685] finished
0709 13:02:41.107301 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.107458 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":845,"raft.queue_time_us":3668,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":17903,"thread_start_us":165461,"threads_started":1,"wal-append.queue_time_us":504954}
I20260709 13:02:41.173553  5269 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.153707  1716 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:41.179612  1565 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:41.183856  5272 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:41.182331  1568 rpcz_store.cc:276] 0709 13:02:39.835263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.558897 (+723634us) service_pool.cc:224] Handling call
0709 13:02:40.599055 (+ 40158us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.617807 (+ 18752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.617846 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:40.631302 (+ 13456us) log.cc:844] Serialized 52 byte log entry
0709 13:02:41.011513 (+380211us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:41.011638 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.011917 (+   279us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.170563 (+158646us) raft_consensus.cc:1685] finished
0709 13:02:41.170694 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.170871 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5357,"raft.queue_time_us":16058,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":11243,"spinlock_wait_cycles":1936256,"thread_start_us":295564,"threads_started":1,"wal-append.queue_time_us":468338}
W20260709 13:02:41.189522  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.171626  5237 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:41.023550  5198 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:41.192111  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:41.192474  1639 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:41.120939  1566 rpcz_store.cc:276] 0709 13:02:39.124620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.340649 (+1216029us) service_pool.cc:224] Handling call
0709 13:02:40.343669 (+  3020us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.437801 (+ 94132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.437858 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:40.476351 (+ 38493us) log.cc:844] Serialized 52 byte log entry
0709 13:02:40.680454 (+204103us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:40.680496 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.680561 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.077820 (+397259us) raft_consensus.cc:1685] finished
0709 13:02:41.077843 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.077990 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69564,"raft.queue_time_us":91548,"raft.run_cpu_time_us":447,"raft.run_wall_time_us":40411,"thread_start_us":203628,"threads_started":1,"wal-append.queue_time_us":598494}
I20260709 13:02:41.194832  4939 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:41.208977  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:41.210453  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:41.211274  1643 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:41.205389  5198 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:02:41.223013  5272 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.218077  5104 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:41.230826  5104 raft_consensus.cc:697] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:41.232136  5104 consensus_queue.cc:237] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:41.237234  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:41.237991  1643 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
W20260709 13:02:41.240139  4544 consensus_meta.cc:287] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.657s	user 0.010s	sys 0.010s
I20260709 13:02:41.244670  4403 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.246085  1533 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:41.242170  4544 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10.
I20260709 13:02:41.251679  4403 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.242983  5131 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:41.255990  1532 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:41.262511  1642 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
W20260709 13:02:41.256894  5069 consensus_meta.cc:287] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.628s	user 0.011s	sys 0.009s
I20260709 13:02:41.266574  5007 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.269374  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:41.276363  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:41.277493  1636 raft_consensus.cc:2410] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:02:41.288189  4403 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:41.267448  5263 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:41.270825  5198 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:41.267946  5266 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.293097  5263 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:41.304862  4831 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:41.310407  4831 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:41.311551  4831 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:41.312819  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 78126.2us
I20260709 13:02:41.275048  5069 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.290663  5248 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.018s
W20260709 13:02:41.318892  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 517312us
I20260709 13:02:41.309026  5263 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.294827  4544 consensus_queue.cc:1059] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:02:41.313045  5131 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.330878  1533 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:41.333655  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 49926.7us
I20260709 13:02:41.324664  4960 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.339696  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2456) took 1238 ms. Trace:
I20260709 13:02:41.334337  4831 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.342459  1563 rpcz_store.cc:276] 0709 13:02:40.101894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.639596 (+537702us) service_pool.cc:224] Handling call
0709 13:02:40.678263 (+ 38667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.735041 (+ 56778us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:02:41.202894 (+467853us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.339102 (+136208us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.339377 (+   275us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:41.339424 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.339490 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.339647 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":380521,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":7915,"thread_start_us":581928,"threads_started":2}
I20260709 13:02:41.347335  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:41.344890  5267 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.348644  4546 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:41.351666  5266 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.356343  5131 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.359812  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1964) took 1022 ms. Trace:
I20260709 13:02:41.365691  1717 rpcz_store.cc:276] 0709 13:02:40.337235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.363604 (+ 26369us) service_pool.cc:224] Handling call
0709 13:02:41.359689 (+996085us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b54000483d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:41.359767 (+    78us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25086848}
I20260709 13:02:41.372735  5337 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:41.373426  5337 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:41.373952  5337 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:41.379544  1635 raft_consensus.cc:2368] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10.
I20260709 13:02:41.342231  4960 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:41.380903  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:02:41.381603  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.383869  1717 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:41.389801  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:02:41.391814  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 47994.9us
I20260709 13:02:41.386857  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:41.397274  1711 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:41.397882  1710 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.388680  5269 raft_consensus.cc:3055] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:41.393306  5201 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.406739  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:41.409417  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:41.411932  1636 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:41.406878  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:41.412900  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:41.413328  2300 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:41.413496  1639 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.414160  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:41.414849  1643 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:41.415017  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:41.404340  5269 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:41.417404  1642 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 10: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:41.407178  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:41.430300  2305 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.429443  1532 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:41.436504  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:41.441388  2297 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:41.442235  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.443638  1533 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:41.445158  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:41.439747  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:41.447131  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:41.448700  4048 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:41.450313  4048 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:41.450733  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:41.447178  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:41.452755  4048 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:41.456982  1710 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:41.458515  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:41.459451  1710 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:41.436087  5198 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:41.460103  2305 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:41.468976  5198 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:41.470484  5198 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:41.471343  2303 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:41.477358  5269 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.478154  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:41.478852  1641 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:41.479291  5269 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.482040  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2458) took 1058 ms. Trace:
I20260709 13:02:41.463147  1711 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:41.448798  4960 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:41.467238  5274 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:41.467058  1533 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:41.467788  5104 raft_consensus.cc:2764] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:02:41.468186  5045 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:41.490734  5301 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:02:41.495343  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.495230  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 160359us
I20260709 13:02:41.436573  5069 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:41.482439  1560 rpcz_store.cc:276] 0709 13:02:40.423656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.430054 (+  6398us) service_pool.cc:224] Handling call
0709 13:02:41.481859 (+1051805us) spinlock_profiling.cc:243] Waited 95.5 ms on lock 0x7b5400325d54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:41.481959 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219687680}
I20260709 13:02:41.499248  1639 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:41.499708  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:41.416360  5201 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.503413  5274 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:41.504012  2297 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:41.446897  5272 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:41.494273  5263 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.507114  1711 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:41.521526  5248 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:02:41.514945  5266 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:41.524235  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1963) took 1187 ms. Trace:
I20260709 13:02:41.524772  1712 rpcz_store.cc:276] 0709 13:02:40.336922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.480642 (+143720us) service_pool.cc:224] Handling call
0709 13:02:41.523214 (+1042572us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b58001e4598. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:02:41.523446 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.523822 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.523850 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.523922 (+    72us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:41.523956 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.524019 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.524132 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44258560}
I20260709 13:02:41.510200  1533 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:41.501777  5301 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:41.515386  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:41.524791  5266 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:41.531338  5339 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:02:41.551599  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:41.564172  4937 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.563992  4960 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:41.566394  4831 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:02:41.571836  4960 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.569382  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:41.574028  4960 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.534569  5294 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.536132  5131 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:41.545960  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.576090  4403 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.576344  5301 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:41.577054  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:41.576752  4403 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:41.578689  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:41.578428  5301 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.581048  4403 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:41.581914  4403 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:41.526814  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:41.555931  5338 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 13:02:41.583451  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 56770.9us
I20260709 13:02:41.584290  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:41.585045  5092 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:41.585072  1643 raft_consensus.cc:1217] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 6.8->[8.9-8.9]   Dedup: 8.9->[]
I20260709 13:02:41.585731  5092 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:41.585915  1643 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:41.586990  4831 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.588570  4831 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.587682  5242 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:41.561470  2269 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:41.589754  5242 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:41.590524  5197 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:41.590988  5197 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.590792  5242 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.591544  5197 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:41.593264  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 1948) took 1382 ms (client timeout 1728 ms). Trace:
W20260709 13:02:41.593861  1718 rpcz_store.cc:269] 0709 13:02:40.211058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.215127 (+  4069us) service_pool.cc:224] Handling call
0709 13:02:41.593037 (+1377910us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b5400239e54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:41.593182 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25778048}
I20260709 13:02:41.595670  5187 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:41.564531  5341 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.596943  4997 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:41.597019  5341 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:41.597708  4997 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:41.599792  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2090) took 1073 ms. Trace:
I20260709 13:02:41.599982  5341 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:41.600747  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 852) took 1202 ms. Trace:
I20260709 13:02:41.601219  2296 rpcz_store.cc:276] 0709 13:02:40.398841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.399055 (+   214us) service_pool.cc:224] Handling call
0709 13:02:41.600547 (+1201492us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b540068d3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:41.600690 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":691706112}
I20260709 13:02:41.601620  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:41.602586  1635 raft_consensus.cc:2410] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:02:41.600399  1714 rpcz_store.cc:276] 0709 13:02:40.526000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.759567 (+233567us) service_pool.cc:224] Handling call
0709 13:02:41.599560 (+839993us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54003d9fd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:41.599710 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304046464}
I20260709 13:02:41.608945  1640 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:41.609938  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 735) took 1110 ms. Trace:
I20260709 13:02:41.610314  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:41.586747  5092 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.589048  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 733) took 1119 ms. Trace:
I20260709 13:02:41.527899  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:41.563704  1609 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:41.591555  5300 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:41.635376  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 71746.8us
W20260709 13:02:41.636188  5323 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 1.094s	user 0.003s	sys 0.002s
I20260709 13:02:41.639312  4939 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:41.641836  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:41.642594  1637 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
W20260709 13:02:41.605384  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.538947  5248 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:02:41.643745  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.644078  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 116285us
I20260709 13:02:41.644789  1533 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:41.646044  1533 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:41.608433  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 734) took 1138 ms. Trace:
I20260709 13:02:41.542654  5069 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.507557  1710 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:02:41.647282  4048 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:41.647820  4048 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.648087  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 726) took 1308 ms. Trace:
I20260709 13:02:41.648273  4048 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.648524  1710 rpcz_store.cc:276] 0709 13:02:40.339441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.500957 (+1161516us) service_pool.cc:224] Handling call
0709 13:02:41.501477 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.647695 (+146218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.647938 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.649243  5272 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:41.649950  1710 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:41.565599  1568 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:41.650647  5266 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.610369  1714 rpcz_store.cc:276] 0709 13:02:40.500024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.609074 (+1109050us) service_pool.cc:224] Handling call
0709 13:02:41.609372 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.609572 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.609600 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.609644 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:41.609709 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.609767 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.609881 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.647107  1718 rpcz_store.cc:276] 0709 13:02:40.470074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.607573 (+1137499us) service_pool.cc:224] Handling call
0709 13:02:41.607880 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.608076 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.608104 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.608172 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:41.608207 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.608266 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.608370 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.652233  1714 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:41.652872  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 737) took 1125 ms. Trace:
I20260709 13:02:41.653190  1714 rpcz_store.cc:276] 0709 13:02:40.527262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.651708 (+1124446us) service_pool.cc:224] Handling call
0709 13:02:41.652045 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.652649 (+   604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.652821 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.546301  1682 leader_election.cc:400] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9.
I20260709 13:02:41.614128  1717 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:41.655056  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:41.655857  4960 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.647236  5248 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:41.656607  5248 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:41.656437  4960 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:41.657056  5248 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:41.657418  4048 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.535677  1713 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:41.659557  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 727) took 1319 ms. Trace:
I20260709 13:02:41.660038  1713 rpcz_store.cc:276] 0709 13:02:40.339715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.535097 (+1195382us) service_pool.cc:224] Handling call
0709 13:02:41.535452 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.659193 (+123741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.659456 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.661957  1713 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:41.582801  4403 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:41.663101  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:41.533562  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1954) took 1320 ms. Trace:
I20260709 13:02:41.630946  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2088) took 1105 ms. Trace:
I20260709 13:02:41.655095  1682 leader_election.cc:403] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:02:41.665038  1712 rpcz_store.cc:276] 0709 13:02:40.525258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.629918 (+1104660us) service_pool.cc:224] Handling call
0709 13:02:41.630286 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.630492 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.630542 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.630593 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:41.630660 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.630729 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.630861 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.606710  5267 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.664757  1709 rpcz_store.cc:276] 0709 13:02:40.213443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.285860 (+ 72417us) service_pool.cc:224] Handling call
0709 13:02:40.286336 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.326907 (+ 40571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.326993 (+    86us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:40.327085 (+    92us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:40.327128 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.533170 (+1206042us) spinlock_profiling.cc:243] Waited 40.3 ms on lock 0x7b540033aed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:41.533245 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.533429 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92581632}
I20260709 13:02:41.624724  1715 rpcz_store.cc:276] 0709 13:02:40.469714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.588104 (+1118390us) service_pool.cc:224] Handling call
0709 13:02:41.588436 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.588699 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.588728 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.588771 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:41.588804 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.588861 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.588968 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.624643  5263 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.644595  5294 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:41.647653  5069 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.623473  4997 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.630285  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:41.652235  1718 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:41.668743  4403 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:41.650691  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2089) took 1125 ms. Trace:
I20260709 13:02:41.678977  1710 rpcz_store.cc:276] 0709 13:02:40.525659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.649430 (+1123771us) service_pool.cc:224] Handling call
0709 13:02:41.649772 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.650411 (+   639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.650628 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.662761  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1970) took 1090 ms. Trace:
I20260709 13:02:41.539258  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:41.675683  1681 leader_election.cc:400] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9.
I20260709 13:02:41.669054  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:41.533593  5201 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:41.684329  1717 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:41.686627  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2094) took 1147 ms. Trace:
I20260709 13:02:41.687083  1718 rpcz_store.cc:276] 0709 13:02:40.531539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.651771 (+1120232us) service_pool.cc:224] Handling call
0709 13:02:41.652070 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.678136 (+ 26066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.678414 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33536}
I20260709 13:02:41.676452  4048 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.654728  1714 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:41.688577  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2095) took 1156 ms. Trace:
I20260709 13:02:41.688792  1565 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:02:41.688948  1714 rpcz_store.cc:276] 0709 13:02:40.531906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.654023 (+1122117us) service_pool.cc:224] Handling call
0709 13:02:41.654336 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.688316 (+ 33980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.688519 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.680730  1713 rpcz_store.cc:276] 0709 13:02:40.572711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.661107 (+1088396us) service_pool.cc:224] Handling call
0709 13:02:41.661497 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.662397 (+   900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.662684 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.681592  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:41.683732  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 137569us
I20260709 13:02:41.692184  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:41.682443  5104 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.693372  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:41.658819  4960 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:41.690806  1714 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:41.686376  5272 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:41.704174  5272 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.704717  5272 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:41.696877  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:41.705763  1563 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.697613  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:41.671418  5187 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:41.714017  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:41.700286  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:41.706734  2305 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
W20260709 13:02:41.706782  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 231734us
W20260709 13:02:41.707926  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 168763us
I20260709 13:02:41.717798  1533 leader_election.cc:400] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10.
I20260709 13:02:41.718293  1533 leader_election.cc:403] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:41.712174  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:41.719254  1568 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.702822  5104 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.720294  1560 raft_consensus.cc:2410] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:02:41.722143  1682 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:41.721215  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.727568  4997 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:41.728380  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 149561us
I20260709 13:02:41.695153  5341 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9.
I20260709 13:02:41.735935  1568 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
W20260709 13:02:41.739006  1464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45164 (request call id 363) took 20389 ms (client timeout 15000 ms). Trace:
I20260709 13:02:41.742208  1565 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:41.711572  4546 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:41.720256  4544 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:41.744779  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:41.746294  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:41.746551  5243 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:41.747052  4048 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:02:41.748581  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:41.742748  1532 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:41.747989  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 172895us
I20260709 13:02:41.731618  5263 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:41.772789  5201 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:41.773528  5201 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:41.754012  1711 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:41.769788  1635 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:41.770355  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:41.776477  1718 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.775151  1566 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:41.782126  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:41.783013  1640 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:41.771919  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:41.785694  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.786157  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.741925  4939 raft_consensus.cc:697] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:41.790431  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:41.793239  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 71216.6us
I20260709 13:02:41.795317  4403 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:41.805670  4403 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:41.764032  4831 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.732380  5187 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.835961  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:41.789613  4546 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:41.849623  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 107978us
I20260709 13:02:41.850477  1566 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: [6.9-6.9]
I20260709 13:02:41.852088  5144 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:02:41.808830  5263 raft_consensus.cc:697] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:41.853195  4939 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.853820  5263 consensus_queue.cc:237] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:41.852303  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:41.866075  1560 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:41.853722  5104 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:41.861337  1683 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:41.864009  2299 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:41.836326  4960 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:41.781203  5292 raft_consensus.cc:2764] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:02:41.859740  4403 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.780380  4544 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:41.885351  5349 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.903301  5349 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:41.904011  1681 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:41.906152  5349 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:41.861637  2302 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:41.907032  4960 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:41.882304  1609 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:41.884634  5267 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.775655  5198 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 13:02:41.854842  5069 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:41.783658  5335 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:41.810545  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:41.899457  5187 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:41.892874  5301 raft_consensus.cc:2764] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:02:41.970403  5301 raft_consensus.cc:2764] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:02:41.860608  5274 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:02:41.774732  5201 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.907627  4960 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:41.912981  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:41.859047  5336 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:41.911197  1609 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:41.931221  1644 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.041s	sys 0.067s
I20260709 13:02:41.923132  1718 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
W20260709 13:02:41.909744  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.978708  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:41.863799  5197 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:41.982235  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 120985us
W20260709 13:02:41.984333  1644 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.041s	sys 0.072s
I20260709 13:02:41.985003  4048 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10.
I20260709 13:02:41.933257  5349 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:41.979439  4546 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:42.029353  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 384957us
I20260709 13:02:41.983017  4960 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.036284  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:42.036974  1568 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:42.037001  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:41.742496  1464 rpcz_store.cc:269] 0709 13:02:21.286221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.286514 (+   293us) service_pool.cc:224] Handling call
0709 13:02:41.669819 (+20383305us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:34.346040 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:34.367619 (+ 21579us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:34.367673 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:02:34.367718 (+    45us) tablet.cc:662] Decoding operations
0709 13:02:34.606981 (+239263us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:34.607099 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:34.607146 (+    47us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:02:34.838681 (+231535us) tablet.cc:701] Row locks acquired
0709 13:02:34.838732 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:02:34.838947 (+   215us) write_op.cc:270] Start()
0709 13:02:34.839140 (+   193us) write_op.cc:276] Timestamp: P: 1783602154838861 usec, L: 0
0709 13:02:34.839158 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:02:34.839934 (+   776us) log.cc:844] Serialized 5590 byte log entry
0709 13:02:37.274691 (+2434757us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:37.885875 (+611184us) op_driver.cc:464] REPLICATION: finished
0709 13:02:38.064372 (+178497us) write_op.cc:301] APPLY: starting
0709 13:02:38.078087 (+ 13715us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:38.095398 (+ 17311us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:38.095444 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:38.097151 (+  1707us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:38.097487 (+   336us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 13:02:38.097529 (+    42us) write_op.cc:312] APPLY: finished
0709 13:02:38.097919 (+   390us) log.cc:844] Serialized 101 byte log entry
0709 13:02:38.098124 (+   205us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:38.098325 (+   201us) write_op.cc:454] Released schema lock
0709 13:02:38.098414 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:38.098849 (+   435us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:38.098882 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":92726,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":130537,"apply.run_cpu_time_us":4026,"apply.run_wall_time_us":55628,"num_ops":14,"prepare.queue_time_us":372423,"prepare.run_cpu_time_us":20394,"prepare.run_wall_time_us":657150,"raft.queue_time_us":197396,"raft.run_cpu_time_us":40542,"raft.run_wall_time_us":73517,"replication_time_us":3046534,"spinlock_wait_cycles":37432064,"thread_start_us":581340,"threads_started":4,"wal-append.queue_time_us":554492,"wal-append.run_cpu_time_us":38010,"wal-append.run_wall_time_us":3693253}]]}
I20260709 13:02:41.913582  5131 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.940896  5267 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:41.999724  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 88642.5us
I20260709 13:02:42.039602  1565 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
W20260709 13:02:42.029057  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 395036us
I20260709 13:02:41.880641  5337 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:41.911475  4937 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.046353  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:42.043752  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 269153us
I20260709 13:02:42.044523  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:42.004575  4831 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:42.053772  1566 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:42.058761  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:42.003669  5349 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:42.035211  4546 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:42.075103  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 288072us
I20260709 13:02:42.076994  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:42.077917  1641 raft_consensus.cc:2410] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:02:42.078444  4048 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:42.077782  1642 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:42.057878  4831 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:42.099025  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.128249  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 335790us
I20260709 13:02:42.042385  5069 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:02:42.074801  5349 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:42.137414  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 91137.7us
I20260709 13:02:42.089769  5092 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:42.046272  5131 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.130973  4831 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:42.160017  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 120567us
I20260709 13:02:42.160215  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:42.097205  5294 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:42.174459  1683 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:42.176517  1683 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:42.092614  4937 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:42.164326  1534 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.632s (SENT) [suppressed 10 similar messages]
W20260709 13:02:42.187431  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 215863us
I20260709 13:02:42.147600  5266 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:42.190378  5294 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:42.182755  5353 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:42.154543  5092 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:42.196391  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:42.054486  4544 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:42.211578  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 401115us
I20260709 13:02:42.216415  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:42.233240  4048 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:42.194845  5302 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.014s	sys 0.010s
W20260709 13:02:42.234725  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.247570  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:42.247900  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:42.186491  5007 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:42.247355  4048 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.199689  5104 raft_consensus.cc:3055] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:42.248924  1640 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:42.215167  5092 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.262507  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:42.249053  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:42.195218  5300 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:42.195914  5197 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:42.248536  1560 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
W20260709 13:02:42.241106  5302 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.014s	sys 0.014s
I20260709 13:02:42.232823  5341 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:42.278344  1563 consensus_meta.cc:287] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.003s
I20260709 13:02:42.281482  1563 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:42.279475  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:42.291394  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:42.292146  4937 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:42.292814  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:42.300125  5267 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:42.259550  5104 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:02:42.302989  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.209940  5266 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.281317  5341 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.225097  5353 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:42.314558  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 138209us
I20260709 13:02:42.276793  5301 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:42.236231  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.327816  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 173455us
I20260709 13:02:42.332376  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:42.333405  1710 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:02:42.321264  5248 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.664s	user 0.007s	sys 0.006s
W20260709 13:02:42.323907  4997 consensus_meta.cc:287] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.582s	user 0.008s	sys 0.003s
W20260709 13:02:42.340051  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.325752  5347 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
W20260709 13:02:42.328944  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 48466.9us
I20260709 13:02:42.358548  5144 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:42.330593  1566 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
W20260709 13:02:42.324204  5272 consensus_meta.cc:287] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.619s	user 0.007s	sys 0.006s
I20260709 13:02:42.353718  1635 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:42.362447  5272 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.329449  1641 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:42.365274  5144 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:42.365306  5272 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:42.365988  5144 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:42.355894  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.368757  5144 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:42.345810  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.382216  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.332573  4937 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:42.401353  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 46525us
I20260709 13:02:42.402254  1681 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:42.325491  5341 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:42.351802  2299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 845) took 3011 ms (client timeout 2873 ms). Trace:
I20260709 13:02:42.325703  5301 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.403946  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:42.406589  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 210284us
I20260709 13:02:42.409679  1635 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:42.411103  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.412667  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:42.406965  2299 rpcz_store.cc:269] 0709 13:02:39.323771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.470562 (+146791us) service_pool.cc:224] Handling call
0709 13:02:42.324096 (+2853534us) spinlock_profiling.cc:243] Waited 2.36 s on lock 0x7b5400708fd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:42.335065 (+ 10969us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5437036288}
I20260709 13:02:42.333436  5358 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:42.413414  1636 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:42.355374  5267 raft_consensus.cc:2764] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:02:42.355458  5266 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:42.424338  1681 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:42.424692  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:42.427897  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:42.406795  4937 raft_consensus.cc:697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:42.414395  5301 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:42.441287  2269 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:42.434743  5007 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:42.447757  5007 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.448235  5007 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:42.448596  5354 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:42.451692  4937 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.452790  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:42.454113  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:42.454387  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:42.455180  1717 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:42.455543  5352 raft_consensus.cc:3055] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:42.456064  5352 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:42.457057  5352 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.457785  4960 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.458866  5335 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:42.457816  5300 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:42.460604  5335 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:02:42.460906  5350 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.335s	user 0.002s	sys 0.000s
I20260709 13:02:42.461033  5335 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:42.463433  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51172us
I20260709 13:02:42.463658  4997 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:42.464519  1643 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:02:42.465883  1683 leader_election.cc:400] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7.
I20260709 13:02:42.466145  1643 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:42.419734  5104 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.467797  5104 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:42.469731  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.455338  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:42.454698  1642 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:42.472635  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:42.455446  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:42.474050  1713 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:42.458201  1681 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:42.476575  1717 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:02:42.355754  5248 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.479786  1710 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:42.462293  4831 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.485025  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:42.472364  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.485945  1715 raft_consensus.cc:2393] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:42.489066  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 54901.1us
I20260709 13:02:42.489810  2299 raft_consensus.cc:2393] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 9: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:42.475785  5198 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.484848  5187 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:42.485013  5356 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:42.520612  4960 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:42.524385  5040 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:42.524907  5040 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.525288  5040 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:42.526073  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 60357.3us
W20260709 13:02:42.508877  5359 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 13:02:42.529287  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:42.530058  5131 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:42.531253  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1977) took 1080 ms. Trace:
I20260709 13:02:42.531720  1709 rpcz_store.cc:276] 0709 13:02:41.450974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.683491 (+232517us) service_pool.cc:224] Handling call
0709 13:02:41.706016 (+ 22525us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:41.738105 (+ 32089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.738149 (+    44us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:41.747442 (+  9293us) log.cc:844] Serialized 78 byte log entry
0709 13:02:42.060992 (+313550us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:42.281109 (+220117us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.440804 (+159695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.440886 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.530984 (+ 90098us) raft_consensus.cc:1685] finished
0709 13:02:42.531004 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.531172 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":192693,"prepare.queue_time_us":109695,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":184610,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":22511,"thread_start_us":442725,"threads_started":2,"wal-append.queue_time_us":515267}
I20260709 13:02:42.533703  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:02:42.534456  1709 raft_consensus.cc:606] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:02:42.497956  5336 consensus_queue.cc:1059] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:02:42.535308  5356 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.536352  5336 raft_consensus.cc:3055] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:42.536728  5336 raft_consensus.cc:740] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:42.537825  5356 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:42.537621  5336 consensus_queue.cc:260] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.512735  5187 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.538702  5187 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:42.499032  5294 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:42.543623  2302 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:42.525228  4831 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:42.547446  4403 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.548475  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.550024  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:42.544023  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:42.547433  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 1973) took 1220 ms. Trace:
I20260709 13:02:42.561228  2305 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:42.540931  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.562422  2297 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:42.560827  1566 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:42.545423  1710 raft_consensus.cc:1275] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:02:42.562796  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:42.549671  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:42.543365  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:42.564118  2303 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.564213  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:42.561198  1637 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:42.566496  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:42.567098  1642 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:42.569169  5353 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:42.570858  4048 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:42.435422  5358 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:42.506829  5337 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:42.575153  1635 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:42.538875  5336 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:42.534485  5007 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:42.582669  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.585155  2269 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:42.561668  1712 rpcz_store.cc:276] 0709 13:02:41.327133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.673556 (+346423us) service_pool.cc:224] Handling call
0709 13:02:41.690857 (+ 17301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.738256 (+ 47399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.738307 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.781473 (+ 43166us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.028568 (+247095us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:42.028622 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.028727 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.547221 (+518494us) raft_consensus.cc:1685] finished
0709 13:02:42.547241 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.547372 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52962,"raft.queue_time_us":47247,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":12011,"thread_start_us":136122,"threads_started":1,"wal-append.queue_time_us":644173}
I20260709 13:02:42.586521  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.587567  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.591825  2298 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:42.538530  4546 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:42.589437  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 119105us
I20260709 13:02:42.590814  5269 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:42.511797  5198 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.511087  5372 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.151s
I20260709 13:02:42.574082  5294 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:42.604314  5349 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.605129  4831 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:42.608405  1682 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:42.599686  2303 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
W20260709 13:02:42.587930  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 146453us
I20260709 13:02:42.601382  5198 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:42.608582  5379 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:42.611575  1609 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:42.599208  1532 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:42.610103  1638 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
W20260709 13:02:42.620622  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 54237.4us
I20260709 13:02:42.621052  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.621222  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:02:42.621456  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:42.621850  1709 raft_consensus.cc:606] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:02:42.622249  1709 raft_consensus.cc:614] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:42.622577  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.623807  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.624203  5301 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:42.625015  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.625782  2298 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:42.626547  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:42.614233  4960 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:42.629125  5341 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:42.630316  1718 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:02:42.632916  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:42.634879  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.634624  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:42.635490  1642 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:42.637276  1641 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:02:42.622259  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.640415  2304 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:42.606715  5349 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:42.644228  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.626680  5301 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:42.648401  2296 raft_consensus.cc:2393] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:42.637738  1641 pending_rounds.cc:85] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 7
I20260709 13:02:42.463472  5335 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.669565  4403 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:02:42.669413  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 57960.5us
I20260709 13:02:42.672399  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.613895  5337 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:42.651355  1641 pending_rounds.cc:107] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 8.8
W20260709 13:02:42.676635  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 55276.3us
I20260709 13:02:42.605690  4831 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:42.680459  1641 raft_consensus.cc:2889] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:42.688200  5104 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:42.648900  5198 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:42.638207  5248 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:42.692309  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 58166.8us
I20260709 13:02:42.695192  1709 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:02:42.695612  4403 raft_consensus.cc:697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:42.701601  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.702222  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.707098  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:42.677806  4960 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.679271  5294 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:42.708740  2296 raft_consensus.cc:2393] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:42.714116  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.717868  2300 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:42.718936  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:42.719417  1642 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:42.719775  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6711207c14044518a57e7c8914ac0091"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:02:42.721935  1710 raft_consensus.cc:606] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:42.722247  1609 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:42.722692  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:42.722285  1710 raft_consensus.cc:614] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:42.725433  4403 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.708009  5341 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:42.749053  5385 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:42.647526  5274 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:02:42.717564  5349 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.721383  5198 raft_consensus.cc:697] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:42.749100  5292 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:42.726822  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:42.771013  5292 raft_consensus.cc:697] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:42.771364  5198 consensus_queue.cc:237] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:42.779464  5366 log.cc:927] Time spent T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:02:42.782449  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 730) took 2441 ms. Trace:
W20260709 13:02:42.783450  5361 log.cc:927] Time spent T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:02:42.756465  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.765616  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:42.749790  2304 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEARNER]: Advancing to term 11
I20260709 13:02:42.673655  5092 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:42.787333  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.788705  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:42.789917  2296 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:42.770043  5391 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:42.769636  5349 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.706576  4937 consensus_queue.cc:1059] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:02:42.800238  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:42.782980  1716 rpcz_store.cc:276] 0709 13:02:40.340612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.548600 (+1207988us) service_pool.cc:224] Handling call
0709 13:02:41.549004 (+   404us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:41.549502 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:02:41.968300 (+418798us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.413980 (+445680us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:42.435278 (+ 21298us) log.cc:844] Serialized 52 byte log entry
0709 13:02:42.435425 (+   147us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:42.435460 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.435751 (+   291us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.782149 (+346398us) raft_consensus.cc:1685] finished
0709 13:02:42.782177 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.782348 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88801,"prepare.queue_time_us":575622,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":92984,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":7493,"thread_start_us":692620,"threads_started":2,"wal-append.queue_time_us":582133}
I20260709 13:02:42.724159  5294 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:42.729511  5335 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.746995  5337 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:42.771962  5292 consensus_queue.cc:237] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:42.794489  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.726827  4960 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.759397  5007 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:42.759207  5385 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:42.799700  5266 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:42.807601  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2041) took 1397 ms. Trace:
W20260709 13:02:42.825748  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.875710  5351 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:02:42.830058  2303 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:42.830709  2300 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:42.701581  5272 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:02:42.831727  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:42.878480  1641 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:42.879803   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:02:42.880460  5349 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:42.882376   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:42.832201  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:42.883299  2296 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.764603  5379 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 13:02:42.886535  4546 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:42.805130  5040 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:42.889705  5040 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:42.893522  5092 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:42.894953  1641 raft_consensus.cc:2410] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:02:42.895875  2296 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
W20260709 13:02:42.897224  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 97085.8us
I20260709 13:02:42.898064  1683 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:42.879925  5392 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:02:42.806697  5294 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:42.903165  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:42.830103  5341 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:42.921671  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.831722  5269 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:42.926483  5266 raft_consensus.cc:697] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:42.926254  5391 raft_consensus.cc:697] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:42.933048  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 144381us
I20260709 13:02:42.896229  5385 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:42.935096  5337 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.875231  1644 rpcz_store.cc:276] 0709 13:02:41.410240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.410374 (+   134us) service_pool.cc:224] Handling call
0709 13:02:41.415329 (+  4955us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.415815 (+   486us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:42.003448 (+587633us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.493373 (+489925us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.511382 (+ 18009us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.511529 (+   147us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:42.511561 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.511622 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.802959 (+291337us) raft_consensus.cc:1685] finished
0709 13:02:42.802978 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.807541 (+  4563us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":113410,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":69,"raft.queue_time_us":174836,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":12893,"thread_start_us":1000350,"threads_started":3,"wal-append.queue_time_us":518518}
I20260709 13:02:42.928530  5007 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:42.941134  5358 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:42.942734  4960 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:42.941135  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:42.953248  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:42.959764  2304 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:42.960922  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:42.954083  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:42.962188  1569 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:42.962476  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:42.963075  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:42.963647  1567 raft_consensus.cc:2376] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7. Re-sending same reply.
I20260709 13:02:42.965489  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:42.966183  1560 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:42.967125  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:42.967765  1562 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
W20260709 13:02:42.968655  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 70732.5us
I20260709 13:02:42.969933  1683 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:42.970675  4960 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:02:42.975448  5007 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:42.979954  1683 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:42.980949  2269 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:42.981439  1683 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:42.981909  5384 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:42.982251  4960 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:42.982362  5384 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.983799  1683 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:42.984680  4960 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:42.985803  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:42.986608  1561 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:42.988612  5352 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:42.991222  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:42.993585  1683 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:42.994315  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:42.994982  1640 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:42.995198  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:42.996421  5040 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:42.996675  1683 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:42.997102  5040 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:42.997613  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:42.998201  5358 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:42.998247  1711 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:42.997740  4939 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.998890  5358 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:42.999651  4939 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:43.000789  1683 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:43.001235  5131 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:43.001639  5131 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:43.001717  4939 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:43.001964  1640 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:43.002110  5131 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.002331  4939 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:43.002533  1683 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:43.003883  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:43.009009  1710 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:42.997995  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:43.011158  1716 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.019865  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:43.018782  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.022732  2299 raft_consensus.cc:2410] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:02:43.025097  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:43.025291  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:43.025889  1561 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:42.982864  5384 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:43.029695  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:43.025758  1717 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:43.031761  5395 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:43.040705  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.014623  5040 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:43.047297  1682 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:42.980967  5007 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:43.039340  4831 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:43.057426  2304 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:43.063402  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:43.066608  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:43.067113  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:43.068099  1717 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:43.070356  4403 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:43.046026  5272 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:43.071205  4403 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.076663  4403 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.075477  4831 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:43.077443  4831 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.078361  4831 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:43.083397  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.084479  4403 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:43.086844  4403 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:43.088718  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:43.089391  1709 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.089828  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.090513  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:43.090508  2296 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 6.
I20260709 13:02:43.091627  2305 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:43.091959  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:43.091744  4831 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:43.092543  2297 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.092684  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.064025  5266 consensus_queue.cc:237] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:43.095286  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.095922  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:43.096246  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:43.096537  1714 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:43.096750  1709 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:43.096815  1566 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:43.100580  1716 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:43.103536  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:43.106035  4403 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:43.107414  4403 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.108599  4403 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:43.109675  1566 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:43.110357  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:43.109696  1714 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:43.077598  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:43.021147  4939 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:43.066685  5187 consensus_meta.cc:287] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.527s	user 0.005s	sys 0.005s
I20260709 13:02:42.970621  5391 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:43.058794  5007 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:43.040555  5274 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:02:43.141561  4544 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:43.145730  4544 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:43.150175  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:43.151559  1566 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 9: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:02:43.156282  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.156608  4403 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:43.157231  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.115720  5269 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:43.143955  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.110499  4960 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:43.162062  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:43.121786  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:43.165235  1713 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:02:43.165540  1716 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 10: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:43.167315  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:43.169973  4048 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:43.170214  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.171828  2304 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:43.175239  1643 raft_consensus.cc:2368] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11.
I20260709 13:02:43.177001  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:43.177933  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:43.179912  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:43.180763  1640 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:43.181025  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.181836  2304 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:43.188081  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.189510  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.191277  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.192806  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:43.196287  5294 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:43.198310  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.198990  2299 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:43.199770  1644 raft_consensus.cc:1275] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:02:43.200408  5404 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.163048  5358 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:43.201020  5404 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:43.116027  5272 raft_consensus.cc:697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:43.191188  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:43.204031  1641 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:43.204275  1636 raft_consensus.cc:2368] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11.
I20260709 13:02:43.205848  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:43.206462  1561 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:43.211377  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:43.212090  2301 raft_consensus.cc:2376] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9. Re-sending same reply.
I20260709 13:02:43.213146  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.190737  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.159402  5007 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:43.228706  4403 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:43.229646  2302 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:43.232875  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.233796  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:43.243763  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.234617  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.244215  1561 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:02:43.235590  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.236951  5404 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:43.246811  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:43.237907  5365 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:43.238679  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:43.245514  2299 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:43.172894  5269 raft_consensus.cc:697] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:43.249332  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.115375  4831 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:43.251796  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.257814  2297 raft_consensus.cc:2410] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:02:43.120769  5187 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:43.262965  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 55275.5us
I20260709 13:02:43.261847  2298 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
W20260709 13:02:43.280864  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 116887us
I20260709 13:02:43.281663  2296 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:43.282243  2302 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:43.255201  5372 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:43.284430  1533 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:43.305974  2305 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:43.308209  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:43.178244  5369 consensus_queue.cc:1059] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:02:43.264783  5365 raft_consensus.cc:697] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:43.316388  2299 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:43.243220  5385 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:43.319167  2300 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:43.325274  1607 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:43.324514  1681 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:43.327989  1607 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:43.332518  4403 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:43.332890  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:43.333325  4403 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:43.334056  4403 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.334264  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:43.335764  5404 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:43.392798  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 108462us
I20260709 13:02:43.320180  5201 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:43.331187  5266 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:43.318362  5272 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:43.416055  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 81884.6us
I20260709 13:02:43.345064  4939 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:43.424440  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:43.418810  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:43.393776  4831 raft_consensus.cc:2764] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:02:43.426550  1533 leader_election.cc:400] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11.
I20260709 13:02:43.427021  1533 leader_election.cc:403] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:02:43.394563  4960 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:02:43.411644  5410 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:02:43.428566  4403 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:43.432453  5349 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:43.432963  5349 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:43.433367  5349 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:43.433761  4403 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:43.423560  5269 consensus_queue.cc:237] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:43.440549  5385 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:43.441819  5335 consensus_meta.cc:287] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.626s	user 0.009s	sys 0.002s
I20260709 13:02:43.452816  4960 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:43.453449  5336 consensus_meta.cc:287] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.863s	user 0.006s	sys 0.006s
I20260709 13:02:43.430857  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:43.416936  5242 raft_consensus.cc:3055] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:02:43.409904  5397 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.446s	user 0.000s	sys 0.003s
I20260709 13:02:43.415429  5144 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:43.472913  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 56341us
I20260709 13:02:43.454087  4960 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:43.491082  5349 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:43.493947  5349 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:43.426688  5266 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:43.498535  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:43.501920  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:43.507881  5369 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:43.516405  5370 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11.
W20260709 13:02:43.518213  5416 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:02:43.521276  5394 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.600s	user 0.000s	sys 0.002s
I20260709 13:02:43.523185  5349 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:43.523947  5349 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:43.525121  5349 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:43.526763  5399 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:43.527385  5399 raft_consensus.cc:697] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:43.528012  1638 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:43.528371  5399 consensus_queue.cc:237] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:43.530755  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:43.531126  2268 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:43.531585  5353 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:43.532871  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:43.535120  5352 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:43.536006  5352 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:43.536823  5352 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:43.470050  5385 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:43.539736  5385 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.540166  5385 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.543295  5294 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:43.545107  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:43.545569  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.424096  5411 consensus_queue.cc:1059] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
W20260709 13:02:43.474416  5406 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:02:43.469794  5201 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:43.468801  5242 raft_consensus.cc:740] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:43.565778  1644 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:02:43.565681  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:43.472131  5144 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:43.567342  1718 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:43.573042  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:43.573674  1711 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:43.577760  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:43.580972  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:43.581710  1717 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:43.563282  5336 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.458498  5365 consensus_queue.cc:237] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:43.588833  1718 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:43.588402  5266 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:43.591190  5242 consensus_queue.cc:260] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:43.570358  5144 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.592273  5336 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:43.594408  5336 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:43.597297  5131 consensus_meta.cc:287] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.595s	user 0.007s	sys 0.004s
I20260709 13:02:43.597756  5131 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:43.600759  5408 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 13:02:43.603158  5187 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:43.614311  5404 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:43.615445  5356 raft_consensus.cc:3055] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:43.615931  5356 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:43.616789  5356 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:43.617344  5242 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.617790  5356 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.635547  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:43.636971  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 55773.4us
I20260709 13:02:43.637728  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:02:43.638386  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.641101  2304 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:43.641745  1711 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:02:43.647316  2301 raft_consensus.cc:1240] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:43.647470  5131 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:43.648439  5294 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:43.656599  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:43.659663  1711 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:43.659765  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.650355  5294 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:02:43.663437  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 242820us
I20260709 13:02:43.672873  1533 leader_election.cc:400] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11.
I20260709 13:02:43.673213  1533 leader_election.cc:403] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:02:43.663043  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:43.660480  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 361635us
I20260709 13:02:43.664594  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:43.674045  1561 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:43.690987  1534 leader_election.cc:336] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.606s (SENT)
I20260709 13:02:43.685977  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:43.676016  1562 raft_consensus.cc:2368] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11.
I20260709 13:02:43.705053  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:43.705739  2296 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.707463  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:43.708117  1568 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
W20260709 13:02:43.708745  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.711611  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:43.711823  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:43.712239  1564 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:43.701844  1534 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:43.714962  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 53253.9us
I20260709 13:02:43.719658  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:43.720517  1711 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 7: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:43.722396  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:43.723178  1562 raft_consensus.cc:2376] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9. Re-sending same reply.
I20260709 13:02:43.675100  5395 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.701503  5187 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:43.678561  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:43.728559  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:43.718713  5384 consensus_meta.cc:287] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.651s	user 0.007s	sys 0.009s
I20260709 13:02:43.675094  5007 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:43.731134  5144 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:43.732097  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:43.732349  1683 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:43.725086  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:43.754863  2300 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:43.755214  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:43.734403  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:43.737937  1564 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
W20260709 13:02:43.739456  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 53603us
I20260709 13:02:43.757313  2296 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:43.740213  1567 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.675733  5269 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:02:43.765101  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:43.743005  1561 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:43.762456  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:43.769675  1564 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:43.772035  1635 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:43.661280  5267 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:02:43.776657  1569 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:43.726353  4960 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.689411  5404 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
W20260709 13:02:43.777971  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 87064.3us
W20260709 13:02:43.784864  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:43.785589  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.785805  5415 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:43.738422  5248 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:43.794224  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:43.795598  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:43.777854  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:43.781780  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:43.796818  1713 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:43.743768  5187 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:43.798496  1562 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:43.757078  5370 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:43.812956  1718 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:43.811614  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:43.783133  5263 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:43.819188  5415 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11.
I20260709 13:02:43.820104  5364 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:43.820724  5364 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.821174  5364 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.781885  5292 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:43.765616  5384 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:43.826372  5384 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:43.802603  5353 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:43.827353  5353 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:43.827884  5353 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.802515  4939 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:02:43.829887  1712 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:02:43.808652  5379 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:02:43.808879  5187 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:43.791484  5248 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:43.803886  5391 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:43.835423  5391 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:43.835975  5391 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:43.836717  5370 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:43.796856  5243 consensus_queue.cc:1059] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:02:43.840365  5243 raft_consensus.cc:3055] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:43.840510  1717 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:43.840868  5243 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:43.847446  4831 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:43.848188  4831 raft_consensus.cc:697] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:43.861527  5423 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.862145  5423 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:43.861542  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 57906.2us
I20260709 13:02:43.849357  4831 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:43.894387  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 82876.7us
I20260709 13:02:43.733677  5301 consensus_queue.cc:1059] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:02:43.744498  5395 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:43.842077  5243 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:43.909062  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 111788us
I20260709 13:02:43.911001  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:43.920993  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:43.792728  4960 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:43.929468  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:43.913301  5007 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:43.916828  5269 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:43.916965  4544 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:43.943686  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 205572us
W20260709 13:02:43.948487  1534 leader_election.cc:336] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.632s (SENT)
I20260709 13:02:43.953863  5144 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:43.954437  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:43.955339  1566 raft_consensus.cc:2410] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:43.957345  5395 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:43.931499  5387 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.252s
I20260709 13:02:43.941532  5352 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:43.918251  5349 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:02:43.980871  5385 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:43.938665  5243 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:43.991111  5404 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:43.994481  5370 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:43.995141  5425 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.950008  5294 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:44.008939  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 60499.7us
W20260709 13:02:44.060429  1534 leader_election.cc:336] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.715s (SENT)
W20260709 13:02:44.061333  1534 leader_election.cc:336] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.733s (SENT)
I20260709 13:02:44.061683  1534 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:44.008579  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 224943us
I20260709 13:02:44.063112  5007 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:44.012382  1567 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:44.048137  5248 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:43.941516  4939 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:44.014371   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:44.015264  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 60917.6us
I20260709 13:02:44.015811  5242 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:44.036275  5352 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:02:44.076757  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 268886us
I20260709 13:02:44.068197  5404 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:44.106204  1681 leader_election.cc:400] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11.
I20260709 13:02:44.106696  1681 leader_election.cc:403] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 13:02:44.111095  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 329403us
I20260709 13:02:44.091356  5248 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:44.088413  5387 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:44.070818  5294 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:44.076781  5372 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 13:02:44.119654  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.121031  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:44.122874  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:44.127247  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.105399  5412 raft_consensus.cc:3055] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:44.128016  1565 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:44.068630  5425 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.130607  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.133733  2302 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:44.133999  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.136500  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.137781  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.139180  1568 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 10. Ops: []
I20260709 13:02:44.139259  5352 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.140841  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.085421  4939 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.134193  1561 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:44.112497  5365 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11.
W20260709 13:02:44.154016  5433 log.cc:927] Time spent T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 13:02:44.157883  5432 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:02:44.121044  5198 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.127712  5387 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:44.174958  5364 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.176764  5387 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:44.159328  5198 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:44.177716  5425 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.179354  5198 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:44.180253  5439 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.180836  5439 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.130401  5412 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:44.184682  1564 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:44.189023  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.196203  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.183651  5412 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:44.201802  5404 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:44.203243  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:44.207633  5364 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.211526  1567 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:02:44.212435  1568 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:44.214854  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:44.215734  1565 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:44.216208  1710 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:44.220479  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.221935  1567 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:44.230904  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.237397  1709 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:02:44.213856  5349 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.258765  1565 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:44.219444  5439 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.235409  5369 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:44.282096  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:44.218611  5364 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:02:44.286000  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.219614  5425 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:02:44.292393  1560 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:44.270555  5242 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.312170  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:44.316342  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.319857  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.321099  2302 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:44.326756  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2064) took 1124 ms. Trace:
I20260709 13:02:44.327384  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:44.328492  2300 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:44.331038  5335 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.331732  5335 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.334177  5335 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.336856  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:44.327600  1644 rpcz_store.cc:276] 0709 13:02:43.202346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.204850 (+  2504us) service_pool.cc:224] Handling call
0709 13:02:43.205222 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:43.566467 (+361245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.326016 (+759549us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b540032c3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:44.326618 (+   602us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":828596480}
I20260709 13:02:44.342931  5425 raft_consensus.cc:3055] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:44.343592  5425 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:44.344719  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:44.344974  5425 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.347167  5272 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:44.347287  5417 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:44.342651  5445 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:44.350449  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:44.351897  1534 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:44.352877  5411 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:44.354274  5411 consensus_queue.cc:1059] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:44.355621  5417 raft_consensus.cc:3055] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:44.356079  5417 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:44.357228  5417 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.359337  5417 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:44.363046  5435 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:44.334803  5335 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:44.364845  5335 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:44.367369  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.368428  2303 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:44.372809  1534 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:44.376078  5353 consensus_meta.cc:287] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.548s	user 0.012s	sys 0.004s
I20260709 13:02:44.377316  5353 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:44.381383  5353 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:44.367914  5335 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:44.271633  5349 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.391212  5242 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.393154  5349 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.313755  5436 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.394183  5242 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:44.394326  5436 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.396404  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.396950  5436 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.397490  2305 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 9: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:44.319921  1560 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:44.399016  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.402904  5417 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:44.405730  5391 consensus_meta.cc:287] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.569s	user 0.014s	sys 0.000s
I20260709 13:02:44.406797  5335 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.407580  5335 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.413985  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.415362  2305 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:44.415903  5436 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.415901  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:44.416754  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:44.416689  5436 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.418227  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:44.421553  5436 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.421856  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.424118  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:44.425109  2305 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:44.425570  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.426275  2302 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:44.431164  2301 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:02:44.431816  2301 pending_rounds.cc:85] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 6
I20260709 13:02:44.432126  2301 pending_rounds.cc:107] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:02:44.432428  2301 raft_consensus.cc:2889] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:44.434657  2303 raft_consensus.cc:1240] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:02:44.434695  5144 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:44.439767  5144 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:44.415452  1644 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:44.441196  5144 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:44.456192  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:44.457908  1637 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 10: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:44.459779  5379 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.460408  5379 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.463189  5131 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:44.460426  5435 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:44.465523  5435 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.462702  5379 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.469889  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:44.470819  1563 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:44.472234  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.416636  5335 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.475492  5435 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.406431  5391 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.486013  5391 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.487402  5372 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.488324  5372 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.509025  5336 raft_consensus.cc:3055] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:02:44.509692  5336 raft_consensus.cc:740] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:44.518134  5385 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.525446  5243 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:44.526081  5243 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.531499  1566 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:02:44.534122  1637 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:44.536755  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.538067  5243 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.525012  5341 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.542189  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:44.542460  5341 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.544720  5341 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.546815  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:44.547232  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.548409  2305 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:44.548781  1607 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:44.554028  5442 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 13:02:44.564601  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:44.565852  5294 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:44.570448  5412 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:44.574193  5420 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.575040  5420 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.576313  4403 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:44.576727  4403 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:44.577486  4403 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.578047  5420 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.578464  4403 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:44.538335  5369 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.581662  5369 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:44.442198  5187 consensus_meta.cc:287] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.609s	user 0.014s	sys 0.001s
I20260709 13:02:44.589012  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:44.591305  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.600415  5379 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:44.600562  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:44.601037  4939 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.601109  5379 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.602649  5294 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:44.603171  5294 raft_consensus.cc:697] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:44.603569  4544 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:44.604297  4544 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:44.604081  5294 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.617832  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:44.346817  5425 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:44.609656  1569 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:02:44.607668  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.622718  1681 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:44.632700  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:44.635061  5423 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.635567  5131 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:44.636111  5131 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:44.636993  5131 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.640208  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.640954  1561 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:44.641120  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2000) took 1483 ms. Trace:
I20260709 13:02:44.614640  4960 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:44.641492  1713 rpcz_store.cc:276] 0709 13:02:43.158161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.180546 (+ 22385us) service_pool.cc:224] Handling call
0709 13:02:43.230991 (+ 50445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.807221 (+576230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.640880 (+833659us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b5400337cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:44.641063 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1300946688}
I20260709 13:02:44.644241  1569 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:44.631662  1681 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:44.650657  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 177097us
I20260709 13:02:44.653648  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:44.657258  1681 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:44.665598  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.666323  2299 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:44.667492  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.671068  2301 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 9: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:44.672416  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 299709us
I20260709 13:02:44.673808  1534 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:44.675426  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:44.676199  1710 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 9: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:44.376209  2300 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:44.690503  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:44.691213  5415 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:44.692189  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:44.692973  1716 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:44.693240  5415 raft_consensus.cc:3055] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:44.693665  5415 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:44.695230  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:44.695555  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:44.696067  1712 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:44.681624  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:44.683054  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:44.698999  1643 raft_consensus.cc:2410] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:02:44.681671  4403 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:44.699626  4403 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.683611  4048 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:02:44.684051  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 95175.5us
I20260709 13:02:44.682657  1640 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:44.689189  1560 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:02:44.704284  1710 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:44.705526  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:44.722797  1710 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:02:44.715758  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 58582.2us
I20260709 13:02:44.717684  1683 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:44.728595  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:44.729974  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:44.731118  1715 raft_consensus.cc:2393] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 10: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:44.732072  4403 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:44.736711  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.718609  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:44.738903  1718 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:44.720140  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:44.739569  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.727525  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 53782.5us
I20260709 13:02:44.740824  1534 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:44.728368  1638 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:44.742352  1713 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:44.743122  1683 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:44.744236  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:44.744737  1709 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 10: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:44.744976  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:44.766139  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:44.759388  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:44.766489  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.771687  2299 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:44.765872  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:44.749250  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 47685.2us
W20260709 13:02:44.753080  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.773350  1712 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:44.674839  4960 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:44.675475  5436 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.695106  5385 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:44.624275  5187 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.778383  1639 raft_consensus.cc:2410] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:44.790709  2302 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:44.782788  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:44.795109  1641 raft_consensus.cc:2368] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10.
I20260709 13:02:44.781610  1534 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:44.695994  5438 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:44.811373  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.799475  2296 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:44.815894  1566 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:44.804762  1716 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:44.782848  1563 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:44.800016  4403 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:02:44.775269  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.812172  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:44.818790  1709 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:44.812562  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:44.768507  5301 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:44.808228  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:44.842255  2303 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:44.741392  5341 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:44.538888  5336 consensus_queue.cc:260] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.694423  5415 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:44.711606  5445 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:44.741294  5411 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:44.823959  1563 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:44.824944  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:44.711192  5439 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:44.858534  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:44.831321  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:45.099342  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:44.837553  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:44.840284  1709 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:44.684602  5365 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.097s
I20260709 13:02:45.109908  5187 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:44.747922  5364 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:45.110985  1715 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:44.748242  5395 raft_consensus.cc:3055] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:44.848169  1561 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:44.697184  5370 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.118s
I20260709 13:02:44.857548  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:44.776091  5447 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:44.782138  5266 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:02:44.859393  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:45.012887  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:44.858603  2302 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:45.147859  1566 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 9: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:45.150462  1566 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:02:45.152912  2304 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.024s	sys 0.047s
W20260709 13:02:45.153406  2304 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.024s	sys 0.047s
I20260709 13:02:45.154899  1566 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:02:45.158035  1566 pending_rounds.cc:85] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 4
I20260709 13:02:44.985530  1642 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:02:44.861619  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.812739  5335 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:44.826831  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:44.813126  5412 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:02:45.096421  1638 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:45.154772  1609 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:44.860378  2298 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:44.834208  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:45.100157  1567 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:45.164364  1566 pending_rounds.cc:107] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:02:45.164958  1566 raft_consensus.cc:2889] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:45.174269  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.174748  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.101969  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:45.175474  1560 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:02:45.103403  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:45.104949  2301 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:45.105319  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:45.107021  5426 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:45.141700  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:45.177160  1715 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:45.178035  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:45.178846  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:45.101682  2303 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:44.843758  5379 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:45.145910  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:45.179808  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:44.876322  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:45.017740  1636 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:44.815063  5436 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.181257  1715 raft_consensus.cc:2410] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:45.112092  5292 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.193868  4403 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:45.194370  4403 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:45.143421  1711 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.923s	user 0.003s	sys 0.035s
I20260709 13:02:45.194738  4403 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:45.195628  1711 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.003s	sys 0.035s
I20260709 13:02:44.860018  5337 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.304s
I20260709 13:02:45.010128  4960 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.153s
I20260709 13:02:45.204731  1533 leader_election.cc:400] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10.
I20260709 13:02:44.849419  5438 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.139343  5395 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:45.205071  1533 leader_election.cc:403] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:02:45.147473  5365 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.170398  5420 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:45.235600  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 387692us
I20260709 13:02:45.216132  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:45.214648  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2125) took 1150 ms. Trace:
I20260709 13:02:45.247754  1716 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
W20260709 13:02:45.233639  1644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.002s	sys 0.041s
W20260709 13:02:45.250296  1644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.002s	sys 0.042s
I20260709 13:02:45.208930  1715 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:45.252708  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:45.247684  1711 rpcz_store.cc:276] 0709 13:02:44.064636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.072704 (+  8068us) service_pool.cc:224] Handling call
0709 13:02:44.089323 (+ 16619us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:44.100943 (+ 11620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.100976 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:44.106367 (+  5391us) log.cc:844] Serialized 52 byte log entry
0709 13:02:44.192130 (+ 85763us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:45.201561 (+1009431us) log.cc:844] Serialized 14 byte log entry
0709 13:02:45.214326 (+ 12765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.214394 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.214434 (+    40us) raft_consensus.cc:1685] finished
0709 13:02:45.214445 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.214579 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23280,"raft.queue_time_us":27818,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":6615,"thread_start_us":1036173,"threads_started":2,"wal-append.queue_time_us":162433}
I20260709 13:02:45.118815  5439 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.142711  5364 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.213162  5426 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:45.218765  5412 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10.
I20260709 13:02:45.200766  5267 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:45.236099  5369 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:45.199409  5435 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:02:45.203402  5292 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:45.300532  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.008s	sys 0.016s
W20260709 13:02:45.300918  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.008s	sys 0.016s
I20260709 13:02:45.277730  4403 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.288767  1711 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:45.283756  5450 log.cc:927] Time spent T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 13:02:45.266903  5452 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:02:45.303637  1635 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.025s	sys 0.026s
W20260709 13:02:45.304100  1635 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.025s	sys 0.026s
I20260709 13:02:45.284333  5385 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:02:45.301662  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.173051  5447 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.319595  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:45.323055  1718 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:45.246521  5365 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:45.256621  5420 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
W20260709 13:02:45.296703  5294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.017s	sys 0.013s
W20260709 13:02:45.329862  1568 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.011s	user 0.007s	sys 0.020s
W20260709 13:02:45.335141  1568 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.007s	sys 0.020s
W20260709 13:02:45.309464  5404 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.000s	sys 0.015s
I20260709 13:02:45.336030  4403 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:45.336586  1709 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:45.289041  5267 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.296613  1462 catalog_manager.cc:5697] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:45.316373  5385 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:45.344345  1562 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.007s	sys 0.007s
W20260709 13:02:45.385097  1683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.001s	sys 0.037s
W20260709 13:02:45.390492  1562 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.007s	sys 0.007s
I20260709 13:02:45.355332  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:45.354671  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:45.370460  2297 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.008s	sys 0.009s
I20260709 13:02:45.324136  5411 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
W20260709 13:02:45.370620  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.340113  5404 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.001s	sys 0.015s
W20260709 13:02:45.372036  2305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.018s	sys 0.024s
W20260709 13:02:45.405386  2305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.018s	sys 0.024s
I20260709 13:02:45.383124  2300 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:45.384516  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2124) took 1320 ms. Trace:
W20260709 13:02:45.333760  5294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.019s	sys 0.015s
I20260709 13:02:45.406211  1714 rpcz_store.cc:276] 0709 13:02:44.064311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.066122 (+  1811us) service_pool.cc:224] Handling call
0709 13:02:44.066513 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.066935 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.066969 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.067459 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:02:44.285457 (+217998us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:44.285498 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.285575 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.384301 (+1098726us) raft_consensus.cc:1685] finished
0709 13:02:45.384322 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.384452 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9785,"raft.queue_time_us":1156,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":6327,"thread_start_us":217613,"threads_started":1,"wal-append.queue_time_us":708002}
W20260709 13:02:45.392102  2297 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.008s	sys 0.009s
W20260709 13:02:45.390765  1683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.001s	sys 0.037s
W20260709 13:02:45.409469  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 665276us
I20260709 13:02:45.422011  1714 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:45.315734  5452 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:45.411312  1683 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:45.307261  5395 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.428462  1639 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:45.396236  5385 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:45.438621  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:45.436225  1714 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:02:45.411526  5337 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:45.479344  1640 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:45.481539  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1063) took 1368 ms. Trace:
I20260709 13:02:45.484894  2298 rpcz_store.cc:276] 0709 13:02:44.113213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.113371 (+   158us) service_pool.cc:224] Handling call
0709 13:02:44.128866 (+ 15495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.169396 (+1040530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.481193 (+311797us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b54003a2354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:45.481447 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1671848064}
I20260709 13:02:45.477463  5335 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:45.477813  5341 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:45.512418  5197 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.513546  5292 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:45.545603  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.556842  1462 catalog_manager.cc:5697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:45.515120  5267 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:45.515766  5007 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.488934  5384 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:45.489530  5395 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:45.520601  1565 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.010s	sys 0.026s
W20260709 13:02:45.574257  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 469036us
I20260709 13:02:45.597831  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 887) took 1933 ms. Trace:
W20260709 13:02:45.599438  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.603165  1565 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.010s	sys 0.026s
I20260709 13:02:45.509042  5438 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:45.599881  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 49642.8us
I20260709 13:02:45.497876  5337 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.519273  5335 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.603436  2304 rpcz_store.cc:276] 0709 13:02:43.664018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.665081 (+  1063us) service_pool.cc:224] Handling call
0709 13:02:43.671966 (+  6885us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:43.707930 (+ 35964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:43.707972 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.323862 (+615890us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b2c00047a38. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:02:44.324488 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.153850 (+829362us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:45.153910 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.154005 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.542606 (+388601us) raft_consensus.cc:1685] finished
0709 13:02:45.542633 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.597737 (+ 55104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9148,"raft.queue_time_us":37441,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":11085,"spinlock_wait_cycles":10912512,"thread_start_us":828851,"threads_started":1,"wal-append.queue_time_us":1065580}
I20260709 13:02:45.603801  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:45.492764  1683 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:45.616971  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 788454us
I20260709 13:02:45.540782  5423 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:45.510255  5187 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 13:02:45.605535  5439 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:45.593075  5301 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.011s	sys 0.026s
W20260709 13:02:45.617442  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 124824us
I20260709 13:02:45.640950  1683 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:45.642410  1683 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:45.598987  5364 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:45.607448  5454 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.314s	user 0.000s	sys 0.001s
W20260709 13:02:45.620911  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 427661us
W20260709 13:02:45.614408  2296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.011s	sys 0.028s
W20260709 13:02:45.693559  2296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.012s	sys 0.029s
I20260709 13:02:45.639619  4403 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:45.610857  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:45.618064  5335 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:45.574585  5197 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.600318  5007 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.614619  5267 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:45.649109  5294 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:02:45.699404  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.696506  5439 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:45.714272  1683 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:45.715487  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 70392.6us
I20260709 13:02:45.769961  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:45.642864  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 438279us
I20260709 13:02:45.620517  5365 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:45.643397  5301 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.011s	sys 0.027s
I20260709 13:02:45.620085  5438 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:45.748852  5426 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.735536  2298 raft_consensus.cc:3055] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:45.736413  5266 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:02:45.780198  5131 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.134s	user 0.000s	sys 0.063s
I20260709 13:02:45.727334  5372 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:45.740823  4403 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:45.769397  5439 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.680469  5423 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:45.747846  5458 log.cc:927] Time spent T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:02:45.863926  5474 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:45.864477  5474 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:45.864998  5474 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:45.865129  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2473) took 1760 ms. Trace:
I20260709 13:02:45.748970  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 889) took 1082 ms. Trace:
I20260709 13:02:45.619326  5144 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:45.867164  2297 rpcz_store.cc:276] 0709 13:02:44.667086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.680319 (+ 13233us) service_pool.cc:224] Handling call
0709 13:02:44.680765 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.711029 (+ 30264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.711079 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.777436 (+ 66357us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.409330 (+631894us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:45.409373 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.409446 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.748694 (+339248us) raft_consensus.cc:1685] finished
0709 13:02:45.748715 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.748891 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":166,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":34503,"thread_start_us":554464,"threads_started":1,"wal-append.queue_time_us":876325}
W20260709 13:02:45.869733  5455 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:02:45.871340  5197 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:45.785503  5301 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:45.764670  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.873018  1532 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:45.872651  5301 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:45.706102  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.764684  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:45.875488  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.696826  5341 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:45.772958  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 58829.4us
I20260709 13:02:45.876654  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:45.728164  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:45.750560  5420 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:45.789510  1681 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.043s	user 0.004s	sys 0.030s
W20260709 13:02:45.880890  1681 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.135s	user 0.004s	sys 0.031s
I20260709 13:02:45.786825  5294 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:45.765616  5267 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.743075  5461 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:45.853262  5439 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:45.754207  5379 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:02:45.924747  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:45.820683  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 667) took 1142 ms. Trace:
I20260709 13:02:45.941263  1569 rpcz_store.cc:276] 0709 13:02:44.678296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.718832 (+ 40536us) service_pool.cc:224] Handling call
0709 13:02:44.777120 (+ 58288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.177292 (+400172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.177322 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.177372 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:45.177402 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.820305 (+642903us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b54001cf054. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:45.820379 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.820590 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":919864192}
I20260709 13:02:45.824728  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:45.943044  1718 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:45.781953  5369 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:02:45.853658  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 83798.1us
I20260709 13:02:45.948159  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:45.866806  5457 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.254s	user 0.000s	sys 0.000s
I20260709 13:02:45.872715  5476 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:45.953624  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2482) took 1488 ms. Trace:
I20260709 13:02:45.954208  1644 rpcz_store.cc:276] 0709 13:02:44.465654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.465839 (+   185us) service_pool.cc:224] Handling call
0709 13:02:44.466204 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.466567 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.466594 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.467130 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.251056 (+783926us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:02:45.251117 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.251206 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.953326 (+702120us) raft_consensus.cc:1685] finished
0709 13:02:45.953359 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.953540 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":693,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":118,"thread_start_us":783452,"threads_started":1,"wal-append.queue_time_us":1050323}
I20260709 13:02:45.874774  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2126) took 1656 ms. Trace:
I20260709 13:02:45.956712  1568 rpcz_store.cc:276] 0709 13:02:44.218196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.235047 (+ 16851us) service_pool.cc:224] Handling call
0709 13:02:44.252671 (+ 17624us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:44.253231 (+   560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.253273 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:44.271641 (+ 18368us) log.cc:844] Serialized 52 byte log entry
0709 13:02:45.339453 (+1067812us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:45.339496 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.339607 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.874528 (+534921us) raft_consensus.cc:1685] finished
0709 13:02:45.874550 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.874700 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23162,"raft.queue_time_us":133,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":15548,"thread_start_us":1020190,"threads_started":1,"wal-append.queue_time_us":1350916}
I20260709 13:02:45.877607  2304 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:45.610268  5465 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.158s
W20260709 13:02:45.806445  5131 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.160s	user 0.000s	sys 0.064s
I20260709 13:02:45.964010  5197 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:45.874341  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:45.787130  2298 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:02:45.885259  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.886003  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:45.888518  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:45.968370  1563 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 7: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:45.731235  1462 catalog_manager.cc:5697] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 2 to 6, now has a pending config: VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193), VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194), VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195), VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 6 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:45.892940  5438 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:45.899459  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:45.988771  1567 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:45.805114  5469 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:45.810181  4403 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:45.902777  5462 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:45.903601  5364 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:45.904098  5463 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 13:02:45.905093  5447 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:46.017679  1637 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.009s	sys 0.014s
I20260709 13:02:45.805896  5426 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.906848  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:45.903662  5468 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:02:45.908552  1681 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:45.946516  5411 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.912484  5461 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.916172  5423 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:45.802037  5384 consensus_queue.cc:1059] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 13:02:45.878036  5341 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.865592  1635 rpcz_store.cc:276] 0709 13:02:44.104709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.104878 (+   169us) service_pool.cc:224] Handling call
0709 13:02:44.105353 (+   475us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:44.105783 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.105831 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:44.106568 (+   737us) log.cc:844] Serialized 52 byte log entry
0709 13:02:44.529611 (+423043us) spinlock_profiling.cc:243] Waited 8.96 ms on lock 0x7b18004d6560. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:02:45.304521 (+774910us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:45.304575 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.304673 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.864685 (+560012us) raft_consensus.cc:1685] finished
0709 13:02:45.864802 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.865064 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17426,"raft.queue_time_us":14713,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":2366,"spinlock_wait_cycles":20614912,"thread_start_us":774686,"threads_started":1,"wal-append.queue_time_us":1488125}
W20260709 13:02:45.893738  1638 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.004s	sys 0.027s
W20260709 13:02:46.034960  1638 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.004s	sys 0.027s
I20260709 13:02:45.925163  5266 consensus_queue.cc:1059] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:02:45.939123  5336 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:45.976403  4960 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.977754  5131 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:45.980774  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:46.038303  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 129906us
I20260709 13:02:45.980113  1462 catalog_manager.cc:5697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 3 to 6, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 6 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:45.939433  5395 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.932243  5007 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:45.974076  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 756) took 1232 ms. Trace:
I20260709 13:02:46.044157  1713 rpcz_store.cc:276] 0709 13:02:44.741947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.743424 (+  1477us) service_pool.cc:224] Handling call
0709 13:02:44.743770 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.256702 (+512932us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.256733 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.256777 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:45.256811 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.973820 (+717009us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b54001356d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:45.973865 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.974017 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1179187968}
W20260709 13:02:45.991997  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.018s	user 0.018s	sys 0.013s
W20260709 13:02:46.046043  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.073s	user 0.018s	sys 0.013s
I20260709 13:02:45.971637  5369 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:46.006273  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.012094  1643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.017s	sys 0.021s
W20260709 13:02:46.053288  1643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.105s	user 0.017s	sys 0.022s
I20260709 13:02:45.979463  5335 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:45.900951  5452 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:45.987097  5473 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 13:02:46.015092  4403 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:46.016125  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 141891us
I20260709 13:02:46.039166  5384 consensus_queue.cc:1243] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:02:45.920719  5460 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.549s	user 0.003s	sys 0.007s
I20260709 13:02:45.900527  5365 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:46.019809  1637 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.009s	sys 0.014s
I20260709 13:02:46.069505  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2085) took 1035 ms. Trace:
I20260709 13:02:46.018007  5364 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:46.021003  2302 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:45.965205  5197 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:46.020725  5435 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:46.020457  5292 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:46.042265  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.901582  5420 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:46.055233  1462 catalog_manager.cc:5697] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:46.037187  5341 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:46.049319  5243 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:46.067826  1532 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:46.091732  1532 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:46.030736  5447 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:02:46.095151  5447 raft_consensus.cc:3055] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:46.095702  5447 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:46.096846  5447 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.099593  5412 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:46.100590  5412 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:46.101086  5412 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.101604  5412 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:46.105403  1532 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:46.107060  5423 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:46.107669  5423 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.108156  5423 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:46.112931  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:46.098637  5447 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:46.077899  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.014s	sys 0.040s
I20260709 13:02:46.070006  1637 rpcz_store.cc:276] 0709 13:02:45.034017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.070879 (+ 36862us) service_pool.cc:224] Handling call
0709 13:02:45.101178 (+ 30299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.101391 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:45.488732 (+387341us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.069053 (+580321us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.069123 (+    70us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:46.069177 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.069271 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.069432 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":686585,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"thread_start_us":926116,"threads_started":2}
W20260709 13:02:46.144297  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.014s	sys 0.040s
W20260709 13:02:46.144814  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 760974us
I20260709 13:02:46.146130  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:46.021900  5462 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:46.032939  5411 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.160645  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 666) took 1483 ms. Trace:
I20260709 13:02:46.161142  1565 rpcz_store.cc:276] 0709 13:02:44.677350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.677918 (+   568us) service_pool.cc:224] Handling call
0709 13:02:44.688620 (+ 10702us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.748710 (+ 60090us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.748755 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.777365 (+ 28610us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.603682 (+826317us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:46.044640 (+440958us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.044938 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.045052 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.160402 (+115350us) raft_consensus.cc:1685] finished
0709 13:02:46.160430 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.160577 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59599,"raft.queue_time_us":60543,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":48107,"thread_start_us":1164670,"threads_started":2,"wal-append.queue_time_us":928414}
W20260709 13:02:46.163862  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 380164us
I20260709 13:02:46.165272  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:46.175616  5465 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:46.135439  5420 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.073226  5266 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:46.064447  5476 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:46.166531  1533 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:46.048092  4960 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:46.052816  5395 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.088084  2304 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
W20260709 13:02:46.232877  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 66441us
I20260709 13:02:46.035281  5294 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.135888  5243 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.048520  5131 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:46.057245  5469 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.139s
I20260709 13:02:46.268577  1713 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:46.194253  5420 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:46.203349  5266 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:46.202706  5435 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:46.178711  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:46.244700  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.283236  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 104625us
I20260709 13:02:46.149034  5292 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.089910  5365 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.261520  5243 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:46.294494  5482 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:02:46.298681  1610 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.000s	sys 0.016s
W20260709 13:02:46.299293  1610 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.000s	sys 0.016s
I20260709 13:02:46.300638  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:46.314778  5480 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:02:46.316526  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 827) took 1006 ms. Trace:
I20260709 13:02:46.316922  1638 rpcz_store.cc:276] 0709 13:02:45.309833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.310059 (+   226us) service_pool.cc:224] Handling call
0709 13:02:45.310560 (+   501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.311097 (+   537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.311145 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.313635 (+  2490us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.035477 (+721842us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:46.035534 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.035627 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.316298 (+280671us) raft_consensus.cc:1685] finished
0709 13:02:46.316319 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.316465 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":161,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":721375,"threads_started":1,"wal-append.queue_time_us":728986}
I20260709 13:02:46.317718  4960 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:46.318976  5144 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:46.320245  5144 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.320791  5144 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:46.309506  5425 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.661s	user 0.008s	sys 0.008s
I20260709 13:02:46.325075  5365 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:46.325692  5369 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:46.314819  5335 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:46.327956  5144 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:46.281260  5266 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:46.278321  5435 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.333586  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2008) took 1513 ms. Trace:
I20260709 13:02:46.333269  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 898) took 1555 ms. Trace:
I20260709 13:02:46.341491  2296 rpcz_store.cc:276] 0709 13:02:44.778155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.843986 (+ 65831us) service_pool.cc:224] Handling call
0709 13:02:44.851594 (+  7608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.694019 (+842425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.694051 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.698667 (+  4616us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.048832 (+350165us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:46.048869 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.048938 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.332950 (+284012us) raft_consensus.cc:1685] finished
0709 13:02:46.332983 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.333188 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":301658,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":3450,"thread_start_us":998272,"threads_started":2,"wal-append.queue_time_us":590564}
I20260709 13:02:46.341079  1716 rpcz_store.cc:276] 0709 13:02:44.820429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.301015 (+480586us) service_pool.cc:224] Handling call
0709 13:02:45.326461 (+ 25446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.326737 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:45.338951 (+ 12214us) log.cc:844] Serialized 14 byte log entry
0709 13:02:45.972355 (+633404us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b180040ba00. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a 00007fd7a1caba2a
0709 13:02:46.307301 (+334946us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.307369 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:46.307421 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.333408 (+ 25987us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.333532 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11998,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":15007,"spinlock_wait_cycles":53591936,"thread_start_us":334699,"threads_started":1}
I20260709 13:02:46.338425  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 753) took 1661 ms. Trace:
I20260709 13:02:46.348420  1710 rpcz_store.cc:276] 0709 13:02:44.677070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.723591 (+ 46521us) service_pool.cc:224] Handling call
0709 13:02:44.723838 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.293001 (+569163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.293040 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.293090 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:45.293123 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.338145 (+1045022us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b54003dc2d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:46.338216 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.338365 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1308512000}
W20260709 13:02:46.358013  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 303101us
I20260709 13:02:46.289587  5466 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:02:46.365212  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 397144us
I20260709 13:02:46.298861  5438 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:46.237545  5476 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:46.295467  5423 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:46.373916  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:46.368402  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:02:46.413841  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 45540us
I20260709 13:02:46.403445  1462 catalog_manager.cc:5697] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 3 to 5, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 5 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.348085  5435 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:46.426728  1565 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:46.242688  5197 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.337141  5372 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:02:46.431761  1564 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:02:46.366611  4960 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:46.331622  5465 consensus_queue.cc:1059] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
W20260709 13:02:46.432524  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 316329us
I20260709 13:02:46.436810  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:46.373654  5335 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:46.436853  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:46.373337  5267 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:46.412832  5469 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:46.501583  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 124997us
I20260709 13:02:46.406577  5466 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:02:46.539575  5481 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.466s	user 0.001s	sys 0.001s
I20260709 13:02:46.594409  1715 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:46.464161  5485 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:02:46.597043  1718 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:46.493642  5509 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:46.486490  5467 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:02:46.531941  5197 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:46.480218  5461 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:46.525509  5364 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:46.615231  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 314679us
I20260709 13:02:46.406010  5438 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.570716  5426 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:46.591331  5411 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:46.602942  5395 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:46.454977  4960 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.449674  5412 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.563969  5335 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.609048  5301 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:46.606089  5469 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:46.629884  5462 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:46.635887  5269 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:46.704293  5489 log.cc:927] Time spent T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
I20260709 13:02:46.677731  5292 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:46.681332  5365 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.694149  5462 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.719538  5462 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:46.720459  5501 log.cc:927] Time spent T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:02:46.695272  5372 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.721220  5372 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.722231  5462 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:46.723420  5372 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:46.723838  5412 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:46.694453  2268 connection.cc:256] RPC call timeout handler was delayed by -1.2315521369999942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1507s for a response
W20260709 13:02:46.724625  2268 connection.cc:256] RPC call timeout handler was delayed by -1.1352521369999522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
W20260709 13:02:46.725023  2268 connection.cc:256] RPC call timeout handler was delayed by -1.1298521369999435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
W20260709 13:02:46.725685  2268 connection.cc:256] RPC call timeout handler was delayed by -1.0749521369999684s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
W20260709 13:02:46.726068  2268 connection.cc:256] RPC call timeout handler was delayed by -1.0263521369999467s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
I20260709 13:02:46.625921  5509 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.719370  5365 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:46.726418  2268 connection.cc:256] RPC call timeout handler was delayed by -1.0092521369999758s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1754s for a response
I20260709 13:02:46.677032  5301 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.729122  5301 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:46.729441  5365 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:46.730283  5438 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:46.645534  5404 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:46.665591  5461 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
W20260709 13:02:46.705466  5447 consensus_meta.cc:287] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.582s	user 0.013s	sys 0.000s
I20260709 13:02:46.733304  2296 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: [9.10-9.10]
W20260709 13:02:46.725795  5336 consensus_meta.cc:287] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.683s	user 0.001s	sys 0.008s
I20260709 13:02:46.740784  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:46.741987  2300 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:46.749356  2268 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:46.753427  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:46.757627  2268 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:46.765067  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:46.767953  2302 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:46.770298  1712 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:46.772408  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:46.772850  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:46.713634  5423 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:46.778223  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:46.775759  5439 consensus_meta.cc:287] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.805s	user 0.007s	sys 0.007s
I20260709 13:02:46.774644  5465 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:02:46.793993  5503 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 13:02:46.795626  5385 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.311s	user 0.013s	sys 0.001s
W20260709 13:02:46.793996  5495 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.250s	user 0.001s	sys 0.001s
I20260709 13:02:46.801873  5435 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:46.789517  5372 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:02:46.735598  5269 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:46.804845  5266 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.805317  5486 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:46.794791  5497 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:02:46.813993  5509 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:46.814118  5385 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:46.815049  5513 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:02:46.778569  5522 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:46.824290  5469 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:46.824636  5335 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:46.818884  5341 consensus_meta.cc:287] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.631s	user 0.010s	sys 0.000s
I20260709 13:02:46.826347  4960 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:46.830542  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2016) took 1800 ms (client timeout 1525 ms). Trace:
W20260709 13:02:46.834237  5243 consensus_meta.cc:287] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.507s	user 0.008s	sys 0.000s
I20260709 13:02:46.742662  5404 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.812148  5476 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:46.808234  5465 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:46.809166  5439 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.840330  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 888) took 2180 ms. Trace:
I20260709 13:02:46.856928  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:46.810123  5352 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:46.868517  1709 rpcz_store.cc:269] 0709 13:02:45.029952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.103147 (+ 73195us) service_pool.cc:224] Handling call
0709 13:02:46.830360 (+1727213us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b5400474854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:46.830469 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367961472}
I20260709 13:02:46.847514  5384 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:46.931066  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 910) took 1052 ms. Trace:
W20260709 13:02:46.862998  5474 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.997s	user 0.015s	sys 0.000s
W20260709 13:02:47.079917  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 301810us
I20260709 13:02:47.080724  2303 rpcz_store.cc:276] 0709 13:02:45.879112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.879282 (+   170us) service_pool.cc:224] Handling call
0709 13:02:45.914027 (+ 34745us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.914466 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.914502 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.915091 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.279497 (+364406us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:46.279577 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.279806 (+   229us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.930771 (+650965us) raft_consensus.cc:1685] finished
0709 13:02:46.930800 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.930990 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25696,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":41201,"thread_start_us":363874,"threads_started":1,"wal-append.queue_time_us":689866}
I20260709 13:02:47.086632  2268 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:47.081073  2305 rpcz_store.cc:276] 0709 13:02:44.659683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.659862 (+   179us) service_pool.cc:224] Handling call
0709 13:02:44.663641 (+  3779us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.664998 (+  1357us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:44.665030 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.665524 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.919848 (+1254324us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:46.839788 (+919940us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b18003ac4e0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1
0709 13:02:46.839938 (+   150us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:46.839973 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.840056 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.840098 (+    42us) raft_consensus.cc:1685] finished
0709 13:02:46.840110 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.840241 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":858,"raft.queue_time_us":1955,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":21905,"spinlock_wait_cycles":35602944,"thread_start_us":740033,"threads_started":1,"wal-append.queue_time_us":705822}
I20260709 13:02:46.862243  5486 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:46.860639  5301 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:47.088634  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:47.145816  1641 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
W20260709 13:02:47.137768  5420 consensus_meta.cc:287] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.853s	user 0.011s	sys 0.003s
I20260709 13:02:46.929950  5404 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.101835  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:46.941766  5435 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:47.198009  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 109443us
I20260709 13:02:47.198216  1709 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 9: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:47.060490  4960 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:47.016613  5507 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.934976  5269 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:47.194698  5364 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:47.088912  5266 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.137145  5439 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.117206  5335 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.084956  5465 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:47.196795  5420 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.090727  5536 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:46.892017  5243 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:47.199085  5535 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:47.371716  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:47.372440  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 171184us
I20260709 13:02:47.348503  5364 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:47.090134  5474 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.090072  5372 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.286s
I20260709 13:02:46.872026  5341 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.403945  5474 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.404357  5341 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.406088  5364 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:47.088325  5384 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.410604  5535 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:47.413139  5420 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.389839  5335 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:47.416002  5335 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:47.416815  1718 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:47.417860  5243 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:47.418306  5465 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.350104  4960 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:47.379169  5301 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:47.086752  5466 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:47.419452  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:47.397388  1709 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:47.398070  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:47.420075  1683 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.530s (SENT) [suppressed 12 similar messages]
I20260709 13:02:47.106665  5415 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:47.424664  2268 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:47.424371  1718 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:47.400784  5438 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:47.427801  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:47.428339  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:47.429415  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:47.430323  1718 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:47.437580  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:47.432840  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:47.439879  1718 raft_consensus.cc:2393] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 10: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:47.428108  5536 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:47.426775  5535 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:47.441854  1718 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:02:47.366940  5507 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.446195  1709 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:02:47.431345  5438 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.450579  1709 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:02:47.452670  1718 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:02:47.447719  5536 raft_consensus.cc:697] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:47.448763  5535 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:47.462541  1709 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 7 index: 9. (term mismatch)
I20260709 13:02:47.463186  1709 pending_rounds.cc:85] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 8
I20260709 13:02:47.463475  1709 pending_rounds.cc:107] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 6.9
I20260709 13:02:47.463765  1709 raft_consensus.cc:2889] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:47.460530  5384 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:47.476655  4960 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:47.484942  5507 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.492710  5535 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.500057  5465 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:47.491822  5536 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.517822  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2009) took 2678 ms. Trace:
I20260709 13:02:47.518440  1711 rpcz_store.cc:276] 0709 13:02:44.839398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.302462 (+463064us) service_pool.cc:224] Handling call
0709 13:02:45.324966 (+ 22504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.490198 (+165232us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:02:45.518076 (+ 27878us) log.cc:844] Serialized 14 byte log entry
0709 13:02:45.993668 (+475592us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.993720 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:45.993767 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.517418 (+1523651us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b5400474354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:47.517488 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.517705 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7714,"prepare.queue_time_us":30335,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":19241,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":7534,"spinlock_wait_cycles":379283200,"thread_start_us":474206,"threads_started":1,"wal-append.queue_time_us":774982,"wal-append.run_cpu_time_us":1274,"wal-append.run_wall_time_us":1017399}
I20260709 13:02:47.522214  5438 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.529507  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 828) took 2141 ms. Trace:
I20260709 13:02:47.530019  1641 rpcz_store.cc:276] 0709 13:02:45.388479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.388646 (+   167us) service_pool.cc:224] Handling call
0709 13:02:45.388984 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.194984 (+1806000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.529152 (+334168us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b5400461ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:47.529399 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4040197376}
W20260709 13:02:47.537410  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 105258us
W20260709 13:02:47.540313  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 120266us
I20260709 13:02:47.546278  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:47.547142  1711 raft_consensus.cc:2368] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9.
I20260709 13:02:47.548125  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:47.562270  1607 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.112s	user 0.007s	sys 0.030s
W20260709 13:02:47.581470  1607 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.131s	user 0.008s	sys 0.031s
I20260709 13:02:47.555365  5438 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.582150  5438 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.584844  5438 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.586803  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:47.606271  5391 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:47.610531  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:47.616134  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:47.617447  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:47.619892  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 758) took 2848 ms. Trace:
I20260709 13:02:47.620415  1717 rpcz_store.cc:276] 0709 13:02:44.771492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.771649 (+   157us) service_pool.cc:224] Handling call
0709 13:02:44.819982 (+ 48333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.293212 (+473230us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:45.767744 (+474532us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.474669 (+706925us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.674816 (+200147us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.674871 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:46.674932 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.619627 (+944695us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b5400339ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:47.619685 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.619827 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8600,"prepare.queue_time_us":8669,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":92,"raft.queue_time_us":407040,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":35566,"spinlock_wait_cycles":1087753600,"thread_start_us":1334205,"threads_started":4,"wal-append.queue_time_us":547266,"wal-append.run_cpu_time_us":2175,"wal-append.run_wall_time_us":1213402}
I20260709 13:02:47.626896  5545 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:47.633056  5545 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:47.634548  5545 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:47.643898  5547 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:47.644933  5547 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:47.642762  5548 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:47.647424  5548 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:47.647894  5548 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:47.660945  5550 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:47.661703  5550 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:47.650301  5546 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.663784  5546 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.671555  5546 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.672191  5546 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:47.672698  5546 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.675323  5545 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.675954  5545 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.678220  5545 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.678745  5545 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:47.682267  1608 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.441s	user 0.015s	sys 0.008s
W20260709 13:02:47.682776  1608 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.442s	user 0.015s	sys 0.009s
I20260709 13:02:47.682637  5550 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:47.681892  5545 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:47.685943  5550 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.687278  5546 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.687892  5546 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.688388  5546 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:47.686543  5550 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.690896  1569 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:47.702708  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2513) took 1215 ms. Trace:
I20260709 13:02:47.703274  1569 rpcz_store.cc:276] 0709 13:02:46.487278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.487455 (+   177us) service_pool.cc:224] Handling call
0709 13:02:47.690456 (+1203001us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.702291 (+ 11835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.702625 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:47.705785  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.706126  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:47.706678  1565 raft_consensus.cc:2410] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:47.707230  1569 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:47.709488  5548 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:47.713076  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.713778  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.714190  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.714946  1565 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:47.715128  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.696349  5550 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.716075  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.716166  2298 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.717233  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.717461  2298 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:47.717759  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.719090  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:47.709347  1534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.411s	user 0.008s	sys 0.008s
W20260709 13:02:47.720069  1534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.422s	user 0.009s	sys 0.009s
I20260709 13:02:47.694332  5545 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:47.720461  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:47.720782  5548 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:47.721099  1569 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:47.717872  1565 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:02:47.725381  5370 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.726070  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:47.725965  5370 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.726662  1565 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:47.727596  5536 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:47.728132  5536 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.728454  5370 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.728946  5370 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.729368  5370 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.730152  5536 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:47.731061  5535 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.731302  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:47.731549  5535 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.732756  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:47.733518  5535 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:47.697346  5546 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:47.734001  5535 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:47.734532  5535 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.736811  5535 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:47.725425  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:47.738001  5385 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.739152  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:47.739306  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:47.741187  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:47.741762  2297 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:47.742574  1534 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:47.745677  1713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2023) took 1633 ms (client timeout 1813 ms). Trace:
W20260709 13:02:47.746148  1713 rpcz_store.cc:269] 0709 13:02:46.112349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.112570 (+   221us) service_pool.cc:224] Handling call
0709 13:02:47.745501 (+1632931us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54003381d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:47.745616 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355731200}
I20260709 13:02:47.748764  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:47.749153  1569 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:47.750036  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:47.740067  5370 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.750860  2297 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:47.753533  5556 consensus_queue.cc:1059] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:47.754751  5556 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:47.741752  1565 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:47.755453  5556 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:47.750036  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.751282  5370 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:47.757285  5131 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:47.757505  5370 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:47.758093  5370 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:47.756173  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:47.759627  1565 raft_consensus.cc:2368] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9.
I20260709 13:02:47.760548  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.757586  1569 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 7.
I20260709 13:02:46.481828  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.763674  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.761197  1565 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:47.764349  1569 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:47.765086  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:47.766539  5470 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:02:47.767215  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:47.768364  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:47.768973  1565 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:47.765292  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:47.775667  1569 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:47.774318  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 906) took 2060 ms. Trace:
I20260709 13:02:47.779202  2298 rpcz_store.cc:276] 0709 13:02:45.705812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.705945 (+   133us) service_pool.cc:224] Handling call
0709 13:02:45.735383 (+ 29438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:47.715790 (+1980407us) spinlock_profiling.cc:243] Waited 99.7 ms on lock 0x7b5400644720. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c1f06f 00007fd7a1c97f60 00007fd7a1c997de 00007fd7a1ca3b49 00007fd7a1ca4416 00007fd7a1ca0d8f 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62
0709 13:02:47.766269 (+ 50479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:47.766298 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:47.766337 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:47.766369 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.766442 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.766558 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"spinlock_wait_cycles":229342464}
I20260709 13:02:47.786073  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:47.791917  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:47.793263  2298 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:47.792730  2297 raft_consensus.cc:2410] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:02:47.806078  1569 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:47.812450  5144 raft_consensus.cc:3055] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:47.813020  5144 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:47.813815  1565 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:02:47.813886  5144 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.814592  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.815165  1565 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 6.
I20260709 13:02:47.816111  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.816632  1565 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:02:47.817190  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.817729  1565 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:47.818287  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:47.818879  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.819500  5144 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:47.819633  1569 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:47.820701  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:47.821792  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.822434  1569 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:47.823300  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.823904  1569 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:47.824750  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.825343  1569 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:02:47.826327  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.789577  5370 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:47.818670  5464 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 2.447s	user 0.014s	sys 0.017s
W20260709 13:02:47.830624  1533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.553s	user 0.000s	sys 0.023s
I20260709 13:02:47.840358  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:47.840744  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.841089  2297 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:47.842031  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:47.842896  5337 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:47.843451  2297 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:02:47.844899  5387 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:47.845522  5387 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:47.845978  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2076) took 3112 ms. Trace:
I20260709 13:02:47.826668  5486 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.846419  1643 rpcz_store.cc:276] 0709 13:02:44.733718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.735276 (+  1558us) service_pool.cc:224] Handling call
0709 13:02:44.735732 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.769808 (+ 34076us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.769845 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.789979 (+ 20134us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.053701 (+1263722us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:46.053743 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.053809 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.845752 (+1791943us) raft_consensus.cc:1685] finished
0709 13:02:47.845776 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.845919 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32869,"raft.queue_time_us":41520,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":78,"thread_start_us":1105796,"threads_started":1,"wal-append.queue_time_us":1441183}
I20260709 13:02:47.848198  5370 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.851198  5387 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:47.851545  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2020) took 2647 ms. Trace:
I20260709 13:02:47.854171  1715 rpcz_store.cc:276] 0709 13:02:45.203850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.932116 (+728266us) service_pool.cc:224] Handling call
0709 13:02:45.932578 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.601775 (+669197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.851290 (+1249515us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b540020ba54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:47.851484 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1521647872}
I20260709 13:02:47.856724  1569 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:47.856673  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 825) took 2592 ms. Trace:
I20260709 13:02:47.864812  1636 rpcz_store.cc:276] 0709 13:02:45.264110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.264244 (+   134us) service_pool.cc:224] Handling call
0709 13:02:45.285114 (+ 20870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.417615 (+132501us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:45.696644 (+279029us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.064672 (+368028us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.064722 (+    50us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:46.064762 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.856478 (+1791716us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b5400125a54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:47.856521 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.856620 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":449440,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":225116,"spinlock_wait_cycles":319732224,"thread_start_us":633313,"threads_started":2,"wal-append.queue_time_us":584254,"wal-append.run_cpu_time_us":1495,"wal-append.run_wall_time_us":1036388}
I20260709 13:02:47.866161  1569 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:02:47.867470  5404 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:47.868064  5404 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:47.870342  5404 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:47.862845  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2146) took 1423 ms. Trace:
I20260709 13:02:47.870934  1560 rpcz_store.cc:276] 0709 13:02:46.439920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.440060 (+   140us) service_pool.cc:224] Handling call
0709 13:02:46.476364 (+ 36304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:46.476860 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:46.476902 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:46.477454 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:02:47.862438 (+1384984us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:47.862489 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.862566 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.862613 (+    47us) raft_consensus.cc:1685] finished
0709 13:02:47.862627 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.862780 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3264,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":41781,"thread_start_us":26813,"threads_started":1,"wal-append.queue_time_us":1320169}
I20260709 13:02:47.818714  1565 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:02:47.863997  1563 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.381s	user 0.002s	sys 0.006s
W20260709 13:02:47.875168  1563 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.392s	user 0.003s	sys 0.006s
I20260709 13:02:47.876114  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2148) took 1398 ms. Trace:
I20260709 13:02:47.876472  1563 rpcz_store.cc:276] 0709 13:02:46.478058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.478216 (+   158us) service_pool.cc:224] Handling call
0709 13:02:46.478576 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:46.482363 (+  3787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:46.482412 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:46.483058 (+   646us) log.cc:844] Serialized 26 byte log entry
0709 13:02:47.570301 (+1087243us) spinlock_profiling.cc:243] Waited 28.7 ms on lock 0x7b48000030d8. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a6bcaaa1 00007fd7a6bc1b37 00007fd7a6bc19ca 00007fd7a6cb926f 00007fd7a6cb8f37 00007fd7a6ccd55c 00007fd7a6ccd508 00007fd7a6ccd488 00007fd7a6ccd443 00007fd7a6ccc6ec 00007fd7a5bf6a96
0709 13:02:47.875621 (+305320us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:47.875670 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.875743 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.875790 (+    47us) raft_consensus.cc:1685] finished
0709 13:02:47.875804 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.876050 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3294,"raft.queue_time_us":7728,"raft.run_cpu_time_us":2816,"raft.run_wall_time_us":1079376,"spinlock_wait_cycles":66034560,"thread_start_us":1392213,"threads_started":1,"wal-append.queue_time_us":236216,"wal-append.run_cpu_time_us":1567,"wal-append.run_wall_time_us":1003625}
I20260709 13:02:47.883323  1565 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:02:47.889448  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.894605  1565 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:47.896926  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2504) took 2014 ms. Trace:
I20260709 13:02:47.897367  1561 rpcz_store.cc:276] 0709 13:02:45.882999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.895700 (+ 12701us) service_pool.cc:224] Handling call
0709 13:02:45.925132 (+ 29432us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:45.925586 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.925623 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:45.926263 (+   640us) log.cc:844] Serialized 52 byte log entry
0709 13:02:46.274231 (+347968us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:02:46.274307 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.274401 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:47.896682 (+1622281us) raft_consensus.cc:1685] finished
0709 13:02:47.896704 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:47.896862 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"thread_start_us":347504,"threads_started":1,"wal-append.queue_time_us":801308,"wal-append.run_cpu_time_us":2660,"wal-append.run_wall_time_us":1164207}
I20260709 13:02:47.899802  5242 raft_consensus.cc:3055] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:47.900265  5242 raft_consensus.cc:740] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:47.894052  1462 catalog_manager.cc:5697] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 4 to 6, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 6 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:47.901543  5242 consensus_queue.cc:260] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.886739  5436 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.900206  1561 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:47.904672  1462 catalog_manager.cc:5697] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 3 to 6, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 6 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:47.897532  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.907522  1462 catalog_manager.cc:5697] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:47.902777  5242 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:47.908879  1565 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:47.928133  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:47.928628  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:47.929443  1718 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:47.947898  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:47.933096  5104 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:47.948457  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:47.949216  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.950668  2297 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:47.953544  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.954114  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:47.951462  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.954454  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:47.954989  1565 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:02:47.955528  5415 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:47.955896  5415 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:47.956275  5415 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:47.956434  1565 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:02:47.958047  1569 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:47.959468  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:47.960115  2297 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:47.963300  1565 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:47.963392  5104 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.969679  2297 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:47.971220  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:47.972414  5387 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:47.973012  5387 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:47.973939  5387 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.974393  5370 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:47.975200  5370 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:47.978407  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:47.981038  5370 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:47.979588  2297 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:47.982815  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:47.983219  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:47.983561  2297 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
W20260709 13:02:47.984218  1712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 759) took 3194 ms (client timeout 1930 ms). Trace:
I20260709 13:02:47.984299  5387 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:47.984491  5370 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:47.989773  5370 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:47.986124  5104 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:47.986683  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:47.987247  1712 rpcz_store.cc:269] 0709 13:02:44.789350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.815237 (+ 25887us) service_pool.cc:224] Handling call
0709 13:02:47.984003 (+3168766us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54003386d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:47.984112 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3732996992}
I20260709 13:02:47.988335  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:47.987425  5387 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.985570  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:47.974316  5415 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:47.993209  2297 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:48.019251  4048 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:48.019683  4048 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.019490  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:48.019806  1534 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:48.019935  4048 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:48.020778  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:48.021606  2297 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:48.018302  5387 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:48.022688  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:48.022594  5104 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:48.024055  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:48.024490  5104 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:48.024699  2297 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:48.024909  5104 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:48.025730  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:48.026262  2297 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:48.027576  5370 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:48.028874  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:48.029289  4048 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.030879  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:48.032425  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:48.032974  5415 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:48.033034  2298 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:48.033499  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.035367  1534 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:48.036197  5370 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:48.036572  5370 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.036913  5370 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:48.039049  1566 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:48.040894  2298 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:48.043424  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:48.044008  2296 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:48.046496  4048 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:48.051002  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:48.051304  2297 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:48.051715  2298 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:48.054701  2296 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:48.061187  2298 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:48.062567  5436 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:48.063364  5436 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.065492  5436 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:48.091382  1607 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:48.093168  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.098517  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:48.099418  2298 raft_consensus.cc:2393] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 6: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:48.092276  5470 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:48.100467  5470 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.102859  5104 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.104527  5391 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:48.104885  5404 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:48.105482  5391 raft_consensus.cc:3055] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:48.105921  5391 raft_consensus.cc:740] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:48.106315  5104 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:48.107074  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:48.107250  5473 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:48.107721  5473 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:48.102900  5470 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:48.108207  5473 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:48.052531  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:48.107730  1712 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:48.115175  5370 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.098299  5472 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:02:48.106778  5391 consensus_queue.cc:260] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.118261  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:48.121569  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 69134.7us
I20260709 13:02:48.123656  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:48.124572  2298 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:48.123433  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:48.141364  4048 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:48.142112  4048 raft_consensus.cc:697] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:48.144456  2298 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:48.167701  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:48.170130  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:48.171137  1712 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:48.182642  1534 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:48.172679  5391 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:48.167153  4048 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.197517  5445 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:48.200629  5445 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:48.201802  5445 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.159085  5104 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:48.208876  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:48.233790  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.197569  5415 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:02:48.284175  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.321344  5370 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:48.248883  5473 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.367935  5473 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:48.510318  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:48.511417  1709 raft_consensus.cc:2368] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10.
I20260709 13:02:48.524132  1716 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:02:48.525516  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:48.526172  1711 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:48.527064  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:48.528242  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:48.528935  1711 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:48.530015  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:48.530613  1711 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:48.556357  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:48.557669  1563 raft_consensus.cc:2410] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:02:48.492940  5415 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:48.549232  5104 raft_consensus.cc:697] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:48.589952  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:48.638674  1713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.000s	sys 0.012s
W20260709 13:02:48.639137  1713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.000s	sys 0.012s
W20260709 13:02:48.615770  1665 debug-util.cc:398] Leaking SignalData structure 0x7b08000cc020 after lost signal to thread 434
I20260709 13:02:48.678396  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:48.654745  1711 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
W20260709 13:02:48.704078  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 454139us
I20260709 13:02:48.703198  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:48.705554  1711 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:02:48.707434  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:48.708438  1711 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
W20260709 13:02:48.712090  5391 consensus_meta.cc:287] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.507s	user 0.007s	sys 0.003s
I20260709 13:02:48.725848  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:48.715047  5387 consensus_meta.cc:287] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.000s
W20260709 13:02:48.729626  1607 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.009s	sys 0.024s
W20260709 13:02:48.730294  1607 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.009s	sys 0.024s
W20260709 13:02:48.733291  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 546091us
W20260709 13:02:48.735788  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:48.736269  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.736642  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:48.737416  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:48.738174  2298 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:48.738657  1607 leader_election.cc:400] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10.
I20260709 13:02:48.739012  1607 leader_election.cc:403] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:48.739543  5391 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:48.692924  5415 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.741582  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:48.742184  1560 raft_consensus.cc:2376] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10. Re-sending same reply.
I20260709 13:02:48.746680  2298 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:02:48.747648  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:48.742722  5473 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:02:48.748867  1534 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:48.729777  5104 consensus_queue.cc:237] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.755307  4048 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:48.748556  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:48.739971  5391 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.756083  4048 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:48.731271  5387 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.741919  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:48.760238  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:48.760339  5473 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10.
I20260709 13:02:48.761536  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:48.762279  5473 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:48.762718  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:48.762928  5473 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.763649  5473 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:48.773609  5387 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:48.757601  5391 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:48.778227  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:48.780997  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:48.781167  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:48.782470  2299 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:48.783078  2300 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:48.784122  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:48.785178  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:48.785712  2300 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:48.787426  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:48.782532  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:48.790045  2297 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:48.790309  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:48.791083  2298 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:48.791308  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1082) took 1061 ms. Trace:
I20260709 13:02:48.792564  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:48.794528  1560 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:02:48.793396  2299 rpcz_store.cc:276] 0709 13:02:47.730292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.780793 (+1050501us) service_pool.cc:224] Handling call
0709 13:02:48.791231 (+ 10438us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:48.795701  4048 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.804170  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:48.806980  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:48.807906  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:48.811707  5243 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.812335  5243 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.813783  2298 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:02:48.815986  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:48.817595  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:48.818703  1560 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:48.821197  5556 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:48.821597  1713 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:48.821769  5556 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:48.821887  5452 consensus_queue.cc:1059] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:48.822901  5556 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.824640  5507 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:48.825121  5507 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:48.825541  5507 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:48.824702  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1076) took 1096 ms. Trace:
W20260709 13:02:48.831758  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.826962  2301 rpcz_store.cc:276] 0709 13:02:47.728660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.823854 (+1095194us) service_pool.cc:224] Handling call
0709 13:02:48.824192 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:48.824373 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.824398 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:48.824442 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:48.824489 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.824542 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:48.824651 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:48.858999  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 94826.1us
I20260709 13:02:48.841694  5243 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:48.862262  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 84110.9us
I20260709 13:02:48.849800  5452 raft_consensus.cc:3055] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:02:48.867395  1610 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.529s	user 0.000s	sys 0.025s
W20260709 13:02:48.878502  1610 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.540s	user 0.000s	sys 0.026s
I20260709 13:02:48.870838  2304 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:02:48.872963  1683 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:48.879685  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1080) took 1149 ms. Trace:
I20260709 13:02:48.880050  2304 rpcz_store.cc:276] 0709 13:02:47.729788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.870112 (+1140324us) service_pool.cc:224] Handling call
0709 13:02:48.870573 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:48.879406 (+  8833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:48.879630 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:48.880409  1683 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:48.887108  1683 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:48.889276  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:48.890959  1683 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:48.899642  5341 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:48.900127  5341 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:48.900522  5341 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:48.904663  5473 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.892549  5243 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:48.912995  5465 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:48.913568  5465 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:48.914177  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.878257  5452 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:48.915863  5452 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.917454  5452 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:48.918123  5516 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:48.918601  5516 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:48.919065  5516 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:48.925017  5465 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:48.932168  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:48.933013  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:48.933828  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:48.938756  1563 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:48.904980  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:48.940860  2302 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:48.941536  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1086) took 1201 ms. Trace:
I20260709 13:02:48.941916  2302 rpcz_store.cc:276] 0709 13:02:47.739946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.904687 (+1164741us) service_pool.cc:224] Handling call
0709 13:02:48.941462 (+ 36775us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:46.521440  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:48.945256  1564 raft_consensus.cc:2368] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10.
W20260709 13:02:48.945699  1564 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2147) took 2478 ms (client timeout 1815 ms). Trace:
W20260709 13:02:48.946084  1564 rpcz_store.cc:269] 0709 13:02:46.467172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.467312 (+   140us) service_pool.cc:224] Handling call
0709 13:02:48.945630 (+2478318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:48.946588  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:48.947299  1563 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:48.948083  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:48.948702  1709 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:02:48.949396  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:48.952467  5516 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:48.954697  5516 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:48.955464  5516 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:48.955265  5341 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.956096  5516 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:48.957419  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2512) took 2476 ms (client timeout 1562 ms). Trace:
I20260709 13:02:48.957147  5516 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:48.957828  1566 rpcz_store.cc:269] 0709 13:02:46.481349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.481506 (+   157us) service_pool.cc:224] Handling call
0709 13:02:48.957206 (+2475700us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b5400325854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:48.957354 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632066944}
I20260709 13:02:48.961158  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:48.961903  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:48.963071  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 68140us
I20260709 13:02:48.963969  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:48.902449  5391 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.971010  4048 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:48.986614  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:48.992659  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.022732  1711 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
W20260709 13:02:48.999262  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 139339us
I20260709 13:02:49.023064  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.025390  1638 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:48.979229  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.025987  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:48.972241  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.026473  1636 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:49.026590  1711 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:49.027091  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:49.026533  1641 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:49.027838  1717 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:48.971788  1532 connection.cc:256] RPC call timeout handler was delayed by -1.2169263909999586s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18130000000000002s for a response
I20260709 13:02:49.022568  4048 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.041186  5520 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:49.041834  5520 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:49.046567  5438 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:49.047204  5587 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:49.047377  5438 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:49.047863  5587 raft_consensus.cc:697] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:49.048352  5438 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:49.048741  5438 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:49.049139  5438 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:49.048920  5587 consensus_queue.cc:237] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.991954  5580 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:49.052585  5580 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.052971  5580 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:49.056186  5391 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:48.994735  5473 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:49.062328  5461 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.983390  5556 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.062903  5461 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.063373  5556 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.063843  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:49.064615  1709 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:49.065868  5556 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:49.068005  5341 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:49.068799  5341 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:49.069169  5341 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:49.069988  5341 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:48.983196  5415 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.071806  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.072073  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:49.072643  1636 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:49.072775  2305 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:49.073657  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.075212  1637 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:49.078802  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.079282  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.080276  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.080649  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.081843  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:49.082669  1713 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 9: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:49.082609  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:48.983363  5581 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:02:49.086180  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.072292  5415 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:49.083915  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:49.089928  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:49.096014  2302 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:49.084262  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:49.073454  2305 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1078) took 1344 ms (client timeout 1530 ms). Trace:
W20260709 13:02:49.097191  2305 rpcz_store.cc:269] 0709 13:02:47.729210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.071795 (+1342585us) service_pool.cc:224] Handling call
0709 13:02:49.073372 (+  1577us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:49.084553  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.080302  1640 raft_consensus.cc:2410] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:02:49.086552  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:49.098402  1643 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:49.098534  1644 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:49.098796  1715 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 9: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:49.099515  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.086663  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:49.100553  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:49.100989  1568 raft_consensus.cc:2368] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10.
I20260709 13:02:49.087173  5520 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:49.087988  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.088382  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.104251  1638 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:49.104597  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:48.969667  5582 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:48.993019  5292 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 13:02:49.089542  2268 leader_election.cc:400] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9.
I20260709 13:02:49.101658  1713 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 9: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:49.101751  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.100560  1643 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:02:49.100579  1641 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:49.107897  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:49.106751  2268 leader_election.cc:403] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:02:49.108929  1715 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:49.109817  2268 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:49.111737  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.103645  1717 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:49.114078  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:49.115231  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:49.116291  1717 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:49.117159  1709 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:49.118345  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:49.118965  1709 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:49.121127  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:49.121176  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:49.122004  4048 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:49.123852  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:49.125356  1711 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:49.128829  4048 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:49.129290  4048 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.129724  4048 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:49.129309  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:49.132861  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 168956us
I20260709 13:02:49.133891  1683 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:49.134307  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:49.142705  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:49.143376  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.143541  1711 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:02:49.144726  1711 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:49.146610  1644 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:49.099555  5391 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:49.151427  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.113400  5582 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:49.159690  1636 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:49.167318  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:49.169767  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 115740us
I20260709 13:02:49.178236  4048 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.184792  4048 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:49.185427  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 124974us
I20260709 13:02:49.170776  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:49.181628  1709 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:49.186717  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:02:49.188109  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 53879.3us
I20260709 13:02:49.190373  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:49.146097  1715 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:49.156806  5461 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:49.158532  5582 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:49.199817  5516 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:49.200357  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:49.200664  5516 raft_consensus.cc:697] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:49.201203  1566 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:49.200474  5595 raft_consensus.cc:2764] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 13:02:49.201275  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 91527.9us
I20260709 13:02:49.201903  5516 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.198203  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:49.205744  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:49.205968  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:49.208804  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:49.209054  1566 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:49.177709  5243 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:49.209599  1636 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:49.207368  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:49.210080  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:49.157167  5391 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:49.149019  5584 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:02:49.171248  5474 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:02:49.181559  5549 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:49.233428  1563 raft_consensus.cc:2410] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:02:49.235491  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.203250  5461 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9.
W20260709 13:02:49.230436  1665 debug-util.cc:398] Leaking SignalData structure 0x7b080021bfc0 after lost signal to thread 1501
I20260709 13:02:49.237567  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:49.243276  1644 raft_consensus.cc:2410] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:02:49.235370  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 49257.8us
W20260709 13:02:49.237867  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 67184.4us
I20260709 13:02:49.236775  5292 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:02:49.254873  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 48127.9us
W20260709 13:02:49.240931  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 98534.5us
I20260709 13:02:49.265137  1683 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:49.267024  5461 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:49.268728  5603 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:49.269594  5603 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:49.241879  5243 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:02:49.246723  5549 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:49.277285  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:49.251761  5292 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.277658  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.251065  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.258833  5452 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:49.291662  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:49.301410  1636 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:49.302851  5243 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:49.303200  5598 raft_consensus.cc:3055] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:49.248469  1681 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:49.282297  1644 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:49.343499  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:49.306650  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.236843  5589 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:02:49.271585  5461 consensus_queue.cc:1059] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:02:49.276685  5549 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:49.320995  2269 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:49.331440  1643 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
W20260709 13:02:49.348224  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 99863.4us
I20260709 13:02:49.280146  5601 raft_consensus.cc:3055] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:49.324681  5292 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:49.324564  5452 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.339896  5243 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.359233  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.363432  1643 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:49.366935  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2496) took 4780 ms. Trace:
I20260709 13:02:49.366477  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:49.367448  1562 rpcz_store.cc:276] 0709 13:02:44.580285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.615520 (+ 35235us) service_pool.cc:224] Handling call
0709 13:02:44.616003 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.728419 (+112416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.728449 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.728930 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.391000 (+662070us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:45.391038 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:47.818159 (+2427121us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:49.360361 (+1542202us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b5400325d54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:49.360430 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:49.360477 (+    47us) raft_consensus.cc:1685] finished
0709 13:02:49.360489 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.360612 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51080,"raft.queue_time_us":51875,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":24755,"spinlock_wait_cycles":179504256,"thread_start_us":578373,"threads_started":1,"wal-append.queue_time_us":595819,"wal-append.run_cpu_time_us":35849,"wal-append.run_wall_time_us":3611789}
W20260709 13:02:49.369580  5593 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 13:02:49.371588  5588 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 13:02:49.372073  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:49.376541  5586 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:49.377131  5586 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:49.381071  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2019) took 4199 ms. Trace:
I20260709 13:02:49.381487  1714 rpcz_store.cc:276] 0709 13:02:45.181544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.479239 (+297695us) service_pool.cc:224] Handling call
0709 13:02:45.787966 (+308727us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:45.806908 (+ 18942us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:45.806989 (+    81us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:45.807846 (+   857us) log.cc:844] Serialized 26 byte log entry
0709 13:02:46.244303 (+436457us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:02:46.244343 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.244414 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:46.922597 (+678183us) raft_consensus.cc:1685] finished
0709 13:02:46.922637 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:49.380813 (+2458176us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b54000483d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:49.381008 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13797,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":11871,"spinlock_wait_cycles":709137152,"thread_start_us":378139,"threads_started":1,"wal-append.queue_time_us":587916,"wal-append.run_cpu_time_us":4500,"wal-append.run_wall_time_us":1448171}
W20260709 13:02:49.383080  1609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.977s	user 0.027s	sys 0.060s
I20260709 13:02:49.383848  4960 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:49.353124  5589 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:49.386094  5589 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.386588  5589 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:49.391208  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 99670.8us
I20260709 13:02:49.341811  5598 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:49.351318  5549 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:49.354269  5604 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:49.279537  5592 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
W20260709 13:02:49.437314  1609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.031s	user 0.027s	sys 0.061s
I20260709 13:02:49.354522  5601 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:49.377965  5586 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.365641  5243 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:49.360174  5474 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:49.362529  5452 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:49.365887  5415 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:49.449849  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.480111  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:49.481125  1643 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 9: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:49.455674  1640 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:02:49.453792  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.481714  1635 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:49.455422  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.482205  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:49.482766  1640 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:02:49.483029  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:49.463816  5507 consensus_meta.cc:287] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.638s	user 0.011s	sys 0.000s
I20260709 13:02:49.421768  5602 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:49.461220  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:49.466902  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 74756.1us
I20260709 13:02:49.471321  5612 raft_consensus.cc:2764] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:02:49.442843  5370 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:49.447911  5610 raft_consensus.cc:2764] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:02:49.519860  1640 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
W20260709 13:02:49.505391  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 295439us
W20260709 13:02:49.500969  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 180058us
I20260709 13:02:49.513046  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.521564  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:49.504814  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:49.465618  5556 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:49.470315  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.525565  1637 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:49.525942  2268 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:49.524573  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.524471  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.527292  1643 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:02:49.502321  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 57319.7us
I20260709 13:02:49.527297  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.527807  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:49.524497  5370 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.499207  5602 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.568060  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.523118  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:49.567824  1681 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:49.469280  5242 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
W20260709 13:02:49.588863  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 65829.2us
I20260709 13:02:49.589442  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:49.577556  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 53062.5us
I20260709 13:02:49.499370  5474 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:49.593178  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:49.531111  5608 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:49.568200  1641 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
W20260709 13:02:49.583683  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 52880.5us
I20260709 13:02:49.510864  5391 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:49.503058  5507 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.462041  5604 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.587535  5611 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:02:49.591374  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.581295  1643 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:49.604346  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.619861  1641 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 5.
I20260709 13:02:49.571370  5612 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:49.586627  5590 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:49.612448  1681 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:49.588070  5602 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:49.594480  5370 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:49.569053  5609 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:02:49.693689  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.617223  5618 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:49.622368  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.569249  5556 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.632670  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.604413  5608 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.697695  5608 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:49.698756  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.700572  5606 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:02:49.703476  5604 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:49.703742  5609 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:49.704095  5615 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:02:49.704254  5609 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:49.704649  5609 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:49.707270  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 94921.8us
I20260709 13:02:49.708061  1681 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:49.709558  1681 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:49.634563  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.611361  5605 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:02:49.690554  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 55902.5us
I20260709 13:02:49.722256  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:49.722635  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:49.729715  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:49.689034  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 161288us
I20260709 13:02:49.737008  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:49.706213  5608 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:49.806012  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:49.625203  4960 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:49.813551  1563 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:02:49.617532  5620 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.815726  1639 raft_consensus.cc:2368] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 12. Current term is 13.
I20260709 13:02:49.816298  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:49.840310  2304 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:49.817308  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:49.818739  1643 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:49.818743  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 109218us
I20260709 13:02:49.823774  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:49.687911  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 83411.3us
I20260709 13:02:49.829402  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:49.848691  1568 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:49.832198  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:49.849874  1711 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
W20260709 13:02:49.836258  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 106629us
I20260709 13:02:49.724481  5599 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:49.524856  5598 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.849198  1607 leader_election.cc:400] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10.
I20260709 13:02:49.772787  5590 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:49.773471  5621 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:49.862164  1607 leader_election.cc:403] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:49.872675  1609 connection.cc:256] RPC call timeout handler was delayed by -2.23352099899995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2126s for a response
W20260709 13:02:49.873226  1609 connection.cc:256] RPC call timeout handler was delayed by -2.0174328159999959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
W20260709 13:02:49.873605  1609 connection.cc:256] RPC call timeout handler was delayed by -1.6497842340000375s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19840000000000002s for a response
I20260709 13:02:49.874310  1609 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:49.876600  1643 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:49.880244  1713 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:49.883049  1609 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:49.850733  1681 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:49.852332  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:49.853211  4960 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.868659  5623 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:49.868762  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:49.809985  5242 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:49.894873  2304 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
W20260709 13:02:49.901254  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.896590  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:49.911428  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.912156  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:49.861658  5335 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:49.922148  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 71500.4us
I20260709 13:02:49.857067  5625 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:49.925453  1563 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:02:49.863348  5556 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:49.913035  5438 consensus_meta.cc:287] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.863s	user 0.010s	sys 0.001s
I20260709 13:02:49.912701  5242 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.915977  5598 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:49.937448  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 85230.3us
I20260709 13:02:49.943961  2302 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:49.944391  5609 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.898419  5611 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:49.938793  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:49.949402  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:49.952972  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:49.950042  2302 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:49.963768  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:49.956660  1563 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:49.954849  1560 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:02:49.950749  2298 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:49.953213  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:49.981132  5516 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:49.995970  5586 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10.
I20260709 13:02:49.997197  5586 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:49.997730  5586 raft_consensus.cc:697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:49.998693  5586 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:49.965747  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 116738us
I20260709 13:02:49.948917  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:49.927049  5625 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:50.057902  5625 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.861925  5620 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:49.923086  5335 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:02:50.006711  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.934439  5556 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.869877  5474 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.011216  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:49.964622  5580 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.911s	user 0.009s	sys 0.003s
I20260709 13:02:49.945742  5242 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:49.963371  5623 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:49.957499  5436 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:50.012676  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:50.014770  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 131839us
I20260709 13:02:49.958140  5629 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:49.879999  5621 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:50.019460  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 60920.3us
I20260709 13:02:50.011315  5624 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:49.948396  5438 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:50.043581  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 94274.7us
I20260709 13:02:50.107415  1682 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:50.109225  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:50.043758  5641 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:02:50.086191  2303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1079) took 2356 ms (client timeout 2662 ms). Trace:
W20260709 13:02:50.090086  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 77497.4us
I20260709 13:02:50.092381  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:50.112865  1562 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:02:50.112914  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:50.112142  2303 rpcz_store.cc:269] 0709 13:02:47.729508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.963183 (+2233675us) service_pool.cc:224] Handling call
0709 13:02:50.086116 (+122933us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:50.101615  1683 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:49.877193  1455 maintenance_manager.cc:643] P 6ffeec09a12e4bd6997354c8ecdff02d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 9.683s	user 0.190s	sys 0.060s Metrics: {"bytes_written":30203,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":34742,"dirs.run_cpu_time_us":463,"dirs.run_wall_time_us":12884,"drs_written":1,"lbm_read_time_us":188257,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_gt_100_ms":1,"lbm_reads_lt_1ms":1,"lbm_write_time_us":70288,"lbm_writes_1-10_ms":2,"lbm_writes_10-100_ms":2,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":12188,"threads_started":1,"update_count":9}
I20260709 13:02:50.016968  5603 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:02:50.068907  5335 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:50.060381  5507 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:50.149729  2268 leader_election.cc:336] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.930s (SENT)
I20260709 13:02:50.150482  2268 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:50.149498  1665 debug-util.cc:398] Leaking SignalData structure 0x7b080021baa0 after lost signal to thread 1593
W20260709 13:02:50.151875  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 50368.3us
W20260709 13:02:50.167291  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 57766.7us
I20260709 13:02:50.104470  5461 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:50.091773  5629 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.170288  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:50.093001  5580 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:50.174327  5601 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.181658  1562 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:02:50.112426  5625 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:50.114332  5608 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:50.092031  5556 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.166606  5603 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:50.204178  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:50.187184  1456 maintenance_manager.cc:419] P 6ffeec09a12e4bd6997354c8ecdff02d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9784 bytes on disk
I20260709 13:02:50.125690  5623 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:50.124017  5187 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
W20260709 13:02:50.270507  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.216048  1569 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:50.185537  5461 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:50.186617  5630 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:50.190267  5599 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:50.191905  5629 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:50.214764  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 125096us
I20260709 13:02:50.305984  1455 maintenance_manager.cc:643] P 6ffeec09a12e4bd6997354c8ecdff02d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.087s	user 0.000s	sys 0.002s Metrics: {"cfile_init":1,"lbm_read_time_us":48355,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":3}
I20260709 13:02:50.271507  1567 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
W20260709 13:02:50.307415  1567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2145) took 3871 ms (client timeout 2131 ms). Trace:
W20260709 13:02:50.307905  1567 rpcz_store.cc:269] 0709 13:02:46.436369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.436532 (+   163us) service_pool.cc:224] Handling call
0709 13:02:50.307322 (+3870790us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:50.209210  5626 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
I20260709 13:02:50.276129  1642 raft_consensus.cc:2368] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13.
I20260709 13:02:50.199772  5587 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:02:50.116194  5624 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.097852  5611 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:50.320078  2268 leader_election.cc:336] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.574s (SENT)
I20260709 13:02:50.319819  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:50.287866  5516 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:50.280009  5461 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:50.321154  1609 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:50.337507  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:50.311023  5647 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:50.318266  5601 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:50.374051  1665 debug-util.cc:398] Leaking SignalData structure 0x7b08003adae0 after lost signal to thread 1668
I20260709 13:02:50.317621  5599 raft_consensus.cc:697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:50.372902  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 244989us
W20260709 13:02:50.357851  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 174578us
W20260709 13:02:50.383613  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 62558.5us
I20260709 13:02:50.390136  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:50.393501  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:50.332415  5609 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:50.400707  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.388697  5435 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:02:50.402653  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:50.403577  2303 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:50.404294  5592 raft_consensus.cc:3055] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:50.404795  5592 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:50.405963  5592 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.373863  5623 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.411468  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.412451  1638 raft_consensus.cc:3055] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:50.472641  1638 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:50.414101  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:50.396292  5636 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:02:50.430989  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:50.481518  2303 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
W20260709 13:02:50.434957  5637 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:02:50.379495  5647 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:50.384575  5630 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
W20260709 13:02:50.436307  5631 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.331s	user 0.001s	sys 0.000s
I20260709 13:02:50.476557  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:50.483328  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:50.486270  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:50.487457  5591 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.229s	user 0.012s	sys 0.017s
I20260709 13:02:50.499435  5556 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:50.490805  1638 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.404361  5435 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.079s
W20260709 13:02:50.488464  5243 consensus_meta.cc:287] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.013s	user 0.007s	sys 0.004s
W20260709 13:02:50.501087  1712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2153) took 1976 ms (client timeout 2036 ms). Trace:
W20260709 13:02:50.519937  1712 rpcz_store.cc:269] 0709 13:02:48.524349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.419113 (+1894764us) service_pool.cc:224] Handling call
0709 13:02:50.501017 (+ 81904us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:50.452051  5452 consensus_meta.cc:287] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.953s	user 0.006s	sys 0.004s
W20260709 13:02:50.431936  5632 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.309s	user 0.001s	sys 0.001s
W20260709 13:02:50.446043  5600 log.cc:927] Time spent T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 1.175s	user 0.002s	sys 0.000s
W20260709 13:02:50.440750  5645 log.cc:927] Time spent T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:02:50.460845  1665 debug-util.cc:398] Leaking SignalData structure 0x7b0800333cc0 after lost signal to thread 1742
I20260709 13:02:50.507429  2301 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:50.475870  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:50.536310  2305 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:50.508196  1711 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:50.511390  4048 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.535660  1665 debug-util.cc:398] Leaking SignalData structure 0x7b08003f6180 after lost signal to thread 1743
I20260709 13:02:50.537525  4048 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.439244  4960 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:50.500366  5620 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:50.512730  5591 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.263s	user 0.012s	sys 0.018s
W20260709 13:02:50.501454  5549 consensus_meta.cc:287] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.039s	user 0.008s	sys 0.003s
I20260709 13:02:50.522329  5610 raft_consensus.cc:3055] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:50.511230  5592 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:50.546222  4048 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:50.515554  5652 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:02:50.539304  5589 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.152s	user 0.008s	sys 0.005s
W20260709 13:02:50.546620  5644 log.cc:927] Time spent T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.321s	user 0.000s	sys 0.002s
I20260709 13:02:50.538673  5641 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:50.540565  5580 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:50.502146  5623 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.538609  5658 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.537338  5452 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:50.507229  5599 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.557459  5610 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:50.558655  5549 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:50.574865  5582 consensus_meta.cc:287] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.343s	user 0.009s	sys 0.003s
I20260709 13:02:50.575086  5624 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:50.575982  5621 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:50.576241  5622 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.563s	user 0.001s	sys 0.001s
I20260709 13:02:50.591712  5469 raft_consensus.cc:3055] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:50.592185  5469 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:50.593300  5605 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.593828  5605 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:50.577967  5647 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:50.572157  5652 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:50.599064  5582 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.600481  5652 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.601725  5582 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:50.603536  5598 consensus_meta.cc:287] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.000s
I20260709 13:02:50.605459  5549 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:50.578902  5655 log.cc:927] Time spent T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:02:50.581065  5597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.361s	user 0.005s	sys 0.033s
W20260709 13:02:50.606083  5602 consensus_meta.cc:287] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.833s	user 0.011s	sys 0.000s
I20260709 13:02:50.589074  5658 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.473723  1532 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:50.676364  1566 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.012s	sys 0.000s
W20260709 13:02:50.677268  1566 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.012s	sys 0.000s
W20260709 13:02:50.676364  1564 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.005s	sys 0.009s
W20260709 13:02:50.677744  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 203906us
W20260709 13:02:50.678578  1564 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.005s	sys 0.009s
I20260709 13:02:50.518787  1638 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:50.575901  5341 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:50.533757  5335 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:50.593004  5469 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.705444  5469 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:50.571126  5589 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.709444  5475 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:50.710001  5475 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.710352  5623 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:50.692355  5610 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:50.609932  5597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.405s	user 0.006s	sys 0.033s
I20260709 13:02:50.711275  5473 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:50.712394  5549 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:50.533761  5243 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.612646  5602 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.624250  5658 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:50.723884  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.710391  5475 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:50.698575  5452 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:50.744029  5520 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:50.621551  5641 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:50.759210  1568 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.007s	sys 0.005s
W20260709 13:02:50.759835  1568 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.007s	sys 0.005s
W20260709 13:02:50.761636  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.763429  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:50.750536  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 70410.9us
I20260709 13:02:50.751252  5335 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:50.751761  1714 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 12: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:50.751860  5589 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:50.778018  1710 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:02:50.779115  5673 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.058s	user 0.000s	sys 0.002s
W20260709 13:02:50.789757  1535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.008s	sys 0.020s
W20260709 13:02:50.790326  1535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.008s	sys 0.020s
I20260709 13:02:50.780162  5243 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:48.645651   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:50.803514  5610 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:50.804543  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:50.803037  5549 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:50.817842  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2498) took 1740 ms (client timeout 1935 ms). Trace:
I20260709 13:02:50.823208  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:50.828506  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2046) took 1022 ms. Trace:
I20260709 13:02:50.829105  1713 rpcz_store.cc:276] 0709 13:02:49.805488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.805753 (+   265us) service_pool.cc:224] Handling call
0709 13:02:49.806063 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.880668 (+ 74605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.827752 (+947084us) spinlock_profiling.cc:243] Waited 69.7 ms on lock 0x7b54003377d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:50.828214 (+   462us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160334720}
I20260709 13:02:50.830880  5658 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.831905  5658 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:50.833585  5391 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:50.834064  5438 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:50.834401  5391 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.825652  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2136) took 4230 ms. Trace:
I20260709 13:02:50.836999  1716 rpcz_store.cc:276] 0709 13:02:46.595672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.595798 (+   126us) service_pool.cc:224] Handling call
0709 13:02:48.523690 (+1927892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:48.524837 (+  1147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.824985 (+2300148us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b54003381d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:50.825288 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4374209408}
I20260709 13:02:50.837512  5391 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:50.838032  5658 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:50.840552  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.783571  5473 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.813510  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2097) took 1349 ms. Trace:
W20260709 13:02:50.818444  1642 rpcz_store.cc:269] 0709 13:02:49.077080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.080388 (+  3308us) service_pool.cc:224] Handling call
0709 13:02:50.817149 (+1736761us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400123c54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:50.817568 (+   419us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2735755136}
W20260709 13:02:50.867489  1533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.014s	sys 0.027s
W20260709 13:02:50.868010  1533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.014s	sys 0.027s
I20260709 13:02:50.869743  5420 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.870695  5420 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.875260  5420 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:50.875960  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 781) took 1059 ms. Trace:
I20260709 13:02:50.876600  1714 rpcz_store.cc:276] 0709 13:02:49.816563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.817049 (+   486us) service_pool.cc:224] Handling call
0709 13:02:50.875648 (+1058599us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b5400337cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:50.875845 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2032469760}
I20260709 13:02:50.886557  5605 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.891968  5602 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:50.892895  2298 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
W20260709 13:02:50.895381  1665 debug-util.cc:398] Leaking SignalData structure 0x7b0800333500 after lost signal to thread 2329
I20260709 13:02:50.899714  5605 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:50.903607  5549 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:50.904201  5438 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.904918  5438 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.907078  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 675) took 3182 ms. Trace:
I20260709 13:02:50.907579  1561 rpcz_store.cc:276] 0709 13:02:47.724673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.904267 (+179594us) service_pool.cc:224] Handling call
0709 13:02:48.960770 (+1056503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:48.961130 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.961158 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:48.961563 (+   405us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.110564 (+149001us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:49.110605 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.110687 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:49.373095 (+262408us) raft_consensus.cc:1685] finished
0709 13:02:49.373114 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.906499 (+1533385us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400325850. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:50.906823 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1764,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":2609,"spinlock_wait_cycles":2428992384,"thread_start_us":140800,"threads_started":1,"wal-append.queue_time_us":186904,"wal-append.run_cpu_time_us":3070,"wal-append.run_wall_time_us":1216716}
W20260709 13:02:50.912483  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.913288  1713 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:50.914183  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:50.915562  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:50.916486  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:50.870664  5466 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.918455  5466 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.919421  5438 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:50.920004  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:50.917423  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:50.842772  1532 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:50.917492  2298 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:50.900645  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.917627  2304 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:50.863276  1639 rpcz_store.cc:276] 0709 13:02:49.463596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.502960 (+ 39364us) service_pool.cc:224] Handling call
0709 13:02:50.812851 (+1309891us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400155754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:50.813176 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535509376}
I20260709 13:02:50.825616  5636 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:02:50.916599  1718 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
W20260709 13:02:50.927554  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 57283.9us
W20260709 13:02:50.935192  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.939329  5617 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:50.940327  5617 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.943250  5420 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.943856  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:50.944231  5420 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.943375  5621 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.953663  2303 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:50.961196  5676 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:50.961640  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:50.957372  4544 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.963116  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 120628us
I20260709 13:02:50.963941  4544 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.967334  5658 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:50.968058  5658 raft_consensus.cc:697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:50.969589  5658 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:50.943266  5647 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:50.974959  5420 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:50.976372  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:50.975914  5647 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:50.977285  1566 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:50.979363  1565 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
W20260709 13:02:50.981127  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 43857.7us
I20260709 13:02:50.981828  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:50.982764  1560 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:50.984911  5621 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:50.985702  5465 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:50.921684  5466 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:50.949154  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:50.966579  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:50.979379  2301 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:50.986814  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:50.999152  1665 debug-util.cc:398] Leaking SignalData structure 0x7b08003791a0 after lost signal to thread 5294
I20260709 13:02:50.995795  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2549) took 1066 ms. Trace:
I20260709 13:02:50.984052  5574 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:02:51.005280  5595 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:02:50.995013  5292 consensus_meta.cc:287] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.615s	user 0.011s	sys 0.010s
I20260709 13:02:51.010092  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:51.008718  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 201822us
I20260709 13:02:51.013445  1683 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:51.015722  1714 raft_consensus.cc:2393] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:51.017287  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:50.962348  5676 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.025070  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:51.027526  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:51.031385  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:51.031399  2298 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:51.032096  1566 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:51.000874  1564 rpcz_store.cc:276] 0709 13:02:49.925502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.926806 (+  1304us) service_pool.cc:224] Handling call
0709 13:02:49.962340 (+ 35534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.962549 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:50.679607 (+717058us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.799301 (+119694us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:50.799406 (+   105us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:50.930445 (+131039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.953408 (+ 22963us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.991991 (+ 38583us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":83316,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":218,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":285,"thread_start_us":780226,"threads_started":2,"wal-append.queue_time_us":180222}
I20260709 13:02:50.993217  5475 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.039335  1560 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
W20260709 13:02:51.040822  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.028496  5617 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:51.051177  5675 raft_consensus.cc:2764] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:02:51.051322  5654 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:51.052109  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:51.052363  5654 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:51.053922  5597 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.054088  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:51.053663  5654 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.054394  1636 raft_consensus.cc:2410] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:02:51.054662  5597 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.055158  5580 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.055693  5580 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:51.057528  5676 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:51.057816  1569 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:51.057396  5461 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.669s	user 0.012s	sys 0.008s
I20260709 13:02:51.059039  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:51.058750  5461 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.060729  5603 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.061333  1564 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 11: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:02:51.061869  5629 consensus_meta.cc:287] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.745s	user 0.008s	sys 0.008s
I20260709 13:02:51.062532  5633 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:51.063321  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.063195  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:51.063892  5611 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:51.060158  5597 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:51.066002  5420 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:51.020471  5292 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.073658  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:51.072773  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 94267.4us
W20260709 13:02:51.070235  5445 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.582s	user 0.020s	sys 0.065s
I20260709 13:02:51.083791  5633 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:51.071255  5601 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.605s	user 0.005s	sys 0.016s
I20260709 13:02:51.066025  5647 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:51.075263  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:51.088253  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2511) took 1745 ms. Trace:
I20260709 13:02:51.089279  2302 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:51.061416  5603 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.090205  5636 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:51.091061  5636 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:51.088979  1636 rpcz_store.cc:276] 0709 13:02:49.343002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.343142 (+   140us) service_pool.cc:224] Handling call
0709 13:02:51.087980 (+1744838us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b540032bed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:51.088178 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3931124992}
I20260709 13:02:51.082600  5243 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:51.106974  5292 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:51.107646  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:51.085490  5445 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.679s	user 0.020s	sys 0.066s
I20260709 13:02:51.109603  2303 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:51.072963  5574 consensus_queue.cc:1059] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
W20260709 13:02:51.109397  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 122746us
I20260709 13:02:51.107959  1639 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:51.117561  1565 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
W20260709 13:02:51.103715  5625 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.814s	user 0.002s	sys 0.018s
I20260709 13:02:51.078227  5629 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:51.122901  1532 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:51.098698  5461 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:51.130002  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:51.124094  1561 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:51.098629  5597 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:51.111888  5647 raft_consensus.cc:3055] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:02:51.125844  5242 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.942s	user 0.009s	sys 0.012s
I20260709 13:02:51.103952  5438 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:02:51.140892  2302 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:51.140007  1566 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:02:51.144922  1717 raft_consensus.cc:2410] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:02:51.140436  1709 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:51.141487  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:51.141629  1638 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:51.145610  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:51.124540  5520 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:51.159538  5597 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:51.160113  1639 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:51.162138  5597 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:51.148654  5475 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:51.148051  5647 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:51.167843  5520 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:51.168493  5647 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:51.139199  5574 consensus_queue.cc:1243] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:02:51.170522  5520 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:51.138741  5636 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:51.142094  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.182614  1635 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:51.185940  2299 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:02:51.188086  1560 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:02:51.190699  1561 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:51.185171  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:51.154385  5243 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:51.199451  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:51.202955  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 104312us
I20260709 13:02:51.208334  1681 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:51.213510  5243 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.194584  5676 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:51.217988  5676 raft_consensus.cc:697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:51.219584  5676 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:51.222996  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 85470.5us
I20260709 13:02:51.224190  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:51.224752  5601 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:51.224987  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.225731  1643 raft_consensus.cc:2393] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 11: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:51.227491  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.157619  5242 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.229398  2269 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:51.229861  2305 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:51.230115  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 103689us
I20260709 13:02:51.231093  5242 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:51.231491  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:51.232753  5420 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:51.238708  1532 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:51.199008  5475 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:51.240002  5420 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:51.243367  5341 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.245601  5473 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:51.254757  2302 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:51.256742  1683 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:51.244227  5341 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.263381  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:51.271469  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:51.275511  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 44196.9us
I20260709 13:02:51.274710  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:51.260567  5461 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:02:51.274132  5438 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:51.297761  5602 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:02:51.298493  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 48969us
I20260709 13:02:51.299044  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.272579  5610 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:02:51.281585  5466 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:51.284724  5292 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:51.314831  2268 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:51.298439  5602 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:51.298297  5242 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:51.318389  1533 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:51.316479  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.323487  1639 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:51.296629  5438 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:51.296015  5654 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:51.325026  1532 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:51.322283  2302 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:02:51.326421  5438 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:51.329911  5520 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.330758  5520 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:51.332782  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2039) took 2230 ms (client timeout 2884 ms). Trace:
I20260709 13:02:51.333227  1715 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:02:51.310825  5466 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.333837  1715 pending_rounds.cc:85] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 8
W20260709 13:02:51.333271  1717 rpcz_store.cc:269] 0709 13:02:49.102263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.123615 (+ 21352us) service_pool.cc:224] Handling call
0709 13:02:51.332446 (+2208831us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b54003dc7d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:51.332643 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4528526080}
I20260709 13:02:51.336846  1715 pending_rounds.cc:107] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:02:51.334678  5292 raft_consensus.cc:697] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:51.337374  1715 raft_consensus.cc:2889] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:51.333622  5520 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:51.338228  5292 consensus_queue.cc:237] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.341754  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.345083  1681 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:51.343930  1635 raft_consensus.cc:2376] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9. Re-sending same reply.
W20260709 13:02:51.348428  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.350430  4544 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:51.344018  5617 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:51.352246  5341 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:51.352990  5341 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:51.353533  5341 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:51.355829  5611 raft_consensus.cc:3055] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:51.356289  5611 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:51.357661  5611 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.361161  5621 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:51.340873  5466 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:51.363063  1681 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:51.339123  5592 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:02:51.366355  5641 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:51.366894  5641 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.367479  5641 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.366667  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:51.368786  1566 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:51.368618  5602 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:51.366406  1560 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:51.371359  1560 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:51.371548  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.372121  1560 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:51.373064  1560 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.376484  1566 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:02:51.353005  5647 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:51.379462  5604 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:51.380028  5604 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:51.381022  5604 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.383764  5611 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:51.384057  5603 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:51.385030  1563 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:51.339787  5629 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:51.395557  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.399711  1563 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:51.390833  1681 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:51.382661  5604 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:51.401607  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:51.402621  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:51.408030  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 108587us
I20260709 13:02:51.417261  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:51.420315  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:51.423321  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:51.427644  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:51.407821  5507 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:51.402643  5658 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:51.433135  1561 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:02:51.434362  1561 pending_rounds.cc:85] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 7
I20260709 13:02:51.434736  1561 pending_rounds.cc:107] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:02:51.435074  1561 raft_consensus.cc:2889] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:51.354578  5341 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:51.441152  2268 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:51.437552  1569 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:02:51.438333  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 144129us
W20260709 13:02:51.443912  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 132025us
I20260709 13:02:51.451644  1562 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 13: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:51.459825  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:51.463467  1564 raft_consensus.cc:2410] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 13:02:51.471062  5689 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.471781  5689 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.474077  5689 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:51.451702  5507 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:51.475818  5507 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.462738  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:51.485047  1560 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:51.496251  1562 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:51.499545  5574 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:51.509989  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:51.510195  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.511020  2303 raft_consensus.cc:2393] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 10: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:51.511648  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.512149  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:51.515215  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 74168.4us
I20260709 13:02:51.516487  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:51.516762  5654 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.517359  2302 raft_consensus.cc:2368] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 11.
I20260709 13:02:51.517391  5654 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:51.518360  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.518823  5580 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:51.519562  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.520462  2297 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:02:51.520462  2304 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:02:51.521422  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:51.522178  1567 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:51.522285  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.496201  5625 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:51.523320  2297 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:51.510854  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:51.531580  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.533746  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.536775  1532 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:51.535195  2298 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:51.494814  5595 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:51.542718  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.545395  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.546430  2299 raft_consensus.cc:2410] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:02:51.549513  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:51.554004  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.555362  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:51.508661  5438 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.558018  2305 raft_consensus.cc:2410] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:02:51.556763  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.541373  5601 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.579197  2298 raft_consensus.cc:2410] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:02:51.540737  5641 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.574833  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:51.599398  5475 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:51.599998  5475 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:51.600463  5475 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.603487  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.606361  2301 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:51.585538  5629 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:51.611228  5629 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.611682  5629 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:51.619139  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 66226us
I20260709 13:02:51.621845  5580 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:51.622865  5520 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:51.623407  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:51.625028  2300 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:51.626113  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:51.610683  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 79219us
I20260709 13:02:51.630369  1532 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:51.631228  5617 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:51.631649  5617 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.632165  5617 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.631721  5621 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:51.633759  5621 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:51.635365  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.636673  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2167) took 1315 ms. Trace:
I20260709 13:02:51.637085  1709 rpcz_store.cc:276] 0709 13:02:50.320846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.320958 (+   112us) service_pool.cc:224] Handling call
0709 13:02:51.636481 (+1315523us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b540023a854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:51.636620 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1795785984}
I20260709 13:02:51.636723  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:51.606637  1682 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:51.554378  5292 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:02:51.648357  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:51.591120  5654 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:51.652243  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 126476us
I20260709 13:02:51.658650  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.659591  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.662612  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:51.662799  2301 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:51.665196  2301 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:51.665596  2304 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:02:51.665866  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.666350  2301 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:51.667855  2301 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:02:51.669050  2301 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:02:51.670295  2301 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:02:51.671525  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.672185  2301 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:02:51.673435  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:51.674163  2301 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:51.675319  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.676260  1643 raft_consensus.cc:2368] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13.
I20260709 13:02:51.677134  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.677604  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.677981  1639 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:51.678609  1635 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:51.677186  5438 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:51.680254  5438 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:51.680678  5438 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:02:51.682196  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.687642  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.573300  5595 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.703414  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:51.710964  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:51.713476  2301 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:51.713888  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:51.714818  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 55367.5us
I20260709 13:02:51.716084  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:51.719372  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:51.719516  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.681063  5438 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:51.719893  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.721249  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:51.723001  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:51.735896  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 114641us
I20260709 13:02:51.736837  1564 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 12: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:51.724424  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:51.688115  5647 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:51.733330  5633 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:51.720716  2301 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:51.738016  1561 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.608104  5601 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.691426  5602 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:51.753916  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.713635  5654 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:51.763700  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.767297  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.768781  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.775365  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:51.776069  2301 raft_consensus.cc:2376] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9. Re-sending same reply.
I20260709 13:02:51.786396  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:51.787745  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:51.788501  2301 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:51.789240  5610 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:51.789655  2301 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:51.789852  5610 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:51.786265  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:51.791313  1709 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:51.791191  5610 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.793010  5610 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:51.793416  5601 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:51.795869  5629 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.807260  2298 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:51.810292  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:51.815367  1534 leader_election.cc:400] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 11.
I20260709 13:02:51.750682  5647 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.818168  1709 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:51.755455  5475 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:51.821031  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 69932us
W20260709 13:02:51.821444  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 239931us
I20260709 13:02:51.822142  1717 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:02:51.822634  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:51.823305  5629 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:51.825867  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2043) took 2454 ms. Trace:
W20260709 13:02:51.828915  5694 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.104s	user 0.000s	sys 0.002s
I20260709 13:02:51.834265  2297 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:51.817086  5602 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:51.838466  4544 consensus_queue.cc:1059] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:51.846450  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:51.770102  5243 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:51.850416  1561 raft_consensus.cc:2410] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:02:51.848057  2301 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:51.842329  1715 rpcz_store.cc:276] 0709 13:02:49.371180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.371766 (+   586us) service_pool.cc:224] Handling call
0709 13:02:51.332660 (+1960894us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:51.339538 (+  6878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:51.825543 (+486005us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54003dc7d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:51.825758 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4509120768}
I20260709 13:02:51.861979  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:51.864559  1569 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:51.860766  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:51.872025  1565 raft_consensus.cc:1240] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:51.875702  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.847962  5647 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:51.880806  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:51.881639  1638 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:51.881052  5243 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.886838  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:51.889278  1639 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:51.816066  5654 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.895416  1718 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:51.912009  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 176974us
W20260709 13:02:51.917496  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 67727us
I20260709 13:02:51.940670  1568 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:51.944545  5623 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:51.948966  5654 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:51.950639  5466 consensus_meta.cc:287] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.564s	user 0.001s	sys 0.010s
I20260709 13:02:51.951251  5466 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:51.952953  2300 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:51.954078  5466 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:51.958624  5243 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:51.960160  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 55592us
I20260709 13:02:51.962445  5420 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:51.963413  5676 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:51.896684  5641 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:51.969396  5687 log.cc:927] Time spent T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.311s	user 0.002s	sys 0.000s
I20260709 13:02:51.968506  5617 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:51.979137  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.882879  5595 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:51.987347  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:51.992137  1567 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:02:52.004657  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.007220  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 88704.5us
I20260709 13:02:52.010265  1717 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:02:52.012152  1643 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:02:52.013351  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:52.014014  1636 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
W20260709 13:02:52.014947  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.016553  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:52.017143  1642 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:52.006219  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:52.024020  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.053571  1715 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
W20260709 13:02:52.059979  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 51927.6us
W20260709 13:02:52.063043  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 148637us
I20260709 13:02:52.063246  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:52.065666  1639 raft_consensus.cc:1240] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:52.065459  1533 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:52.067071  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.067266  5621 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:52.068100  1717 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 10: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:52.069272  5689 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.069566  1533 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:52.065382  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:52.069974  5689 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.071766  5621 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:52.072912  5689 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:52.076038  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:52.078204  5415 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:52.079878  5602 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:52.080678  2297 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:52.079900  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:52.003676  5604 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:52.087365  1567 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:52.090063  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 235516us
I20260709 13:02:52.094058  2302 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:02:52.107165  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:52.108027  2302 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:52.109118  1642 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:02:52.110710  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:52.112859  2269 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:52.114096  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:52.115949  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:52.118225  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:52.120290  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:52.127856  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 239514us
I20260709 13:02:52.128633  1566 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:52.128804  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:52.122453  5603 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:52.136868  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:52.091320  5604 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.146327  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:52.131973  5580 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.008s	sys 0.012s
I20260709 13:02:52.128857  4544 consensus_queue.cc:1243] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:02:52.145253  5520 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.010s	sys 0.024s
I20260709 13:02:52.139245  5592 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:52.161248  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:52.163784  2268 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:52.166343  2302 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:52.167955  2297 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:02:52.154419  5580 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.008s	sys 0.013s
W20260709 13:02:52.173311  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 93394.1us
W20260709 13:02:52.158241  5520 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.010s	sys 0.024s
I20260709 13:02:52.156562  5700 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.186362  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:52.195815  1713 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:52.190713  1709 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:52.190191  1716 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:52.193184  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.190055  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:52.199393  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.183677  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.225935  1569 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:52.227433  1711 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:52.227947  1639 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:02:52.227746  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 53503.8us
I20260709 13:02:52.230816  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:52.157842  5603 raft_consensus.cc:697] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:52.194221  4544 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:52.235745  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:52.236759  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:52.252462  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.207145  5704 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:02:52.269498  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.199734  5705 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:52.160250  5475 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:52.209676  1466 catalog_manager.cc:5697] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:52.266237  2297 raft_consensus.cc:2393] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:52.197937  5636 raft_consensus.cc:3055] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:52.192615  5604 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:52.271618  1683 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:52.276284  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:52.173583  5580 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:52.278469  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:52.279027  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:52.286602  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 132569us
I20260709 13:02:52.282398  1566 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:02:52.208525  5617 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:52.293439  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 65922.9us
I20260709 13:02:52.241694  1681 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:52.270104  5574 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:52.229374  5647 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:52.211550  5676 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:02:52.293452  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:02:52.270779  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 115744us
I20260709 13:02:52.310070  1683 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:52.295331  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 53732.8us
I20260709 13:02:52.192607  5465 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:52.296803  2268 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:52.317637  1681 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:52.183509  5712 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:52.292817  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:52.270619  5520 consensus_queue.cc:1243] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:02:52.280786  5636 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:52.198040  5710 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.137s
I20260709 13:02:52.264756  5469 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
W20260709 13:02:52.318569  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 49499.2us
I20260709 13:02:52.320673  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.333976  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:52.319267  1713 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:52.214521  5689 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:52.289148  5629 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:52.278138  5461 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:02:52.291127  5438 consensus_meta.cc:287] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.001s
I20260709 13:02:52.336165  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.314601  5420 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:52.315860  4544 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:52.281657  5466 consensus_queue.cc:1059] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:02:52.344833  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:52.314883  5641 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:52.337680  1714 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:02:52.344612  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.347013  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.333973  5595 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:52.359889  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.395571  5710 raft_consensus.cc:3055] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:52.396214  5710 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:52.396714  1636 raft_consensus.cc:3055] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:52.397104  1636 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:52.397192  5710 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.397926  1636 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.398980  1636 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:52.399305  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 74731.9us
I20260709 13:02:52.399992  1641 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:52.401153  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:52.402016  1641 raft_consensus.cc:2393] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:52.402451  1609 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:52.340257  5465 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:52.414027  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:52.403293  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:52.288688  5697 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:02:52.403389  5580 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:52.431624  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.341388  5420 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:52.404314  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:52.404488  5341 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:02:52.404620  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 69293.9us
I20260709 13:02:52.405277  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:52.404326  5507 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:02:52.403434  5718 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:52.402853  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:52.411736  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:52.288800  5700 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.413115  1636 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:02:52.410571  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 64881.6us
I20260709 13:02:52.432605  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.420871  1638 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:52.326645  5292 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 13:02:52.416953  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 200027us
I20260709 13:02:52.398841  5710 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:52.433547  1643 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:52.434917  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.455309  1716 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:52.423554  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.436436  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:52.437672  1644 raft_consensus.cc:2410] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:02:52.350195  5602 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:52.348433  5641 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:52.459338  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.404062  1710 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 6.
I20260709 13:02:52.341065  5438 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:52.449262  1639 raft_consensus.cc:2410] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:02:52.452150  1713 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:52.418138  5465 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.438072  5420 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:52.488046  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:52.492708  1714 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:52.493605  1713 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:52.443341  5689 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.496183  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:52.505913  1713 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:02:52.346441  5603 consensus_queue.cc:237] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.516500  1718 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:52.522368  1534 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:52.523871  1534 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:52.524688  1534 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:52.520244  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:02:52.523025  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.483543  5641 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:52.486536  5469 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.434357  5636 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.353607  5712 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:52.488099  5589 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.488476  5507 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:52.488194  5721 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.488886  5465 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:52.529633  1640 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:52.530376  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:52.525888  1716 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:52.531783  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:52.509991  5610 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:02:52.545805  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 228363us
I20260709 13:02:52.533777  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:52.525892  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:52.550417  1709 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:52.530685  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 61936.2us
I20260709 13:02:52.519419  1682 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:52.564844  1532 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:02:52.530375  5713 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
I20260709 13:02:52.576594  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:52.531083  5475 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:52.530853  5676 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:52.534346  5623 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:52.578135  1713 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:02:52.582103  1567 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:52.531215  5520 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:52.540263  5621 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:52.532042  5700 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:52.530623  5617 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:52.544214  5720 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:52.588068  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:52.594389  1533 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:52.512526  5466 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:02:52.591339  5292 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:52.525892  5611 consensus_queue.cc:1059] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:02:52.515195  5689 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.520843  5658 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
W20260709 13:02:52.596719  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 149741us
I20260709 13:02:52.599248  1638 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:52.602861  1637 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:52.602993  5617 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.587412  5475 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:52.603852  5415 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:52.604310  5520 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.605382  5621 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:52.605752  5716 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:02:52.606034  5724 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:52.595237  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.612996  5507 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.531973  5697 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:02:52.606483  5720 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:52.667702  5720 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:52.669260  5720 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:52.573927  5721 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:52.532258  5722 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.164s
W20260709 13:02:52.616694  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 90920.9us
W20260709 13:02:52.626719  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 61918.2us
I20260709 13:02:52.587532  5623 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:52.526208  5602 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.568993  5589 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.556005  5469 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:52.686005  5623 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:52.687481  5623 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:52.687731  5469 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:52.626564  5617 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:52.604347  5438 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:52.637663  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:52.691668  1711 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:52.692973  5602 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:52.693827  5726 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.105s
I20260709 13:02:52.695123  5589 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:52.655356  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.626902  5604 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.657244  1715 raft_consensus.cc:1240] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:02:52.662612  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 83591.2us
I20260709 13:02:52.652068  5415 raft_consensus.cc:697] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:52.666321  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 925) took 1115 ms. Trace:
I20260709 13:02:52.634250  5475 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:52.631654  5466 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:52.675505  1716 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
W20260709 13:02:52.698448  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 388486us
W20260709 13:02:52.639994  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 237660us
I20260709 13:02:52.646697  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:52.629213  5676 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:52.720816  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:52.710573  2299 rpcz_store.cc:276] 0709 13:02:51.551098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.555249 (+  4151us) service_pool.cc:224] Handling call
0709 13:02:51.579110 (+ 23861us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:51.987731 (+408621us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:51.987761 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:51.989620 (+  1859us) log.cc:844] Serialized 26 byte log entry
0709 13:02:52.136585 (+146965us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:52.136662 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.136924 (+   262us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:52.666074 (+529150us) raft_consensus.cc:1685] finished
0709 13:02:52.666100 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:52.666234 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":429339,"raft.queue_time_us":400376,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":4333,"thread_start_us":136297,"threads_started":1,"wal-append.queue_time_us":190663}
W20260709 13:02:52.723218  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 234245us
I20260709 13:02:52.593118  4544 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:52.726894  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:52.724684  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 88857.9us
I20260709 13:02:52.734795  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:52.729776  1682 leader_election.cc:400] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13.
W20260709 13:02:52.746635  1607 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.795s (SENT) [suppressed 57 similar messages]
I20260709 13:02:52.748793  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:52.719940  5466 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:52.751654  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:52.752784  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:52.757022  1711 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:02:52.733714  5602 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:52.726562  5574 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:52.758841  2297 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: [7.8-8.9]
I20260709 13:02:52.739111  5697 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:52.716598  5604 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:52.763896  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 90268.7us
I20260709 13:02:52.767376  1609 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:52.744095  5724 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:52.727525  5654 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:02:52.743108  5722 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:02:52.742939  5438 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:02:52.761421  5712 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:52.763218  5697 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:52.782223  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 58069.7us
W20260709 13:02:52.805090  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 58506us
I20260709 13:02:52.769539  5292 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:52.781340  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 108655us
I20260709 13:02:52.771878  5724 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:52.776437  5602 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:52.770318  5734 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:02:52.853758  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:52.758750  5415 consensus_queue.cc:237] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.790212  5711 consensus_queue.cc:1059] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:02:52.855712  5711 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:52.856098  5711 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:52.763293  5739 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:52.856956  5711 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.767520  5420 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:52.840749  5697 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:52.781629  5603 consensus_queue.cc:1059] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:02:52.857441  5724 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:52.892381  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:52.858678  5712 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:02:52.811584  5466 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.879601  5629 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:52.851197  5292 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.897624  1643 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:52.886433  5739 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.906265  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:52.906456  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:52.907058  2300 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEARNER]: Advancing to term 12
I20260709 13:02:52.908114  2303 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:02:52.885104  5520 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:52.910236  1640 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:02:52.910717  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 67359.9us
I20260709 13:02:52.911936  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:52.914448  5715 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:02:52.915225  5740 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:02:52.973157  2301 raft_consensus.cc:2410] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:02:52.916350  5733 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:52.916901  5731 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.148s
I20260709 13:02:52.977959  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:52.921764  5473 raft_consensus.cc:3055] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:52.988690  5473 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:52.989714  5473 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:52.991251  5520 raft_consensus.cc:2764] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:02:52.996191  5292 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:53.001935  5617 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.010242  5420 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:53.011510  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:53.012213  1717 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:53.011843  5641 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:53.014081  5641 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:52.952751  5602 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.017468  5604 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:53.020001  1717 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:53.020730  1710 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:53.021803  5733 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.024382  5733 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:52.925753  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:53.025570  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:53.025751  2304 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:53.026448  1710 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:52.895206  5689 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:52.909597  5711 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:53.030661  1563 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:02:52.940109  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.037611  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:53.039004  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:52.888084  5738 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:02:53.041589  1562 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:02:53.042742  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:53.043464  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.966995  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 125045us
I20260709 13:02:52.928481  5722 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:53.044986  1533 leader_election.cc:400] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 12. Current term is 13.
W20260709 13:02:52.976811  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 70434us
I20260709 13:02:52.978904  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:52.979385  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 189238us
W20260709 13:02:53.005713  5710 consensus_meta.cc:287] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.001s
I20260709 13:02:52.983314  5746 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:52.987578  5731 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:53.043408  1564 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
W20260709 13:02:52.918246  5730 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:02:53.049540  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:52.981134  1682 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:53.054522  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2530) took 1296 ms. Trace:
I20260709 13:02:53.064790  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:53.064751  2268 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:53.067914  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 89076.1us
I20260709 13:02:53.068888  1635 rpcz_store.cc:276] 0709 13:02:51.758599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.758739 (+   140us) service_pool.cc:224] Handling call
0709 13:02:53.050822 (+1292083us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b5400155754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:53.054442 (+  3620us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1858989312}
I20260709 13:02:53.079355  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:53.080034  1644 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:52.962903  5721 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:53.076951  1683 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:53.082048  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:53.085549  2303 raft_consensus.cc:2410] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:52.926981  5739 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:53.087730  1713 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:02:53.090157  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:53.090530  1609 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:53.074944  5520 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:53.078173  5473 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:53.069360  5731 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:52.972533  5580 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.122s
I20260709 13:02:53.084429  5689 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:53.093071  5601 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:53.111630  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.106412  5747 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:02:53.106570  5610 raft_consensus.cc:3055] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:53.106845  5629 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:53.095446  5604 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:53.120030  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:53.118798  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.099656  5654 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:53.069526  5420 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:02:53.080698  5722 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:02:53.099900  5710 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:53.075338  5597 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:02:53.124225  5461 raft_consensus.cc:3055] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:53.086982  5438 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
W20260709 13:02:53.139335  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 158297us
I20260709 13:02:53.140447  1682 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:53.117983  5610 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:53.122305  5507 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.150007  5420 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.098740  5721 consensus_queue.cc:1059] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:02:53.133842  5654 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.075176  1710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:02:53.136003  5710 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.136246  5461 raft_consensus.cc:740] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:53.107452  5743 consensus_queue.cc:1059] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:02:53.253136  1710 raft_consensus.cc:606] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:53.265902  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 127213us
W20260709 13:02:53.272697  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.258091  5720 consensus_meta.cc:287] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.588s	user 0.008s	sys 0.005s
I20260709 13:02:53.253715  5654 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:53.304790  1607 leader_election.cc:336] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.036s (SENT)
I20260709 13:02:53.305452  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:53.256594  5710 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:53.269114  5629 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:53.256369  5733 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:53.321283  5731 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.343973  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:53.346521  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2542) took 1053 ms. Trace:
I20260709 13:02:53.347740  1636 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:53.350894  1642 rpcz_store.cc:276] 0709 13:02:52.293013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.293187 (+   174us) service_pool.cc:224] Handling call
0709 13:02:53.346277 (+1053090us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b5400123c54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:53.346451 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1891675520}
W20260709 13:02:53.356190  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 51456.2us
I20260709 13:02:53.251840  5420 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.360677  1681 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:53.273177  5712 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:02:53.258533  5507 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.372040  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:53.372186  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:53.326459  5689 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.259791  5726 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
W20260709 13:02:53.360365  5475 consensus_meta.cc:287] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.636s	user 0.012s	sys 0.001s
I20260709 13:02:53.417927  5475 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.418762  5574 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.419591  5689 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.381817  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:53.421799  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:02:53.422096  5689 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:53.365880  5633 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.422564  1717 raft_consensus.cc:606] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:53.423159  5633 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.376600  5714 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.426545  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:53.390110  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:53.427453  5714 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.428494  1565 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:53.429760  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:53.376410  5602 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:53.431861  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.388899  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:53.432885  2299 raft_consensus.cc:2376] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10. Re-sending same reply.
I20260709 13:02:53.400643  1635 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:02:53.402843  5465 consensus_meta.cc:287] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.819s	user 0.005s	sys 0.005s
I20260709 13:02:53.366532  5617 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:53.444293  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:53.444442  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2056) took 1256 ms. Trace:
I20260709 13:02:53.445149  1568 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:53.446117  5466 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:53.446717  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2066) took 1237 ms. Trace:
I20260709 13:02:53.447008  1567 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:53.448323  1566 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:53.447188  1714 rpcz_store.cc:276] 0709 13:02:52.208847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.493781 (+284934us) service_pool.cc:224] Handling call
0709 13:02:52.494136 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.695241 (+201105us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:52.772879 (+ 77638us) log.cc:844] Serialized 14 byte log entry
0709 13:02:52.909586 (+136707us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:52.909640 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:52.909691 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.446521 (+536830us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400136ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:53.446564 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.446665 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4992,"prepare.queue_time_us":146328,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":13156,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3552,"spinlock_wait_cycles":461801856,"thread_start_us":135574,"threads_started":2,"wal-append.queue_time_us":142336}
I20260709 13:02:53.449508  5465 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.418751  5733 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.404510  5610 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.355720  5722 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:53.418387  5726 raft_consensus.cc:993] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:02:53.444919  1718 rpcz_store.cc:276] 0709 13:02:52.187820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.194453 (+  6633us) service_pool.cc:224] Handling call
0709 13:02:52.194755 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.522337 (+327582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.444192 (+921855us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b5400474354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:53.444351 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":739518336}
I20260709 13:02:53.299857  5721 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:02:53.419541  5574 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.391824  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:53.440761  2300 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:02:53.376984  1607 leader_election.cc:336] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.199s (SENT)
I20260709 13:02:53.491458  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:53.443387  1569 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:02:53.444960  1564 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:53.495091  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:53.426002  5633 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:53.456861  2302 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:53.458415  1532 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:02:53.498553  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.403613  5731 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:53.504392  1714 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:02:53.504427  5617 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:53.506127  5739 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:53.472067  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:53.507525  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:53.476224  5610 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:53.510345  5739 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:02:53.477237  5724 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.587s	user 0.011s	sys 0.000s
W20260709 13:02:53.477525  5753 log.cc:927] Time spent T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 13:02:53.478860  5752 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 13:02:53.479712  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:53.510900  5739 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:53.489168  2296 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:53.496287  1638 raft_consensus.cc:2376] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10. Re-sending same reply.
W20260709 13:02:53.503247  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.556990  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 85562.1us
I20260709 13:02:53.506305  5617 raft_consensus.cc:697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:53.484712  5507 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:53.511214  5724 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.386098  5461 consensus_queue.cc:260] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.550062  5739 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:53.514626  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:53.552152  5743 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.470954  5465 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:53.554849  2297 raft_consensus.cc:2410] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:02:53.555189  5689 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.521034  5726 raft_consensus.cc:1081] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:02:53.565761  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:53.585139  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:53.585954  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:53.570056  1565 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:53.571738  1567 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260709 13:02:53.575150  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.575066  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 207190us
I20260709 13:02:53.588147  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:53.590235  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.576617  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:53.557447  5676 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.026s	sys 0.031s
W20260709 13:02:53.568545  5711 consensus_meta.cc:287] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.538s	user 0.005s	sys 0.007s
I20260709 13:02:53.568409  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:53.594409  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.598276  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.611724  1640 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:53.612167  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:53.616284  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:53.591928  5676 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.026s	sys 0.032s
W20260709 13:02:53.622799  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.567423  5633 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:02:53.610146  5461 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:53.646560  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:53.647387  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:53.652323  1638 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:53.655318  2269 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:53.660729  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:53.661963  1638 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:02:53.675926  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.681619  1644 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:53.607649  5689 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.585412  5743 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.686291  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:53.663539  5604 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:53.689894  5604 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.690356  5604 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:53.690440  5724 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:53.693024  5689 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:53.695436  2301 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:02:53.695869  2301 pending_rounds.cc:85] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 9
I20260709 13:02:53.696089  2301 pending_rounds.cc:107] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:02:53.696306  2301 raft_consensus.cc:2889] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:53.687328  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:53.697896  1638 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:02:53.685871  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 178445us
I20260709 13:02:53.694175  5710 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.708981  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:53.710079  1644 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:53.712036  1564 raft_consensus.cc:1240] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:53.703191  5574 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:53.735773  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:53.736125  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:53.737151  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:53.682629  5617 consensus_queue.cc:237] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.755960  1642 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:02:53.759092  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:53.773813  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.782377  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:02:53.758687  5697 consensus_meta.cc:287] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.876s	user 0.006s	sys 0.007s
I20260709 13:02:53.783744  1681 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:53.817780  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 103199us
I20260709 13:02:53.817219  5726 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:53.794605  5602 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:53.823252  5724 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:02:53.837700  1569 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:53.823277  5597 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:02:53.839342  5636 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:53.824983  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:53.842706  1562 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "520f39ecc282462782487a3651e28b44"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:02:53.847692  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:53.863376  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:53.890189  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:53.846431  5602 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.795959  5697 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:53.842787  5733 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:53.915422  1565 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:02:53.832428  5711 consensus_queue.cc:1059] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:02:53.859933  5721 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:02:53.994475  1561 raft_consensus.cc:3055] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:53.994861  1561 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:53.995281  5721 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.995693  1561 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.993721  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1104) took 1242 ms. Trace:
I20260709 13:02:53.996008  5721 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.996621  2302 rpcz_store.cc:276] 0709 13:02:52.751454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.752486 (+  1032us) service_pool.cc:224] Handling call
0709 13:02:53.993436 (+1240950us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b5400686ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:53.993621 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1616844672}
W20260709 13:02:54.005412  5754 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.636s	user 0.002s	sys 0.002s
I20260709 13:02:54.012563  5475 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:54.018326  1567 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:54.023383  5611 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.024225  5611 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.025624  5736 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.027457  5720 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:54.028012  5720 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.028332  5720 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:54.033475  5654 consensus_meta.cc:287] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.706s	user 0.004s	sys 0.005s
I20260709 13:02:54.034037  5654 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:53.900102  5507 consensus_queue.cc:1059] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.135s
I20260709 13:02:54.038214  2305 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53448
I20260709 13:02:54.038554  5469 raft_consensus.cc:3055] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:54.039551  5469 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:54.039671  5736 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.040692  5469 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.044723  5697 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:54.052155  5714 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:54.053059  5466 consensus_meta.cc:287] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.004s
I20260709 13:02:54.053900  2303 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:53.907793  5602 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.039104  2305 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:54.055677  2305 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:54.035096  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2557) took 1590 ms. Trace:
I20260709 13:02:54.058666  1636 rpcz_store.cc:276] 0709 13:02:52.444791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.529805 (+ 85014us) service_pool.cc:224] Handling call
0709 13:02:53.347545 (+817740us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.351050 (+  3505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.034843 (+683793us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b5400123c50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:54.035012 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1879895936}
I20260709 13:02:54.062583  1466 catalog_manager.cc:5697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:02:53.998746  5721 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:54.084815  5776 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 13:02:54.050098  5420 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:54.130808  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 305888us
W20260709 13:02:54.105083  5754 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 13:02:54.136101  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 274138us
I20260709 13:02:54.135349  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:54.147222  5721 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:54.147789  5721 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.148193  5721 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:54.151110  5697 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:54.151737  4544 raft_consensus.cc:3055] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:54.152236  4544 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:02:54.153236  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.147229  5580 consensus_queue.cc:1059] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:02:54.154111  5580 raft_consensus.cc:3055] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:54.154551  5580 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:54.155903  1561 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:54.157312  5611 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:54.111100  5759 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.015s	sys 0.050s
I20260709 13:02:54.167563  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2072) took 1398 ms. Trace:
I20260709 13:02:54.134714  5589 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.168227  5759 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.015s	sys 0.051s
I20260709 13:02:54.172621  1709 rpcz_store.cc:276] 0709 13:02:52.769293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.770421 (+  1128us) service_pool.cc:224] Handling call
0709 13:02:52.770773 (+   352us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:52.811096 (+ 40323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:52.811131 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:52.867702 (+ 56571us) log.cc:844] Serialized 52 byte log entry
0709 13:02:53.067632 (+199930us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:53.115833 (+ 48201us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.116037 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.116110 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.004585 (+888475us) spinlock_profiling.cc:243] Waited 38.3 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:02:54.136852 (+132267us) raft_consensus.cc:1685] finished
0709 13:02:54.167237 (+ 30385us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.167463 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52254,"prepare.queue_time_us":34853,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":3120,"raft.queue_time_us":71274,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":26767,"spinlock_wait_cycles":88072960,"thread_start_us":173541,"threads_started":1,"wal-append.queue_time_us":273753}
I20260709 13:02:54.214037  5602 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:54.218161  5602 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:54.230312  5461 consensus_meta.cc:287] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.578s	user 0.006s	sys 0.002s
W20260709 13:02:54.235909  5784 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:02:54.155800  5580 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.240072  5580 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:54.247125  5654 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:54.187812  5743 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:54.168627  5710 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:54.198477  5604 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.348133  1561 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:02:54.348758  1561 pending_rounds.cc:85] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 8
I20260709 13:02:54.349268  1561 pending_rounds.cc:107] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:02:54.349771  1561 raft_consensus.cc:2889] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:54.250880  5589 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.358440  2305 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:54.370188  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 549474us
I20260709 13:02:54.370256  2305 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:54.318202  5721 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:54.385512  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 208887us
W20260709 13:02:54.376816  5745 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:02:54.399029  5589 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:54.403909  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.404240  5797 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.411607  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.372957  5720 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:54.417167  5720 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:54.418303  5721 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:54.403278  5636 consensus_meta.cc:287] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.002s
I20260709 13:02:54.442631  1635 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:02:54.443177  1635 pending_rounds.cc:85] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 9
I20260709 13:02:54.443481  1635 pending_rounds.cc:107] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:02:54.443778  1635 raft_consensus.cc:2889] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:54.453589  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 1.31327e+06us
I20260709 13:02:54.456147  5461 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.456866  5461 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:54.458177  5791 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.310s	user 0.003s	sys 0.000s
I20260709 13:02:54.459682  5461 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:54.465691  2303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1107) took 1401 ms (client timeout 1832 ms). Trace:
I20260709 13:02:54.470084  5636 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.471100  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.470752  5636 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.454672  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:54.466480  2303 rpcz_store.cc:269] 0709 13:02:53.064407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.103138 (+ 38731us) service_pool.cc:224] Handling call
0709 13:02:54.465398 (+1362260us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b5400642154. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:54.465595 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2141674368}
I20260709 13:02:54.411573  5797 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.488009  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:54.488667  2299 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
W20260709 13:02:54.489727  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.491184  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.496652  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.497905  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:54.503479  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.480048  5721 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.520498  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:54.527261  2305 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:02:54.483398  5589 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:54.497678  5743 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.508144  5243 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:54.510776  1682 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:54.511541  5795 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:54.522962  2298 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:02:54.532431  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:54.523193  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:54.525305  5636 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:54.525724  5604 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:54.527899  2302 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEARNER]: Advancing to term 13
I20260709 13:02:54.551645  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:54.559898  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:54.525360  5797 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:54.581526  2305 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:02:54.592662  2302 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:54.593961  5292 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.594616  5292 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.593582  5602 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:54.596619  5602 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:54.597062  5602 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:54.597283  5292 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:54.599210  5797 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.599747  5797 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.631373  1709 raft_consensus.cc:1240] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:54.632578  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:54.632922  1713 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:54.633368  1714 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:54.640322  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:54.641213  1709 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:54.643847  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:54.644763  1713 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:54.650756  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:54.652439  1713 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:54.653401  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:54.655556  1709 raft_consensus.cc:3055] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:54.656059  1709 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:54.529671  5721 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:54.548660  5243 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:54.666496  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 155834us
I20260709 13:02:54.662397  1709 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.540797  5743 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.669075  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:54.669739  1716 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.657632  5292 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:54.663527  5595 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.672902  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:54.671383  2268 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:54.670629  5689 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:54.686652  1716 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:54.676739  5595 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.695465  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 874) took 1296 ms. Trace:
W20260709 13:02:54.704368  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.732110  1640 rpcz_store.cc:276] 0709 13:02:53.398566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.399345 (+   779us) service_pool.cc:224] Handling call
0709 13:02:53.399707 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.612416 (+212709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.695145 (+1082729us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b5400155254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:54.695384 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482038016}
I20260709 13:02:54.698318  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2197) took 1261 ms. Trace:
W20260709 13:02:54.705884  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 334961us
I20260709 13:02:54.702653  1533 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:54.702991  5794 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.703204  5420 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.706424  5804 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.678241  5461 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:54.729812  1534 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:54.776502  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:54.776876  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 47152.6us
I20260709 13:02:54.707826  1562 raft_consensus.cc:606] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:54.777872  1562 raft_consensus.cc:614] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:54.675963  5292 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.734566  1560 rpcz_store.cc:276] 0709 13:02:53.436771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.436892 (+   121us) service_pool.cc:224] Handling call
0709 13:02:53.443091 (+  6199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.443286 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:53.625423 (+182137us) log.cc:844] Serialized 14 byte log entry
0709 13:02:54.108760 (+483337us) log.cc:844] Serialized 14 byte log entry
0709 13:02:54.697420 (+588660us) spinlock_profiling.cc:243] Waited 5.95 ms on lock 0x7b1800130d20. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb2ec0a 00007fd7a826e219 00007fd7a5b7558f 00007fd79ebb4571 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd
0709 13:02:54.697966 (+   546us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:54.698022 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:54.698067 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.698134 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.698254 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":707262,"prepare.queue_time_us":249002,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":78,"raft.queue_time_us":671369,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":73721,"spinlock_wait_cycles":13694336,"thread_start_us":577176,"threads_started":3,"wal-append.queue_time_us":451548}
I20260709 13:02:54.706974  5623 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:54.780406  1710 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:54.752311  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:54.702531  5745 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:54.754956  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 87613.4us
I20260709 13:02:54.753829  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:54.757860  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:54.704463  5785 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.785858  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:54.785996  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:54.757696  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:54.766987  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1159) took 1208 ms. Trace:
W20260709 13:02:54.779255  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 239722us
I20260709 13:02:54.756098  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:54.719337  5415 consensus_queue.cc:1059] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
W20260709 13:02:54.781867  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 79288.7us
I20260709 13:02:54.788965  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:54.782495  1568 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:54.786756  1564 raft_consensus.cc:2410] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:02:54.781193  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:54.790299  5749 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:54.780884  5420 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:54.793591  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:54.794278  5292 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:54.787523  2301 rpcz_store.cc:276] 0709 13:02:53.559024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.616391 (+ 57367us) service_pool.cc:224] Handling call
0709 13:02:53.616771 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:53.696715 (+ 79944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.766654 (+1069939us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b54003a2354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:54.766887 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155878784}
I20260709 13:02:54.776147  5731 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.799553  1717 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:02:54.799721  5731 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.800223  5595 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:54.707918  5689 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.756160  5733 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:54.794663  5420 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:54.808898  5292 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:54.809432  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:54.838258  1717 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:54.787532  5475 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:02:54.801937  5676 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:54.843652  5475 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:54.809460  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:54.810053  5602 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:54.810544  5415 raft_consensus.cc:3055] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:54.812606  5641 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:54.816025  1533 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:54.804265  5785 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.820804  5736 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:54.823443  5689 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:54.782006  5739 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:54.855945  5811 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:02:54.825523  5731 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:54.825739  5813 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:02:54.858680  5733 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.861397  5733 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:54.827702  5797 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:54.831758  5790 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.676s	user 0.004s	sys 0.000s
I20260709 13:02:54.807158  1565 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
W20260709 13:02:54.875639  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 164832us
W20260709 13:02:54.835846  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 53938.3us
I20260709 13:02:54.833770  1683 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:54.784343  5623 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:54.800853  5794 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:54.844601  1716 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:02:54.848984  5243 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:54.845497  5415 raft_consensus.cc:740] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:54.851300  5743 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:54.852160  5721 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:54.852830  1563 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:54.804373  5507 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:54.867638  1569 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:54.833019  5745 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:54.907769  5745 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:54.788362  5804 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.843935  5676 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:54.847483  5475 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:54.913925  5804 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:54.877110  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 43513.2us
I20260709 13:02:54.917162  5507 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:54.917191  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:54.917984  5415 consensus_queue.cc:260] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:54.919014  1712 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:02:54.919584  5415 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:54.920380  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:54.922290  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:54.922749  1564 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:54.923108  5794 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:54.880095  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:54.872275  4544 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.923816  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:54.889931  5787 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:54.924422  4544 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.878585  5623 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.925707  2297 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:02:54.898506  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.926761  1561 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:54.886585  5819 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:54.927865  5819 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:54.922969  2300 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
W20260709 13:02:54.903438  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 87492.8us
I20260709 13:02:54.928949  5819 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.913861  5676 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:54.931280  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:54.932042  1565 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:54.930682  1533 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:54.933727  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57836 (request call id 18) took 1140 ms. Trace:
W20260709 13:02:54.934100  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 57167.9us
I20260709 13:02:54.936595  5292 consensus_queue.cc:1059] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:54.937803  5243 raft_consensus.cc:3055] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:54.938256  5243 raft_consensus.cc:740] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:54.939097  5243 consensus_queue.cc:260] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:54.940924  5243 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:54.876406  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:54.943504  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:54.922351  2299 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:54.944983  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:54.945996  5602 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:54.946173  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:54.946964  2302 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:54.941246  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:54.947861  1563 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:54.950390  5469 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:54.953023  5689 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:54.953425  5743 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:02:54.953051  5799 log.cc:927] Time spent T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.757s	user 0.001s	sys 0.001s
I20260709 13:02:54.954193  5743 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:54.957772  5743 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:54.958596  5507 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:54.959937  1466 catalog_manager.cc:5697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 3 to 7, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:54.962994  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:54.963848  1643 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:54.964238  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:54.964874  1640 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:54.965168  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:54.965787  1639 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:54.887524  5617 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:02:54.966864  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:54.967578  5755 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:54.967965  5755 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:54.967308  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:54.968670  1636 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:54.968887  5755 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:54.969991  5755 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:54.972258  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.949036  1683 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:54.962332  1466 catalog_manager.cc:5697] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 4 to 8, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 8 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:54.976553  2269 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:54.977486  1561 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
W20260709 13:02:54.978379  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.978564  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:54.979483  1638 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
W20260709 13:02:54.979575  1607 leader_election.cc:336] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.795s (SENT)
W20260709 13:02:54.982124  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.982542  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:54.985294  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:54.986647  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 110327us
I20260709 13:02:54.934324  1562 rpcz_store.cc:276] 0709 13:02:53.815290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.823843 (+  8553us) service_pool.cc:224] Handling call
0709 13:02:54.933457 (+1109614us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b5400179dd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c9b853 00007fd7a6c6960a 00007fd7a01ef7d4 00007fd7a01ef762 00007fd7a01ef691 00007fd7a01ef60c 00007fd7a01ee8c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:54.933606 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1904449408}
I20260709 13:02:54.987644  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:54.980250  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:54.990320  1644 raft_consensus.cc:2410] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:02:54.993510  1569 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:02:55.000381  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:55.004603  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:55.005861  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:55.008749  1609 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:55.010741  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:55.023041  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:55.080783  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.090845  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:55.092181  1640 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:55.093963  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2120) took 1598 ms. Trace:
I20260709 13:02:55.094419  1635 rpcz_store.cc:276] 0709 13:02:53.495426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.495629 (+   203us) service_pool.cc:224] Handling call
0709 13:02:53.556475 (+ 60846us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.444372 (+887897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.093721 (+649349us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b5400124654. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:55.093892 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2036956416}
I20260709 13:02:55.099651  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:55.100695  1561 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:55.104440  1639 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
W20260709 13:02:55.108589  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 129066us
W20260709 13:02:55.108968  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.110782  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:55.094830  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 157974us
I20260709 13:02:55.113684  1682 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:55.118057  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 187465us
I20260709 13:02:55.119109  1533 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:55.096226  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 109376us
I20260709 13:02:55.121044  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:55.123564  1533 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:55.123690  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.127241  1682 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:55.127610  1711 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:02:55.128429  1711 raft_consensus.cc:606] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:55.129540  1565 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
W20260709 13:02:54.990358  5438 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.029s	sys 0.037s
I20260709 13:02:55.128304  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:54.988117  5797 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.987712  5721 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.149075  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:54.987612  5736 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:55.169610  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:55.152032  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 143390us
I20260709 13:02:55.171433  1609 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:55.152398  4544 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:55.172993  4544 raft_consensus.cc:697] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:55.161877  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.173911  1716 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:02:55.149603  1561 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:55.159730  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:55.167249  1607 leader_election.cc:336] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.858s (SENT)
W20260709 13:02:55.188938  1607 leader_election.cc:336] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.903s (SENT)
W20260709 13:02:55.189750  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.189473  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.189121  1466 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 6 to 8, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:02:55.139735  5438 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.029s	sys 0.037s
I20260709 13:02:55.175186  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:02:55.224938  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 65339us
I20260709 13:02:55.231501  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:55.235128  1682 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:55.120749  5787 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:55.236922  4544 consensus_queue.cc:237] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:55.133184  5731 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:55.132829  5461 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:55.234879  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 111400us
I20260709 13:02:55.255911  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:55.140280  5762 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:55.245170  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 70100.1us
I20260709 13:02:55.091292  5595 consensus_queue.cc:1059] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.134s
I20260709 13:02:55.224633  5724 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:55.135202  5589 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:55.248642  1466 catalog_manager.cc:5697] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:02:55.162376  5714 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:55.161108  5420 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:55.230824  5507 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:02:55.278868  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.228088  5785 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:55.214052  5611 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:55.230638  5803 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:55.353648  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:55.156018  5292 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.191s
I20260709 13:02:55.135913  5602 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.125s
I20260709 13:02:55.266045  5762 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:02:55.359459  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.194123  5721 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:55.254801  5810 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:02:55.364467  1712 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c129438e9d58493a93890208b25765cf"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
W20260709 13:02:55.377054  1710 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.003s	sys 0.033s
I20260709 13:02:55.377158  1712 raft_consensus.cc:606] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:55.377554  1710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.003s	sys 0.033s
I20260709 13:02:55.377635  1712 raft_consensus.cc:614] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:55.378499  1561 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:55.253526  5760 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:55.255079  5629 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:55.255782  5604 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:55.191099  5797 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:55.266383  5794 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:55.228240  5736 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:55.367183  5785 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:55.181331  5743 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.203s
I20260709 13:02:55.356333  5731 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:55.381258  5461 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:55.361971  5420 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:55.362542  5676 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:55.354228  5822 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:02:55.408913  5732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.021s	sys 0.020s
W20260709 13:02:55.415949  5732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.021s	sys 0.020s
W20260709 13:02:55.435559  1560 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.049s	sys 0.023s
W20260709 13:02:55.437258  1560 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.049s	sys 0.023s
W20260709 13:02:55.442662  2301 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.007s	sys 0.065s
W20260709 13:02:55.443497  2301 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.007s	sys 0.065s
I20260709 13:02:55.354058  5724 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:02:55.370016  5621 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.019s	sys 0.022s
I20260709 13:02:55.364516  5611 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.254246  5787 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:55.391858  1563 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:55.394213  1609 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:55.370167  1568 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.022s	sys 0.023s
W20260709 13:02:55.467501  1568 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.023s	sys 0.023s
I20260709 13:02:55.395447  1638 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:55.387050  5604 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.384104  5760 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.376778  5762 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:55.367424  5714 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:55.385715  5641 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:55.466544  5611 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:55.463423  5734 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:55.359169  1566 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.010s	sys 0.050s
I20260709 13:02:55.428010  5822 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:55.460956  5621 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.020s	sys 0.023s
I20260709 13:02:55.409490  5785 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:55.440655  5647 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:55.479404  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 306786us
I20260709 13:02:55.428207  5461 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:55.492962  5775 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.002s	sys 0.012s
I20260709 13:02:55.483598  5636 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:02:55.493144  1681 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.011s	sys 0.010s
W20260709 13:02:55.557683  1681 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.012s	sys 0.011s
W20260709 13:02:55.521278  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.517401  5804 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:55.460927  5724 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:55.496364  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 490594us
W20260709 13:02:55.549279  5775 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.003s	sys 0.013s
I20260709 13:02:55.520856  5760 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:55.536419  5734 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:55.558948  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.413188  5731 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:55.518765  1566 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.010s	sys 0.050s
W20260709 13:02:55.493749  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 517264us
W20260709 13:02:55.559159  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 754203us
I20260709 13:02:55.565454  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:55.509582  5604 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:55.569665  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 355432us
I20260709 13:02:55.527506  5822 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:55.572678  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.580647  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.582455  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:02:55.575901  5636 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.588387  5602 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:55.592695  5507 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:55.610028  2302 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.010s	sys 0.018s
W20260709 13:02:55.610436  2302 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.010s	sys 0.018s
I20260709 13:02:55.593354  5721 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:55.597931  5828 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:55.614530  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 384555us
I20260709 13:02:55.615665  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:55.621622  5837 raft_consensus.cc:3055] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:55.614116  5507 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:55.622093  5837 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:55.622242  1567 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:55.623288  1568 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:55.623046  5828 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:55.628078  5507 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:55.623070  5837 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:55.630445  5837 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:55.634459  5785 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:55.639138  1682 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:02:55.645658  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 418441us
I20260709 13:02:55.651886  1466 catalog_manager.cc:5697] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 2 to 5, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193), VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194), VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195), VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 6 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:55.656494  5731 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:55.657099  5731 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:55.659235  5787 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:55.661226  5823 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.139s	user 0.000s	sys 0.003s
I20260709 13:02:55.612432  5736 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:55.639987  5469 consensus_meta.cc:287] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.003s
I20260709 13:02:55.598876  5636 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:55.611954  5797 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:55.615716  5602 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:55.647931  5623 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.723s	user 0.009s	sys 0.003s
I20260709 13:02:55.638373  5507 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:55.597023  5714 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:55.662336  5420 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:55.691387  5623 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:55.699102  5602 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:55.645889  5818 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:02:55.690958  5736 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:55.701359  5736 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:55.701558  5589 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:55.702368  5832 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:02:55.702456  1715 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:55.703591  5475 consensus_meta.cc:287] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.789s	user 0.003s	sys 0.009s
I20260709 13:02:55.704118  5475 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:55.703626  5834 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:02:55.707243  5415 consensus_meta.cc:287] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.787s	user 0.012s	sys 0.000s
I20260709 13:02:55.706852  5475 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:55.666077  5785 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:55.719214  1607 leader_election.cc:336] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.125s (SENT)
I20260709 13:02:55.747840  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:55.719626  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 154276us
I20260709 13:02:55.750779  2304 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:02:55.752100  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:55.752928  2297 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:55.718626  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 142408us
I20260709 13:02:55.754115  2269 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:55.755239  2269 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:55.758025  2299 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:55.763484  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:55.764312  2300 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:55.647891  5654 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:02:55.690574  5797 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:55.721805  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 43524.3us
I20260709 13:02:55.764564  2299 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:02:55.774806  2299 pending_rounds.cc:85] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 6
W20260709 13:02:55.725544  5755 consensus_meta.cc:287] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.755s	user 0.007s	sys 0.007s
W20260709 13:02:55.768993  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.765412  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:55.702595  5469 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:55.783174  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:55.783713  2299 pending_rounds.cc:107] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:02:55.784080  2299 raft_consensus.cc:2889] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:55.778714  2300 raft_consensus.cc:1217] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 6.7->[7.8-8.9]   Dedup: 7.8->[8.9-8.9]
I20260709 13:02:55.782809  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:55.784833  2300 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: [7.8-8.9]
I20260709 13:02:55.784005  2303 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
W20260709 13:02:55.777311  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.671270  5731 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:55.787639  1642 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 10: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:02:55.784519  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 256523us
I20260709 13:02:55.765743  5856 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:55.815174  1609 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:55.795459  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:55.794981  2305 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:02:55.816269  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:55.819774  2300 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 7: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:55.817638  1644 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:55.821440  2304 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:55.784637  5841 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:55.826429  1642 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:02:55.783239  5689 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:55.827265  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:55.828477  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 109298us
I20260709 13:02:55.760938  5785 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:55.785481  5461 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:55.830485  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:55.863466  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.864153  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:55.865180  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:55.874703  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.867580  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.702409  5736 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:55.870244  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.871485  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:55.880869  1607 leader_election.cc:336] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.830s (SENT)
I20260709 13:02:55.874620  1711 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:55.867040  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:55.875389  1710 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:55.878639  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:55.881217  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:55.872332  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:55.884289  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:55.879948  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 91355.7us
W20260709 13:02:55.885970  5243 consensus_meta.cc:287] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.944s	user 0.013s	sys 0.001s
I20260709 13:02:55.887470  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:55.888324  5863 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:02:55.816651  5469 raft_consensus.cc:697] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:55.891708  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.921469  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.786549  5797 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:55.927096  5755 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:55.928126  5797 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:55.936017  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:55.936996  1643 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:55.825352  5714 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:02:55.939126  5855 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:55.896176  1642 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:55.939847  5855 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:55.896631  1635 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:55.943897  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:55.945883  1635 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:02:55.897013  1636 raft_consensus.cc:2410] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:02:55.825855  5856 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.948112  5731 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:55.894903  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.949692  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.949941  1636 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:55.950434  1711 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:55.895773  1641 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:55.897190  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:55.904309  1637 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:02:55.951707  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.952137  1637 pending_rounds.cc:85] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 6
I20260709 13:02:55.952463  1637 pending_rounds.cc:107] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:02:55.952742  1637 raft_consensus.cc:2889] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:55.904765  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:55.954190  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 73393.2us
I20260709 13:02:55.826180  5734 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:55.918393  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:02:55.954851  1607 leader_election.cc:336] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.046s (SENT)
I20260709 13:02:55.955621  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:55.956514  5785 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:55.956809  1607 leader_election.cc:336] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.085s (SENT)
I20260709 13:02:55.957043  5623 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:55.918772  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:55.919623  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:55.957706  5731 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:55.958140  1717 raft_consensus.cc:2368] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11.
I20260709 13:02:55.958535  5731 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:55.957285  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:55.922423  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.957741  1710 raft_consensus.cc:2376] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9. Re-sending same reply.
I20260709 13:02:55.959900  1712 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:55.960559  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.960902  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.961658  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.962322  1712 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:55.830794  5841 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.926036  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:55.963151  5841 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:55.948313  5856 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:55.893393  1713 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:55.967506  5636 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:55.830343  5859 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:55.892925  5844 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.138s
W20260709 13:02:55.968778  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.969570  1607 leader_election.cc:336] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.117s (SENT)
I20260709 13:02:55.970045  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:55.970775  5739 consensus_meta.cc:287] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.115s	user 0.012s	sys 0.001s
I20260709 13:02:55.971251  5739 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:55.971336  1607 leader_election.cc:336] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.144s (SENT)
I20260709 13:02:55.835109  5762 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:55.972919  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2598) took 1174 ms. Trace:
I20260709 13:02:55.973894  5762 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:55.977089  5469 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:55.887516  5760 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:55.980055  1636 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:02:55.981655  5760 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:55.822715  5507 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:02:55.982963  1710 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:02:55.890348  5724 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:55.985997  5724 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:55.981699  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.990067  2305 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:55.973240  1567 rpcz_store.cc:276] 0709 13:02:54.803073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.623010 (+819937us) service_pool.cc:224] Handling call
0709 13:02:55.623334 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.623525 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:55.623838 (+   313us) log.cc:844] Serialized 14 byte log entry
0709 13:02:55.972599 (+348761us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:55.972650 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:55.972692 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.972753 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.972867 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3524,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":57,"thread_start_us":348423,"threads_started":1,"wal-append.queue_time_us":168228}
I20260709 13:02:55.959041  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:55.980114  5828 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:55.996088  1639 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:55.979125  5797 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:56.002336  1716 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:02:55.654389  1466 catalog_manager.cc:5697] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 3 to 6, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 6 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:56.000785  2297 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:56.001322  1642 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:56.002342  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2217) took 1206 ms. Trace:
W20260709 13:02:55.997732  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.005198  1565 rpcz_store.cc:276] 0709 13:02:54.795912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.379789 (+583877us) service_pool.cc:224] Handling call
0709 13:02:55.393955 (+ 14166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.466057 (+ 72102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.466197 (+   140us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:55.466240 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:55.466268 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.934693 (+468425us) spinlock_profiling.cc:243] Waited 69 ms on lock 0x7b5400217354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:55.934768 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.002264 (+ 67496us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158809472}
I20260709 13:02:56.015331  5869 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.017020  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:56.018190  5461 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:02:56.018414  2297 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:02:56.018774  5856 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.019992  2268 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:56.020387  5292 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:56.021034  5461 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:56.021554  5461 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:56.022033  5461 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:56.025527  1466 catalog_manager.cc:5697] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 5 to 9, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 9 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:02:55.942724  5855 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:56.026716  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:56.027833  1718 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
W20260709 13:02:56.029958  1607 leader_election.cc:336] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.520s (SENT)
I20260709 13:02:56.030787  1643 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:55.924751  5837 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:55.970814  5721 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:56.036335  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:56.038759  2268 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:55.873895  5689 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.006842  5859 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:56.040695  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:56.040382  1466 catalog_manager.cc:5697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:56.037349  5595 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:56.043771  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 225452us
I20260709 13:02:56.045854  1609 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:56.016019  5869 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.047035  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:55.926031  5853 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.114s
I20260709 13:02:55.962723  5822 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.048553  2268 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:56.046517  1466 catalog_manager.cc:5697] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 1 to 7, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 7 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:56.056365  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:56.057677  2299 raft_consensus.cc:2410] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:02:56.058290  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:56.057433  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:56.061908  1466 catalog_manager.cc:5697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 2 to 7, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 7 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:02:56.063815  2268 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:56.065472  1718 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:56.068039  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:56.068701  2299 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:56.044996  5855 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:56.059217  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:56.070998  2300 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:02:56.071923  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:56.072532  2300 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:02:56.074503  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:56.075127  2305 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:02:56.047312  5860 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:56.060119  5868 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:56.077837  5595 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:56.078613  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2085) took 1307 ms. Trace:
I20260709 13:02:56.096309  1715 rpcz_store.cc:276] 0709 13:02:54.771704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.775289 (+  3585us) service_pool.cc:224] Handling call
0709 13:02:54.793196 (+ 17907us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.703256 (+910060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.078403 (+375147us) spinlock_profiling.cc:243] Waited 909 ms on lock 0x7b5400268ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:56.078559 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2090853248}
I20260709 13:02:56.079326  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:56.079864  5867 raft_consensus.cc:2764] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:02:56.080494  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:56.098371  1718 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:02:56.098789  1718 pending_rounds.cc:85] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 8
I20260709 13:02:56.099027  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:56.081664  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:56.082933  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:56.099745  1466 catalog_manager.cc:5697] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 3 to 4, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:56.100734  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:56.101706  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:56.099045  1718 pending_rounds.cc:107] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:02:56.112103  5689 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:56.113134  5611 consensus_meta.cc:287] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
I20260709 13:02:56.074115  5855 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:56.115629  5855 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:56.119809  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:56.130076  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:56.130909  1638 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:56.099872  1717 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:56.160441  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:56.092887  2301 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:56.083657  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:56.162415  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:56.163473  1710 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:56.058017  5731 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:56.040124  5837 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.103808  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:56.110569  1718 raft_consensus.cc:2889] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:56.164397  1712 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:02:56.104497  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:56.165323  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:56.105321  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:56.105841  5604 consensus_meta.cc:287] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.520s	user 0.008s	sys 0.000s
I20260709 13:02:56.092207  1609 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:56.170468  1637 raft_consensus.cc:2410] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:02:56.169246  5731 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.190433  5869 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:56.190799  5629 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:56.190481  5731 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:56.191322  5822 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:56.192062  5837 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:56.191350  5629 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:56.194789  5507 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:56.193533  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:56.113601  5611 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.197000  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:56.197710  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:56.199016  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:56.203568  1466 catalog_manager.cc:5697] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:56.212003  1638 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:56.215371  1466 catalog_manager.cc:5697] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 4 to 7, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:56.217257  1718 raft_consensus.cc:3055] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:56.098592  5860 raft_consensus.cc:697] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:56.217630  2299 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:02:56.110549  5841 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:56.228009  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 108322us
I20260709 13:02:56.228825  2296 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:56.232049  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 139954us
I20260709 13:02:56.322283  1609 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:56.323499  1609 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:56.240108  5797 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:56.205848  5604 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.278491  5507 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:56.294679  1709 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:56.296818  5611 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:56.304921  1718 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:56.167352  1682 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:56.281518  1466 catalog_manager.cc:5697] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 3 to 6, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:56.329793  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:56.358459  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 191206us
I20260709 13:02:56.377102  1718 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:56.388652  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:56.327733  5797 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.329461  5859 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:56.329361  5875 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:56.403527  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:56.404458  1718 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:56.376231  5724 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:56.377977  5589 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:56.314225  5461 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.376015  5732 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:56.340291  5731 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.329528  5629 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.379662  5636 consensus_queue.cc:1059] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:02:56.376518  5507 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.411303  1466 catalog_manager.cc:5697] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:56.441633  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:56.443375  1712 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
W20260709 13:02:56.410202  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 346452us
I20260709 13:02:56.431546  4544 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:02:56.432956  5859 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.442548  5875 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.464083  4544 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:56.416404  5797 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:56.469410  2298 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:56.470132  1466 catalog_manager.cc:5697] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:56.503896  1718 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:56.444571  5438 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:56.450099  5855 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.513988  2303 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:56.517040  5855 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:56.472337  5859 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:56.473064  5589 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:56.549544  1569 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:56.549551  1466 catalog_manager.cc:5697] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:02:56.475600  5875 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:56.567741  5886 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:56.447036  5860 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:56.570022  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.595184  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.597054  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:56.598749  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:56.571982  5595 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:02:56.575527  5878 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:02:56.576391  5885 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.458652  5732 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.589742  1466 catalog_manager.cc:5697] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:56.538847  5654 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
W20260709 13:02:56.633306  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.633999  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:56.596608  5724 raft_consensus.cc:3055] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:56.639564  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:02:56.641104  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.601366  5884 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:56.647565  1718 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:56.640393  2305 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:56.661085  5595 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.003s
W20260709 13:02:56.686910  5879 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 13:02:56.692289  5629 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:56.633718  5885 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:56.708951  5507 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:56.709112  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:56.711597  5732 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:56.713210  5731 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:56.640692  5724 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:56.716885  1466 catalog_manager.cc:5697] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 3 to 6, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:56.756207  1709 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
W20260709 13:02:56.757773  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 821) took 2107 ms (client timeout 2150 ms). Trace:
I20260709 13:02:56.771560  1466 catalog_manager.cc:5697] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 3 to 6, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 6 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:02:56.977705  1709 rpcz_store.cc:269] 0709 13:02:54.650260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.653069 (+  2809us) service_pool.cc:224] Handling call
0709 13:02:55.148090 (+495021us) spinlock_profiling.cc:243] Waited 58.7 ms on lock 0x7b1800059d60. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1cdc646 00007fd7a1c1658e 00007fd7a1c97f9e 00007fd7a1c997de 00007fd7a1ca527c 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c
0709 13:02:56.294232 (+1146142us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b580057bcd0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1bfd0aa 00007fd7a1c7f447 00007fd7a1c97fe7 00007fd7a1c997de 00007fd7a1ca527c 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c
0709 13:02:56.757686 (+463454us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1063062656}
I20260709 13:02:56.980337  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2082) took 2209 ms. Trace:
I20260709 13:02:56.980731  1714 rpcz_store.cc:276] 0709 13:02:54.770632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.772357 (+  1725us) service_pool.cc:224] Handling call
0709 13:02:54.775532 (+  3175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.797769 (+ 22237us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:54.798082 (+   313us) log.cc:844] Serialized 14 byte log entry
0709 13:02:55.246360 (+448278us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:55.246429 (+    69us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:55.246478 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.986765 (+740287us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b18003aa320. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb822a6 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419
0709 13:02:56.980133 (+993368us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b54003377d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:56.980171 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.980288 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2794,"prepare.queue_time_us":987,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":95,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":9905,"spinlock_wait_cycles":62286464,"thread_start_us":447869,"threads_started":1,"wal-append.queue_time_us":752395,"wal-append.run_cpu_time_us":4986,"wal-append.run_wall_time_us":1071041}
I20260709 13:02:56.782629  2297 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:02:56.702760  5885 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:56.662338  5884 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:56.854856  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 258920us
I20260709 13:02:56.985086  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:56.985822  5749 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.986658  5884 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:56.986466  5749 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:56.869212  1535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.599s	user 0.013s	sys 0.024s
I20260709 13:02:56.886284  5875 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:56.988035  1535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.718s	user 0.014s	sys 0.024s
I20260709 13:02:56.989113  5749 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:56.989547  5875 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:56.867379  5859 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.990371  5749 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:56.992033  5859 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:56.992815  5749 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:56.994531  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:56.996527  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.997812  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:56.998731  1565 raft_consensus.cc:2393] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 9: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:56.997913  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.000073  1567 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:02:57.000442  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.002035  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.003551  5886 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.004345  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.004139  5886 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.005532  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.007050  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:57.008436  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.009795  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:57.010313  1565 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:57.011569  5853 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:57.012800  5886 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.013618  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:57.014333  1564 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:57.019744  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:57.021883  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.022091  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.022852  1565 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:02:57.021914  1564 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:02:57.025044  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:57.027537  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.028483  1563 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:02:57.028661  5912 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:57.029152  5912 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.029543  5912 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:57.033560  4544 raft_consensus.cc:3055] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:57.034157  4544 raft_consensus.cc:740] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:57.043025  5912 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.046088  5912 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.047044  5912 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:57.050179  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:57.061470  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:57.057252  2305 raft_consensus.cc:2410] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:02:57.051153  4544 consensus_queue.cc:260] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:57.067159  4544 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:57.070312  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.072537  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.080621  1567 raft_consensus.cc:2393] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:02:57.081962  2302 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.004s	sys 0.003s
W20260709 13:02:57.082438  2302 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.006s	sys 0.004s
I20260709 13:02:57.079188  5912 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.090607  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.094775  5916 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:57.095134  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1119) took 1310 ms. Trace:
I20260709 13:02:57.093641  5912 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.104763  5912 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:02:57.103420  2302 rpcz_store.cc:276] 0709 13:02:55.784743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.784937 (+   194us) service_pool.cc:224] Handling call
0709 13:02:55.785213 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.785404 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:56.296779 (+511375us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b18000d5760. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb1455 00007fd7a56993c7 00007fd7a1cab9a7 00007fd7a1cbb0bf
0709 13:02:56.296919 (+   140us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.094325 (+797406us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.094743 (+   418us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:57.094802 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:57.094853 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.094951 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.095080 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":105307,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":7427,"raft.queue_time_us":2306,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":200,"spinlock_wait_cycles":3345664,"thread_start_us":989158,"threads_started":3,"wal-append.queue_time_us":561334}
I20260709 13:02:57.096592  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.106395  1567 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:02:57.109334  1567 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 13:02:57.112589  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 87640.3us
I20260709 13:02:57.114742  5912 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:57.115466  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:57.117887  5912 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:57.121378  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:57.122659  1567 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:57.123769  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.126266  5826 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 1.558s	user 0.009s	sys 0.000s
I20260709 13:02:57.127560  1534 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:57.129117  2302 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:02:57.129292  5915 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:57.129918  5915 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:57.129145  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:57.131047  2305 raft_consensus.cc:2376] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9. Re-sending same reply.
I20260709 13:02:57.131258  5915 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.133155  2302 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:02:57.135375  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:57.135562  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:57.136309  2305 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:57.136791  2302 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:57.138370  1567 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:57.144505  2302 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:02:57.146070  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2215) took 2350 ms. Trace:
I20260709 13:02:57.144596  5623 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:57.147977  5856 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:02:57.150152  1532 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.807s	user 0.002s	sys 0.026s
I20260709 13:02:57.149528  1569 rpcz_store.cc:276] 0709 13:02:54.795390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.353319 (+557929us) service_pool.cc:224] Handling call
0709 13:02:55.353760 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.550048 (+1196288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.145809 (+595761us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54001ce654. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:57.146004 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2749865088}
W20260709 13:02:57.151222  1532 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.808s	user 0.002s	sys 0.026s
I20260709 13:02:57.150060  5623 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:57.152401  5623 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:57.153288  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:57.154330  5856 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:57.154811  5856 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.155287  5856 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:57.160583  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.163861  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 423413us
I20260709 13:02:57.165901  1681 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:57.166715  5905 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:57.170207  5907 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.170725  5907 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.172931  5907 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.174628  1681 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:57.176308  5856 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.193004  5856 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:57.178210  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:57.187314  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.194528  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:57.195266  5856 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:57.196208  5856 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:57.196926  2268 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:57.199313  5736 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:57.199816  5736 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:57.200721  5736 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:56.769804  5292 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.202104  5292 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.202834  5915 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:02:57.203624  1462 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45142 (request call id 349) took 34003 ms (client timeout 15000 ms). Trace:
I20260709 13:02:57.204003  5915 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:57.203830  2268 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:57.198426  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.000s	sys 0.019s
W20260709 13:02:57.205500  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.000s	sys 0.020s
I20260709 13:02:57.197842  5469 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:57.208938  5868 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:57.209515  5868 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:02:57.204067  1462 rpcz_store.cc:269] 0709 13:02:23.200033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.200369 (+   336us) service_pool.cc:224] Handling call
0709 13:02:57.203393 (+34003024us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:47.953476 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:47.954042 (+   566us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:47.954097 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:02:47.954122 (+    25us) tablet.cc:662] Decoding operations
0709 13:02:47.955304 (+  1182us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:47.955383 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:47.955407 (+    24us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:02:47.956015 (+   608us) tablet.cc:701] Row locks acquired
0709 13:02:47.956054 (+    39us) write_op.cc:260] PREPARE: finished
0709 13:02:47.956189 (+   135us) write_op.cc:270] Start()
0709 13:02:47.956303 (+   114us) write_op.cc:276] Timestamp: P: 1783602167956134 usec, L: 0
0709 13:02:47.956316 (+    13us) op_driver.cc:348] REPLICATION: starting
0709 13:02:47.956871 (+   555us) log.cc:844] Serialized 3940 byte log entry
0709 13:02:48.108317 (+151446us) op_driver.cc:464] REPLICATION: finished
0709 13:02:48.497103 (+388786us) write_op.cc:301] APPLY: starting
0709 13:02:48.497198 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:48.497984 (+   786us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:48.498118 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:48.500740 (+  2622us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:48.501119 (+   379us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 13:02:48.501197 (+    78us) write_op.cc:312] APPLY: finished
0709 13:02:48.501846 (+   649us) log.cc:844] Serialized 71 byte log entry
0709 13:02:48.502978 (+  1132us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:48.503125 (+   147us) write_op.cc:454] Released schema lock
0709 13:02:48.503208 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:48.503759 (+   551us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:48.503787 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2770048,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":388466,"apply.run_cpu_time_us":4234,"apply.run_wall_time_us":6645,"num_ops":9,"prepare.queue_time_us":3374,"prepare.run_cpu_time_us":6712,"prepare.run_wall_time_us":40785,"raft.queue_time_us":77607,"raft.run_cpu_time_us":32894,"raft.run_wall_time_us":395384,"replication_time_us":151844,"thread_start_us":504609,"threads_started":4,"wal-append.queue_time_us":66393,"wal-append.run_cpu_time_us":22905,"wal-append.run_wall_time_us":2137666}]]}
W20260709 13:02:57.205921  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 745564us
I20260709 13:02:57.211416  1534 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:57.212937  5438 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:57.214895  1562 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:57.214645  5438 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:57.215930  5469 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:02:57.216387  5469 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:02:57.217717  1562 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2602) took 2380 ms (client timeout 2602 ms). Trace:
I20260709 13:02:57.217522  5469 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.204623  5915 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:57.219260  1563 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:57.219292  1682 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:57.221573  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:57.222360  1567 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:02:57.218274  1562 rpcz_store.cc:269] 0709 13:02:54.837734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.022591 (+184857us) service_pool.cc:224] Handling call
0709 13:02:57.217461 (+2194870us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b54002c5e54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:57.217610 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4851079040}
I20260709 13:02:57.228039  1682 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:57.233242  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:57.233881  1567 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:02:57.235689  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:57.206081  5922 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:02:57.236070  5898 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:57.235013  5905 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:02:57.205675  5292 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.210418  5868 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.239686  5732 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:57.240512  5689 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:57.270179  5933 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:02:57.268891  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 56259.4us
I20260709 13:02:57.255347  5915 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:57.275053  5932 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:57.275816  5915 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:57.271816  1534 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:57.273159  5689 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:57.272912  5732 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:57.274351  5898 raft_consensus.cc:697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:57.307667  5898 consensus_queue.cc:237] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:57.313817  5689 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:57.306408  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 69085.8us
W20260709 13:02:57.322499  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.302305  5933 raft_consensus.cc:697] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:57.284725  5912 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:02:57.326224  5905 raft_consensus.cc:697] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:57.306836  5292 raft_consensus.cc:993] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:02:57.333086  5933 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.335029  5732 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:57.335712  5905 consensus_queue.cc:237] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:57.324396  5937 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:57.350764  5292 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:57.351357  5934 raft_consensus.cc:1081] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:02:57.352197  5292 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.347470  5937 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:02:57.455284  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 171901us
W20260709 13:02:57.455518  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 183828us
I20260709 13:02:57.457743  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:57.460840  1714 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:02:57.462387  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:57.463073  1713 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:57.463212  1715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:02:57.463907  1715 raft_consensus.cc:606] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:57.457855  5939 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:57.456585  5937 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:57.457316  5898 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:57.474615  1713 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:57.460484  5905 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:57.477135  1564 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:02:57.488559  5292 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.490592  5868 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:57.491276  5938 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:57.479188  1637 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60390
I20260709 13:02:57.492089  5938 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:57.492950  5828 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.004s	sys 0.016s
I20260709 13:02:57.493050  1637 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:02:57.493526  5828 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.004s	sys 0.016s
I20260709 13:02:57.493640  1637 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:57.487211  5898 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.497951  5939 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.501102  5939 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.462762  5932 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:57.506242  5898 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.506557  5932 raft_consensus.cc:697] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:57.509662  5932 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:57.511830  1609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.083s	user 0.008s	sys 0.002s
W20260709 13:02:57.512351  1609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.084s	user 0.008s	sys 0.002s
I20260709 13:02:57.517800  2302 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:57.530411  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:57.531687  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:57.505187  5938 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.535624  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:57.536427  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:57.537647  1635 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:57.538558  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:57.539208  2302 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:57.521929  5905 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.543174  5623 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:57.549391  1637 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.556690  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:57.557685  1635 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:57.542755  5905 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.559635  1635 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: []
W20260709 13:02:57.562184  1533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.295s	user 0.017s	sys 0.028s
W20260709 13:02:57.562704  1533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.296s	user 0.017s	sys 0.028s
W20260709 13:02:57.563400  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 2.30763e+06us
I20260709 13:02:57.564131  1533 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:57.564401  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:57.564585  5937 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.567207  5917 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:02:57.565582  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:56.770491  5724 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.570272  5724 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:57.571170  5905 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:57.571632  5905 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:57.572044  5905 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:57.574771  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:57.574785  5623 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:57.580845  1637 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.584295  5937 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.583608  5942 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:02:57.586427  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.586225  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:57.587495  5942 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:57.580276  5933 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:57.593045  2302 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:02:57.596279  5944 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:57.598817  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1114) took 1836 ms. Trace:
I20260709 13:02:57.599314  2304 rpcz_store.cc:276] 0709 13:02:55.762762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.762945 (+   183us) service_pool.cc:224] Handling call
0709 13:02:55.763273 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.822035 (+ 58762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.598568 (+1776533us) spinlock_profiling.cc:243] Waited 53.7 ms on lock 0x7b54001aa754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:57.598755 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123420416}
I20260709 13:02:57.603169  2304 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:02:57.610561  5905 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.611114  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 972) took 1556 ms. Trace:
I20260709 13:02:57.611532  2298 rpcz_store.cc:276] 0709 13:02:56.055141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.072937 (+ 17796us) service_pool.cc:224] Handling call
0709 13:02:56.073218 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.532736 (+459518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.610845 (+1078109us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b5400583ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:57.611033 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":909452544}
I20260709 13:02:57.613572  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:57.614166  1637 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:57.617990  2305 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
W20260709 13:02:57.618907  1566 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.041s	user 0.003s	sys 0.008s
W20260709 13:02:57.619371  1566 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.041s	user 0.003s	sys 0.008s
I20260709 13:02:57.614630  5917 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:57.620981  1534 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:57.621192  5940 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:57.621393  5617 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:57.620949  5937 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:57.621492  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2213) took 2835 ms. Trace:
I20260709 13:02:57.622244  5912 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:57.622633  5940 raft_consensus.cc:3055] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:57.623224  5940 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:57.622798  5912 raft_consensus.cc:697] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:57.622560  1566 rpcz_store.cc:276] 0709 13:02:54.786100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.791384 (+  5284us) service_pool.cc:224] Handling call
0709 13:02:54.791665 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:54.791843 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:55.575223 (+783380us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.621137 (+2045914us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:57.621187 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:57.621230 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.621310 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.621422 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1366,"prepare.queue_time_us":583746,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":2294,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"thread_start_us":2824694,"threads_started":2,"wal-append.queue_time_us":166967,"wal-append.run_cpu_time_us":1396,"wal-append.run_wall_time_us":1024860}
I20260709 13:02:57.624496  5912 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.620867  5917 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:57.624267  5940 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.627331  5944 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:57.620688  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:57.632406  5932 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:57.615984  5905 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.633607  5940 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:57.663249  1637 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:57.665028  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:57.666106  5905 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:57.666729  5905 raft_consensus.cc:697] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:57.667784  5905 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.683296  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 977) took 1357 ms. Trace:
I20260709 13:02:57.683745  2303 rpcz_store.cc:276] 0709 13:02:56.325692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.378405 (+ 52713us) service_pool.cc:224] Handling call
0709 13:02:56.401191 (+ 22786us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.514588 (+113397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.683008 (+1168420us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54006414d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:57.683228 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258785152}
W20260709 13:02:57.690905  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.698345  5841 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:57.707372  5871 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 1.609s	user 0.006s	sys 0.007s
W20260709 13:02:57.716542  2299 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.271s	user 0.008s	sys 0.004s
W20260709 13:02:57.717015  2299 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.272s	user 0.009s	sys 0.005s
I20260709 13:02:57.721527  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:57.722309  1641 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 6.
I20260709 13:02:57.726361  1682 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:57.727451  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 975) took 1402 ms. Trace:
I20260709 13:02:57.727865  2299 rpcz_store.cc:276] 0709 13:02:56.325038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.325848 (+   810us) service_pool.cc:224] Handling call
0709 13:02:56.326310 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:56.335482 (+  9172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.335531 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:56.390331 (+ 54800us) log.cc:844] Serialized 26 byte log entry
0709 13:02:57.727167 (+1336836us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:57.727214 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.727267 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:57.727298 (+    31us) raft_consensus.cc:1685] finished
0709 13:02:57.727304 (+     6us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.727409 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8710,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":22747,"thread_start_us":1271973,"threads_started":1,"wal-append.queue_time_us":446990}
I20260709 13:02:57.730361  5838 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:57.732131  5838 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.735016  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:57.735337  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.736063  1566 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:02:57.737500  2268 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:57.738389  5869 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:57.738824  5869 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:57.739283  5869 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:57.743402  5844 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.744081  5844 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:57.747560  5461 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:57.748787  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:57.749477  1641 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:57.750623  1682 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:57.751497  5438 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:57.751910  5438 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.752291  5438 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:57.753286  5838 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:57.746691  5844 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:57.754654  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:57.754803  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:57.755347  1641 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:57.755988  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:57.756875  5892 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.757218  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:57.757400  5892 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.758034  1640 raft_consensus.cc:2393] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 11: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:02:57.758935  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.759133  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:57.760730  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.761375  1609 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:57.762928  5438 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:57.763406  5724 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:57.763969  5724 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.765493  5438 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:57.766239  5724 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.767778  5859 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:57.768299  5859 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:57.759692  5892 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.769157  5859 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.769776  5892 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:57.771965  1641 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:02:57.731083  5905 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:02:57.774744  5905 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:57.775177  5905 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:02:57.750765  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.730403  5841 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:57.801530  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:57.802198  1635 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:57.770766  5892 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:57.804108  1609 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:57.766811  5724 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.801106  5838 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:57.810626  1641 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:02:57.811262  5755 raft_consensus.cc:1064] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
W20260709 13:02:57.811771  5755 raft_consensus.cc:1068] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Unable to promote non-voter 6208c2fa49f643b5a50ab8f6d292fab2: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:02:57.811367  5636 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:57.813242  5897 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:02:57.814422  5897 raft_consensus.cc:3055] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:57.814963  5724 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.811290  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: true dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:57.805656  5892 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:57.817198  5724 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.817873  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:57.817111  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:57.819617  2298 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:57.820168  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:02:57.818945  1607 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.231s (SENT) [suppressed 22 similar messages]
I20260709 13:02:57.820868  1641 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:57.820979  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:57.807760  5841 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.823158  2299 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:57.823238  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:57.824052  1640 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:02:57.826773  1635 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:57.808320  5839 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:57.820595  5838 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.830061  5838 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:57.830320  2299 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
W20260709 13:02:57.821142  1607 connection.cc:256] RPC call timeout handler was delayed by -1.0342538730000115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2008s for a response
W20260709 13:02:57.831058  1607 connection.cc:256] RPC call timeout handler was delayed by -1.0329335140000353s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2618s for a response
I20260709 13:02:57.831243  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.831993  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2209) took 3073 ms. Trace:
I20260709 13:02:57.832813  5841 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:57.833315  1682 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:57.834103  5438 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:57.834609  5438 raft_consensus.cc:697] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:57.835374  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.835503  5438 consensus_queue.cc:237] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:57.836212  1568 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:02:57.838690  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.842576  1590 debug-util.cc:398] Leaking SignalData structure 0x7b08001cdee0 after lost signal to thread 434
I20260709 13:02:57.814826  5897 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:57.844254  5897 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.845490  5897 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:57.832418  1560 rpcz_store.cc:276] 0709 13:02:54.758243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.779054 (+ 20811us) service_pool.cc:224] Handling call
0709 13:02:54.876437 (+ 97383us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:54.905188 (+ 28751us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.905220 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:54.906118 (+   898us) log.cc:844] Serialized 52 byte log entry
0709 13:02:55.439140 (+533022us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:55.439218 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:55.439339 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.970062 (+1530723us) raft_consensus.cc:1685] finished
0709 13:02:56.970150 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.831730 (+861580us) spinlock_profiling.cc:243] Waited 83.5 ms on lock 0x7b54001ff250. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:57.831920 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25681,"raft.queue_time_us":30047,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":122,"spinlock_wait_cycles":192105728,"thread_start_us":532401,"threads_started":1,"wal-append.queue_time_us":688521,"wal-append.run_cpu_time_us":2574,"wal-append.run_wall_time_us":1111347}
I20260709 13:02:57.817983  5892 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.851009  5724 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.851285  5892 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:57.851768  5724 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.854365  5724 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:57.854462  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:57.855626  1640 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:57.857547  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:57.857862  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:57.858192  1635 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:02:57.765331  5869 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.858158  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:57.860562  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:57.862804  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:57.863853  5839 raft_consensus.cc:697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:57.864743  4544 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.864850  5839 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.865365  4544 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:57.867193  4544 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:57.868378  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:57.869537  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:57.870172  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.870268  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:57.870628  1567 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:02:57.871649  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:57.872180  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:57.871757  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:57.872407  1714 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:57.877137  1641 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:57.881291  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:57.882004  1715 raft_consensus.cc:3055] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:57.882757  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:57.882505  1715 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:57.888187  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:57.889245  1637 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:57.883669  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.883827  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:57.884387  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:57.890944  1562 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:57.885962  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:57.891053  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:57.891469  1564 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:57.893668  1566 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:57.893791  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:57.894418  2298 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:02:57.896015  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.897539  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:57.898437  1709 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:02:57.898993  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:57.899783  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:57.900656  1643 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:02:57.882042  5916 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:57.901698  5869 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:57.889026  1715 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:57.902807  1715 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:57.906972  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:57.906679  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:57.907902  1568 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:57.909534  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:57.909917  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:57.911886  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:57.920194  1564 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:02:57.910185  1641 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
W20260709 13:02:57.902834  5797 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.365s	user 0.013s	sys 0.002s
I20260709 13:02:57.933327  1533 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:57.934466  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:57.929780  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:57.939777  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:57.955395  2298 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:02:57.956892  1533 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:57.908290  5870 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:58.015271  1609 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:02:58.017515  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 60745.8us
I20260709 13:02:58.021342  1533 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:02:58.021589  4544 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:02:58.022538  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 65373.8us
I20260709 13:02:58.022709  1533 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:57.944924  5903 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:57.956285  5940 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:02:57.953029  5886 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:57.926752  5636 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:57.922060  5755 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:57.956141  5912 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:58.026787  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:58.028044  1636 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:58.025460  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:58.026211  5714 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:57.953733  5647 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:02:58.021086  5932 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:57.901741  5916 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.053165  1712 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:58.053483  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:58.054229  1714 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:02:57.921912  5724 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 13:02:58.025990  5917 raft_consensus.cc:2764] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:02:57.905156  5905 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.037344  1564 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:02:58.066442  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:58.067938  1711 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 8: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:58.024710  5903 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:58.026087  5915 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:57.970417  5797 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.024925  5942 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:58.049371  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2225) took 1020 ms. Trace:
I20260709 13:02:58.029119  5755 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:58.057242  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:58.080101  1563 rpcz_store.cc:276] 0709 13:02:57.029373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:57.029480 (+   107us) service_pool.cc:224] Handling call
0709 13:02:57.219015 (+189535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.219711 (+   696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.049178 (+829467us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54002c5e50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:58.049309 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435201920}
I20260709 13:02:58.084806  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:58.085489  1717 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.064936  1568 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:58.066442  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:58.081811  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:58.069639  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:58.090466  2268 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:58.027443  5884 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:02:58.040050  5714 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.097345  5714 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:58.077461  5838 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:02:58.088897  5755 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:58.039867  5875 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:58.108510  5875 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:58.109393  5875 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:58.112331  5869 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:58.113076  5869 raft_consensus.cc:697] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:58.115131  5841 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:58.115813  5841 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.118527  5841 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:58.066833  1715 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:02:58.070098  5647 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:58.068943  5912 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:02:58.142333  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:58.095050  5903 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.053169  5916 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:58.092228  5858 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:58.123255  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:58.125361  5905 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:58.126643  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2579) took 1968 ms. Trace:
I20260709 13:02:58.127180  5797 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:58.114118  2296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1122) took 2299 ms (client timeout 2246 ms). Trace:
I20260709 13:02:58.128844  1635 raft_consensus.cc:2393] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 12: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:58.135743  1563 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
W20260709 13:02:58.136793  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 100785us
I20260709 13:02:58.155632  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:58.156859  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:58.141607  5647 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:58.140347  1717 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:02:58.163121  1712 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:02:58.147702  2298 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:02:58.148815  1642 rpcz_store.cc:276] 0709 13:02:56.158343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.162052 (+  3709us) service_pool.cc:224] Handling call
0709 13:02:56.162424 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:56.228457 (+ 66033us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.228487 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:56.234213 (+  5726us) log.cc:844] Serialized 26 byte log entry
0709 13:02:56.361014 (+126801us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:56.405467 (+ 44453us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.641709 (+236242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.641788 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:58.124216 (+1482428us) spinlock_profiling.cc:243] Waited 6.34 ms on lock 0x7b18004d7340. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:02:58.126304 (+  2088us) raft_consensus.cc:1685] finished
0709 13:02:58.126448 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.126576 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65602,"prepare.queue_time_us":30941,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":211896,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":64547,"spinlock_wait_cycles":14570880,"thread_start_us":351662,"threads_started":2,"wal-append.queue_time_us":216251}
I20260709 13:02:58.093693  1714 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 12: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:02:58.156652  2296 rpcz_store.cc:269] 0709 13:02:55.814209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.814345 (+   136us) service_pool.cc:224] Handling call
0709 13:02:58.113774 (+2299429us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b54003a2d54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:58.113982 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":816848256}
I20260709 13:02:58.159313  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:58.166268  2305 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:02:58.168874  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:58.170614  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:58.171844  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:58.176440  1716 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:02:58.179971  1643 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:58.168030  5647 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:58.181736  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 155027us
I20260709 13:02:58.114193  5869 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.170708  5903 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:58.180907  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2582) took 2019 ms. Trace:
I20260709 13:02:58.204144  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:58.175822  5885 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:58.157924  5858 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.179406  5797 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:58.181021  5894 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:58.216434  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:58.180914  5617 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:02:58.224885  1714 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:58.207345  5859 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:58.178596  5839 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:58.206995  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1127) took 1616 ms. Trace:
I20260709 13:02:58.179757  5940 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:58.218150  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:58.174988  5838 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:02:58.205870  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:58.226217  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:58.265280  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:02:58.258136  5647 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:58.212447  5970 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:58.229773  5859 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.217089  5858 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:58.207391  1644 rpcz_store.cc:276] 0709 13:02:56.161891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.162817 (+   926us) service_pool.cc:224] Handling call
0709 13:02:56.192232 (+ 29415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:56.231393 (+ 39161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.231435 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:56.238589 (+  7154us) log.cc:844] Serialized 26 byte log entry
0709 13:02:56.382140 (+143551us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:56.412860 (+ 30720us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.138060 (+725200us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b480015da58. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a6bcaaa1 00007fd7a6bc1b37 00007fd7a6bc19ca 00007fd7a6cb926f 00007fd7a6cb8f37 00007fd7a6ccd55c 00007fd7a6ccd508 00007fd7a6ccd488 00007fd7a6ccd443 00007fd7a6ccc6ec 00007fd7a5bf6a96
0709 13:02:58.082991 (+944931us) spinlock_profiling.cc:243] Waited 3.46 ms on lock 0x7b18004d7280. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb1455 00007fd7a1c99060 00007fd7a1cab716 00007fd7a1cbc287 00007fd7a1cbc208
0709 13:02:58.180028 (+ 97037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.180157 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:58.180204 (+    47us) raft_consensus.cc:1685] finished
0709 13:02:58.180216 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.180841 (+   625us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":666266,"prepare.queue_time_us":30681,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":2896,"raft.queue_time_us":176447,"raft.run_cpu_time_us":8771,"raft.run_wall_time_us":900360,"spinlock_wait_cycles":11864576,"thread_start_us":221037,"threads_started":2,"wal-append.queue_time_us":282997,"wal-append.run_cpu_time_us":2823,"wal-append.run_wall_time_us":1134115}
I20260709 13:02:58.230499  2297 rpcz_store.cc:276] 0709 13:02:56.590258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.987189 (+396931us) service_pool.cc:224] Handling call
0709 13:02:56.987463 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:57.100331 (+112868us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:57.100379 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:57.100441 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:57.100490 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.206742 (+1106252us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b540068d3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:58.206791 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.206940 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258903296}
I20260709 13:02:58.260565  1563 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
W20260709 13:02:58.285537  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 79823.3us
I20260709 13:02:58.267751  5897 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:58.321759  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 53488.6us
I20260709 13:02:58.212951  5917 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:02:58.323212  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:58.324846  2298 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:58.260083  5940 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:58.331914  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.332795  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:58.343778  1714 raft_consensus.cc:2376] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7. Re-sending same reply.
I20260709 13:02:58.323711  5839 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:58.323390  5937 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:58.340474  5859 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:58.359110  1716 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:02:58.360430  1714 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:02:58.341724  5969 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:58.351164  5940 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:58.375515  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:58.379264  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:58.380044  1714 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:58.342866  5870 consensus_queue.cc:1059] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:02:58.424058  5973 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:02:58.433182  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:58.434180  5972 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:02:58.434345  1717 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 11: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:58.436468  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:58.449390  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.452162  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 89698.2us
I20260709 13:02:58.380374  1607 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:58.355741  5647 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:58.367188  5858 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:58.382508  5839 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:58.367660  5885 consensus_queue.cc:1059] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:02:58.390043  1714 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
W20260709 13:02:58.470132  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 89852.7us
I20260709 13:02:58.367456  5937 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.402303  5617 raft_consensus.cc:3055] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:02:58.404415  5964 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.242s	user 0.002s	sys 0.000s
I20260709 13:02:58.414032  5969 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:02:58.455435  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:02:58.457734  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.462410  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:58.463208  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:58.466221  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:58.487850  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:02:58.487615  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.468039  2297 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 13:02:58.461498  5892 consensus_meta.cc:287] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.610s	user 0.004s	sys 0.008s
I20260709 13:02:58.487893  1642 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:58.475328  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:58.492710  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:58.495306  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:02:58.497606  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 43749.9us
W20260709 13:02:58.539376  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 46747.7us
W20260709 13:02:58.539991  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 50755.2us
I20260709 13:02:58.540536  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:02:58.486593  5867 consensus_queue.cc:1059] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:02:58.542040  5867 consensus_queue.cc:1243] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:02:58.542239  1561 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:58.542542  5867 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:58.542891  5969 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:58.536342  5461 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:58.543128  5867 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.544088  5461 raft_consensus.cc:3055] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:58.544445  5461 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:58.545439  5867 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:58.536224  5841 consensus_queue.cc:1059] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:02:58.545310  5461 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.546490  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:58.547204  1562 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:02:58.480820  5937 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:58.460726  5932 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:02:58.551218  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:58.467192  5858 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:58.561324  5973 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:58.562114  5973 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:02:58.562451  5973 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:58.563088  1718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.894s	user 0.006s	sys 0.015s
W20260709 13:02:58.564013  5968 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
I20260709 13:02:58.487623  5797 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:58.556959  5858 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:58.481161  5617 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:58.566495  5617 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.546756  5841 raft_consensus.cc:3055] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:02:58.567498  5841 raft_consensus.cc:740] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:58.567615  5617 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:58.568431  5841 consensus_queue.cc:260] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.569692  5841 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:02:58.545490  5940 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:58.571914  1590 debug-util.cc:398] Leaking SignalData structure 0x7b08003f8a40 after lost signal to thread 1501
W20260709 13:02:58.573288  1590 debug-util.cc:398] Leaking SignalData structure 0x7b08001bbda0 after lost signal to thread 1593
I20260709 13:02:58.573335  5940 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:58.573745  1590 debug-util.cc:398] Leaking SignalData structure 0x7b08001bbd80 after lost signal to thread 1594
I20260709 13:02:58.562772  5973 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:58.581578  5604 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:58.546789  5461 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:58.471128  5714 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:58.587333  1590 debug-util.cc:398] Leaking SignalData structure 0x7b080039ac00 after lost signal to thread 1668
W20260709 13:02:58.587742  1590 debug-util.cc:398] Leaking SignalData structure 0x7b080036f780 after lost signal to thread 1669
I20260709 13:02:58.551875  1566 raft_consensus.cc:2376] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9. Re-sending same reply.
W20260709 13:02:58.589421  1590 debug-util.cc:398] Leaking SignalData structure 0x7b080038e940 after lost signal to thread 1742
W20260709 13:02:58.589720  1590 debug-util.cc:398] Leaking SignalData structure 0x7b080026e900 after lost signal to thread 1743
I20260709 13:02:58.587356  5714 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:58.587664  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1110) took 2837 ms. Trace:
I20260709 13:02:58.591058  2301 rpcz_store.cc:276] 0709 13:02:55.750023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.750867 (+   844us) service_pool.cc:224] Handling call
0709 13:02:55.776083 (+ 25216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.166226 (+390143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.587400 (+2421174us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b5400516dd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:02:58.587579 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":708752640}
I20260709 13:02:58.553318  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:58.593230  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:58.594118  1637 raft_consensus.cc:2410] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:02:58.594244  1638 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:58.594698  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:58.595263  1639 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.595355  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:58.598886  1643 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:02:58.600028  1636 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 12: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:58.600713  1563 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:02:58.602402  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 49192.1us
I20260709 13:02:58.468423  5755 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.609277  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:58.619096  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:58.619808  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:58.642853  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:58.644708  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:58.652520  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:58.653709  2299 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:58.649435  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:58.671818  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:58.672820  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:58.678151  2299 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:02:58.680969  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:58.690976  1635 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:58.688057  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:58.693964  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:58.683073  1639 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:02:58.705343  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:58.706197  1567 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:58.717299  5636 raft_consensus.cc:3055] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:02:58.721052  2269 connection.cc:256] RPC call timeout handler was delayed by -1.3447385469999631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1617s for a response
W20260709 13:02:58.721526  2269 connection.cc:256] RPC call timeout handler was delayed by -1.007556516999955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
I20260709 13:02:58.648680  5870 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.735231  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:58.724665  1718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.056s	user 0.006s	sys 0.016s
I20260709 13:02:58.737056  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 837) took 2070 ms. Trace:
I20260709 13:02:58.665194  5886 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:58.651290  5689 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:58.673918  5936 raft_consensus.cc:3055] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:58.735458  1642 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:58.737515  1718 rpcz_store.cc:276] 0709 13:02:56.666669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.666868 (+   199us) service_pool.cc:224] Handling call
0709 13:02:56.667364 (+   496us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:56.667817 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.667859 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:56.668693 (+   834us) log.cc:844] Serialized 52 byte log entry
0709 13:02:58.736618 (+2067925us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:58.736674 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:58.736766 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:58.736811 (+    45us) raft_consensus.cc:1685] finished
0709 13:02:58.736823 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.736986 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25903,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"thread_start_us":2067467,"threads_started":1,"wal-append.queue_time_us":219832,"wal-append.run_cpu_time_us":2314,"wal-append.run_wall_time_us":1004900}
I20260709 13:02:58.725023  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:58.739151  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:58.740223  1590 debug-util.cc:398] Leaking SignalData structure 0x7b08003a11e0 after lost signal to thread 2329
I20260709 13:02:58.752071  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:58.752277  1590 debug-util.cc:398] Leaking SignalData structure 0x7b08001052e0 after lost signal to thread 5621
I20260709 13:02:58.752485  1640 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:02:58.752938  1590 debug-util.cc:398] Leaking SignalData structure 0x7b080022b540 after lost signal to thread 5733
W20260709 13:02:58.753275  1590 debug-util.cc:398] Leaking SignalData structure 0x7b08001b6300 after lost signal to thread 5739
W20260709 13:02:58.753594  1590 debug-util.cc:398] Leaking SignalData structure 0x7b08003dd7a0 after lost signal to thread 5762
W20260709 13:02:58.753829  1590 debug-util.cc:398] Leaking SignalData structure 0x7b080037a120 after lost signal to thread 5775
I20260709 13:02:58.607265  5912 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 13:02:58.722111  2269 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:02:58.754236  1590 debug-util.cc:398] Leaking SignalData structure 0x7b08002c6ea0 after lost signal to thread 5824
W20260709 13:02:58.754673  1590 debug-util.cc:398] Leaking SignalData structure 0x7b0800335900 after lost signal to thread 5825
I20260709 13:02:58.754194  1609 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:58.755491  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:58.756434  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:58.718588  5636 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:58.758744  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:58.759552  2296 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:58.756002  2269 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:58.745893  1566 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:58.682159  5839 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:58.671104  5903 consensus_queue.cc:1059] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:02:58.766990  2269 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:58.768144  1566 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:58.733466  5916 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:58.738346  5867 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:58.743892  5942 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:58.777684  5461 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:58.779240  5916 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:58.780146  5858 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.780596  5916 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.781059  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:58.782001  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:58.782316  5858 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:58.783186  2299 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:58.750432  5936 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:58.763957  5839 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.784279  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:58.788416  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:58.822077  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 281392us
I20260709 13:02:58.784723  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:58.827912  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.763031  5886 raft_consensus.cc:697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:58.845638  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:58.830572  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:58.763195  5797 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.769059  5869 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:58.836166  1683 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:58.750841  5870 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:58.749719  5937 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:58.861419  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:58.862707  1561 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:02:58.846799  5839 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.785234  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:02:58.837316  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 381293us
I20260709 13:02:58.866938  1532 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:58.855830   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:02:58.848730  5903 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:58.875422  1532 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:58.863519  5907 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:58.877849  5907 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:58.878335  5907 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:58.880896  5944 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:58.882230  1532 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:58.843358  2300 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:58.766858  5978 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:58.760270  5689 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.829109  5942 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.786054  5867 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.840705  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:58.865260  1682 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:58.854985  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:58.931721  1565 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:02:58.887909  5933 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:58.837024  5636 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:58.902968  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.940196  2304 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:02:58.940447  1565 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:58.889248  5969 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:58.928038  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:58.889072  1532 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:58.946187  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 57207.5us
I20260709 13:02:58.947829  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:58.951143  5636 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:58.951494  5984 log.cc:927] Time spent T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:02:58.955019  5689 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:58.955039  5867 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:58.970399  5944 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:02:58.971482  5985 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:02:58.939122  5933 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:58.865676  5936 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:58.973501  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:58.975060  5933 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:58.980374  5932 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:58.981194  5932 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:58.982332  5932 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:58.984587  5983 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:02:58.910667  5979 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.996537  5944 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:58.941697  2304 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:58.942997  2299 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:58.903525  5978 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:58.944276  1566 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:58.975584  5912 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:02:58.889820  1681 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:58.988829  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:58.989166  1638 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:02:58.887084  5940 consensus_queue.cc:1059] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.125s
I20260709 13:02:58.851472  5438 consensus_queue.cc:1059] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
W20260709 13:02:58.998034  1684 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.017s	sys 0.051s
I20260709 13:02:58.887629  5654 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:02:59.010521  5978 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.008961  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:59.007429  5990 consensus_queue.cc:1059] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:02:59.031358  5990 consensus_queue.cc:1243] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:02:58.890455  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:59.022706  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:59.032327  5629 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:59.033128  5912 consensus_queue.cc:1243] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:02:59.005153  5979 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.033649  1638 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:58.998385  5905 raft_consensus.cc:993] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:02:59.035187  5979 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:58.936405  5886 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.013196  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:58.891793  5973 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:59.008224  5755 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:59.035795  5905 raft_consensus.cc:1081] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
W20260709 13:02:59.026727  1684 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.018s	sys 0.051s
I20260709 13:02:59.004232  5243 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:02:58.877116  5797 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.039448  2302 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 7.
I20260709 13:02:58.853983  5885 raft_consensus.cc:3055] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:02:59.047777  5870 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:59.023149  5647 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:02:59.057818  5243 raft_consensus.cc:697] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:59.058279  1718 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:59.059068  5243 consensus_queue.cc:237] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:59.062036  5859 consensus_meta.cc:287] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.681s	user 0.006s	sys 0.003s
I20260709 13:02:59.049903  5885 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:59.062526  5859 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.063869  1644 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 10: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:02:59.063532  5885 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.063957  5937 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:59.064232  1568 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:02:59.022835  5903 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 13:02:59.065498  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 52456.2us
I20260709 13:02:59.065577  5647 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:59.065867  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:59.066293  5647 raft_consensus.cc:697] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:59.067310  5937 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.067838  5937 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.068713  5859 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:59.069470  5973 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:59.064919  5885 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:02:59.080042  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:59.080869  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:59.074163  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:59.074906  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:59.082198  1715 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:02:59.082302  1718 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 12: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:02:59.078184  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:59.083036  1711 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:59.079136  1712 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:02:59.079970  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:59.072793  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:59.081566  2268 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:59.097204  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:59.100056  2300 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:59.122869  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:59.039373  5935 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:02:59.129204  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:59.062053  5797 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:59.140909  1565 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:02:59.080904  5937 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:59.130882  5867 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:59.066712  5933 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.156289  1711 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:02:59.159653  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:59.160616  5461 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:59.161115  5461 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:59.137282  5869 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:02:59.144999  5993 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:02:59.172940  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 406060us
I20260709 13:02:59.067272  5647 consensus_queue.cc:237] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:59.173251  5461 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.176954  5933 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:59.179911  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:59.183054  1644 raft_consensus.cc:2393] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:02:59.166363  5867 raft_consensus.cc:697] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:02:59.187422  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 322235us
I20260709 13:02:59.173913  2269 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:59.192109  5942 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:59.195834  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.188313  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:59.198196  1644 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:02:59.198844  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:59.199642  1590 debug-util.cc:398] Leaking SignalData structure 0x7b08002face0 after lost signal to thread 5833
I20260709 13:02:59.205792  5933 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.195291  2269 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:02:59.201588  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.214450  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:59.217195  2305 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:02:59.196761  5838 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:59.206547  5933 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:59.181865  5993 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:59.224578  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:59.225175  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.225203  1644 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:02:59.227505  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:59.229007  5892 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:59.229453  5892 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:02:59.229097  5969 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:02:59.230860  5969 raft_consensus.cc:697] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:59.231017  5867 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.231747  5969 consensus_queue.cc:237] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:59.234762  5982 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.674s	user 0.001s	sys 0.006s
I20260709 13:02:59.238456  5933 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:59.242038  5469 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:59.242653  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:59.245085  1644 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:59.247931  1644 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:02:59.250448  1644 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 7. Current term is 8. Ops: []
I20260709 13:02:59.230247  5892 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:59.223827  5979 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:59.260478  1644 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:02:59.207912  5839 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:59.269097  1644 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:02:59.254086  5944 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.250960  5933 consensus_queue.cc:1059] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:02:59.266012  5892 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.255990  5940 consensus_queue.cc:1243] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:02:59.268998  5979 raft_consensus.cc:697] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:02:59.294126  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:59.295769  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:59.296463  1712 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
W20260709 13:02:59.297904  1562 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.002s	sys 0.011s
W20260709 13:02:59.298409  1562 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.003s	sys 0.011s
W20260709 13:02:59.299489  1563 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.007s	sys 0.006s
W20260709 13:02:59.301398  1563 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.007s	sys 0.006s
I20260709 13:02:59.293437  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:02:59.254820  5992 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:02:59.266244  5905 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:02:59.312067  1567 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:59.315683  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 425973us
I20260709 13:02:59.317462  1681 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:59.344343  1681 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:59.350935  6010 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:59.351527  6010 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.351945  6010 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:59.355252  1681 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:59.356130  5243 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:59.356626  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:59.357846  1681 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:59.347381  5942 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:59.369413  6017 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:59.370260  6017 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.282011  6009 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:02:59.277662  5944 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:59.339303  5996 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:02:59.307250  5617 consensus_queue.cc:1059] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 13:02:59.380817  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.382398  1681 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:02:59.392244  5988 log.cc:927] Time spent T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.456s	user 0.000s	sys 0.002s
I20260709 13:02:59.394385  5654 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:59.397672  1681 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:59.399359  1681 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:59.401072  1681 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:02:59.401394  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 62707.2us
I20260709 13:02:59.402336  1532 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:59.395076  5654 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:59.410641  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:59.421185  5654 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:59.421377  6003 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:59.412945  5916 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:59.423338  5916 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:59.423749  5916 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:59.432253  1532 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:02:59.406062  5942 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:59.437935  1564 raft_consensus.cc:3055] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:02:59.438393  1564 raft_consensus.cc:740] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:59.439311  5932 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:59.439985  5967 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:02:59.440054  5932 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.405963  5438 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:59.439482  5991 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:02:59.412339  6017 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:59.477478  6013 log.cc:927] Time spent T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:02:59.442080  1564 consensus_queue.cc:260] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:59.482096  5994 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:59.415230  5992 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:02:59.443252  5932 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:59.408100  6009 consensus_queue.cc:1059] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:02:59.503612  1564 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:02:59.483739  5907 consensus_meta.cc:287] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.001s
W20260709 13:02:59.485093  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.527534  1709 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:59.439329  6007 consensus_queue.cc:1059] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.125s
I20260709 13:02:59.413067  5979 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.529937  5907 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.532946  5907 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:59.534508  1564 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:02:59.536180  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:59.536844  2298 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:02:59.549901  6014 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 13:02:59.557503  1713 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
W20260709 13:02:59.587121  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.528275  5839 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:59.600076  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:59.600932  1638 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:02:59.604629  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:02:59.606065  6006 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 13:02:59.594247  5886 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.607438  5992 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:59.607914  5992 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:59.608119  6012 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:02:59.608282  5992 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:59.609438  6010 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.607450  5886 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.613116  5886 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:59.613888  6010 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:59.623943  5992 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.628072  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:59.628947  1643 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:59.630898  2298 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:02:59.536024  5978 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.643028  1643 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:02:59.645607  5944 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:59.648821  5993 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:59.644078  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:59.626487  5992 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:59.661969  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:59.665349  2298 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:02:59.666116  6029 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:59.664496  5647 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:59.667080  5647 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:02:59.668116  5647 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:02:59.670439  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.666723  6029 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.644721  5839 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:02:59.673452  6027 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:59.674041  6027 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.676170  6027 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:02:59.677030  1564 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:02:59.678308  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:59.678637  5749 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:59.679291  5749 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.680104  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2614) took 1073 ms. Trace:
I20260709 13:02:59.680473  1568 rpcz_store.cc:276] 0709 13:02:58.607082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.616765 (+  9683us) service_pool.cc:224] Handling call
0709 13:02:59.064051 (+447286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.064673 (+   622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.679921 (+615248us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54003e8350. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:02:59.680045 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1027717888}
I20260709 13:02:59.684181  5936 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:59.684917  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:02:59.685600  1709 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:02:59.687273  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:02:59.690106  1567 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:59.691447  1643 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:02:59.693841  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:59.696960  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:59.697651  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:59.698367  1565 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:59.698652  5749 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:59.698745  5944 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.699471  5749 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:59.700757  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:59.701514  1638 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:02:59.702728  1568 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:59.704679  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:59.709995  1565 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:02:59.699568  5944 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.714140  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:59.720054  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:02:59.720638  1561 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:59.719333  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 193500us
I20260709 13:02:59.723155  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:02:59.724733  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.725431  1565 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:59.717908  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:59.726191  1562 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
W20260709 13:02:59.733175  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.733701  1716 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:59.701751  5894 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.713248  6027 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:59.701524  5993 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
W20260709 13:02:59.740288  5604 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.158s	user 0.011s	sys 0.019s
I20260709 13:02:59.712723  5839 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.759944  2305 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:59.760077  1561 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:02:59.762042  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:59.739773  6027 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:59.745906  6029 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:02:59.752444  5604 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.170s	user 0.011s	sys 0.019s
I20260709 13:02:59.751546  5994 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:59.769750  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:59.777231  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:02:59.781999  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:02:59.782595  1635 raft_consensus.cc:1275] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:02:59.744042  5894 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:59.786121  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:02:59.753067  6028 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:02:59.755013  5979 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:02:59.765916  6027 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:59.761633  6038 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:59.781121  5714 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.810746  6038 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.808149  6028 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:02:59.793704  5617 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:02:59.810442  5979 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.833015  2298 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:02:59.811298  5994 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.812727  5942 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.814224  6042 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.877781  2298 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:02:59.864033  5243 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:59.823959  6009 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:02:59.860176  5993 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:02:59.886060  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:59.827901  5992 consensus_queue.cc:1059] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
W20260709 13:02:59.888010  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 67868us
I20260709 13:02:59.818216  5916 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:02:59.891791  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 105739us
I20260709 13:02:59.866531  5978 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:59.880193  5749 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:59.897363  6037 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:02:59.899231  6054 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:02:59.899847  6054 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.900236  5935 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:59.901036  5935 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.901475  5935 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:59.902091  6054 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:59.904758  5935 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:59.906363  5629 raft_consensus.cc:3055] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:59.906826  5629 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:02:59.907752  5629 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.912051  5944 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:02:59.912711  5981 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 1.140s	user 0.001s	sys 0.003s
I20260709 13:02:59.891642  6007 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.917097  6007 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:59.827714  5714 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.920394  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:02:59.921802  1638 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:02:59.924019  1636 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:02:59.925136  1562 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:59.926355  1564 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:59.866974  5942 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:59.888520  5932 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:02:59.907778  6041 log.cc:927] Time spent T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:02:59.932677  1635 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:02:59.931852  1533 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:02:59.866811  5994 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:02:59.939311  5839 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:02:59.938942  6046 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:59.879913  6042 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:02:59.944245  6007 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:59.938880  6056 consensus_queue.cc:1059] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:02:59.929911  5617 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.947269  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:02:59.934244  5936 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:02:59.949407  5469 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:59.950600  1569 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:02:59.939199  6035 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:59.983781  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.998584  5994 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:02:59.964259  5602 raft_consensus.cc:3055] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:00.017418  2304 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:03:00.028098  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:02:59.952044  6046 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:00.043260  5916 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:00.035480  5915 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.046186  5915 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.048669  6047 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:03:00.049093  5915 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:00.049606  6047 consensus_queue.cc:1243] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:02:59.999235  5617 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.050187  6047 raft_consensus.cc:3055] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:00.050567  6047 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:00.052160  5617 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:00.019320  5602 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:00.054636  5602 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.056421  5602 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:00.063941  6038 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:00.069806  5749 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:00.071256  6048 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:03:00.070616  5749 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:00.007910  6035 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:00.081717  6035 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:00.082512  5942 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:00.083392  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:00.084069  1644 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:03:00.084897  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:00.085994  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:00.086700  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:00.087141  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:00.087455  6024 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.713s	user 0.003s	sys 0.002s
I20260709 13:03:00.089099  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 909) took 1495 ms. Trace:
I20260709 13:03:00.071802  5940 raft_consensus.cc:3055] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:00.089902  5797 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:00.090378  5989 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:03:00.090893  5797 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.091727  1641 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:00.092777  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:00.093093  5797 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:00.093158  5469 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:00.093801  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:00.094412  1638 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:00.024703  5979 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:00.024147  6028 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:00.030705  6046 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:00.095263  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:00.097087  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:00.072589  5932 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:00.089483  1640 rpcz_store.cc:276] 0709 13:02:58.593581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.596191 (+  2610us) service_pool.cc:224] Handling call
0709 13:02:58.752302 (+156111us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.752984 (+   682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.088904 (+1335920us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54002d8550. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:00.089040 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":305444608}
I20260709 13:03:00.102180  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:00.105337  1638 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:00.093374  1641 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
W20260709 13:03:00.106506  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.110773  2305 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:00.087414  1643 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:00.086725  1644 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:00.117101  1609 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:00.127758  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.134980  2305 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:00.136555  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 348260us
I20260709 13:03:00.138350  1640 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:03:00.147284  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:00.151057  2269 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:00.152627  1639 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:03:00.156564  2297 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:00.089689  6042 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:00.090395  5940 raft_consensus.cc:740] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:00.163282  4544 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:00.165874  4544 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:00.166726  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.133203  5979 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:00.169866  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:00.177556  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.190264  1569 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: [10.11-12.12]
I20260709 13:03:00.190805  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:00.191733  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:00.191955  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:00.192456  1567 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:00.194303  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:00.192742  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:00.192384  1569 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:03:00.196983  1562 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:00.197639  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:00.195375  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:00.169693  6029 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:00.204691  4544 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:00.165207  5724 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:00.203081  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:00.206661  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:00.135020  6027 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.208074  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:00.209154  1566 raft_consensus.cc:2410] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:03:00.210857  1560 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:00.211014  1567 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
W20260709 13:03:00.213835  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 57625.3us
I20260709 13:03:00.215328  2269 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:00.175603  5979 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.181001  6054 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.226019  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:00.196326  5912 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.262513  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:00.267442  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:00.225441  5934 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:00.225245  6038 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:00.275827  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:00.276057  6038 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:00.276376  6052 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:00.276556  6038 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:00.276968  6052 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:00.279026  6039 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.114s
I20260709 13:03:00.223423  5724 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:00.280762  6039 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:00.281179  6039 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:00.204416  5461 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:03:00.281600  6039 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:00.283053  6059 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:00.283499  6059 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.283881  6059 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:00.175024  6047 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.286221  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:00.290660  1560 raft_consensus.cc:2368] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11.
I20260709 13:03:00.223810  6029 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:00.298579  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 268667us
I20260709 13:03:00.288461  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:00.305115  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:00.308602  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:00.263710  6056 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:03:00.281491  5724 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:00.312232  2296 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:00.289398  5934 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.251550  6054 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.266258  5912 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.283984  5749 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.330958  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:00.333141  2296 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
W20260709 13:03:00.335376  6068 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:03:00.271814  5940 consensus_queue.cc:260] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.320292  5934 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:00.322942  5967 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:00.348982  5967 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:00.329334  5933 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:00.351373  5933 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:00.354478  5933 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:00.351482  6034 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:00.357026  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:00.357826  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.358044  2296 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:00.363991  6027 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:00.326685  5979 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:00.370790  1567 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:03:00.372972  5986 log.cc:927] Time spent T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 1.603s	user 0.012s	sys 0.000s
I20260709 13:03:00.384198  5937 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:00.331216  6052 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:00.375880  6011 raft_consensus.cc:3055] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:00.389950  5937 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:00.398245  6011 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:00.402220  6011 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.403453  6011 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:00.395430  6029 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.369817  6034 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.410037  6034 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:00.407446  5937 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:00.411099  5937 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:00.421948  1638 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:00.432602  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:00.530496  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 97990.3us
W20260709 13:03:00.536495  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.537343  1635 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:00.547173  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:00.549226  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:00.547493  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 332265us
I20260709 13:03:00.550180  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:00.550832  2269 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:00.552824  1563 raft_consensus.cc:2410] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:03:00.550781  1569 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:03:00.553076  1642 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:03:00.560719  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:00.566083  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:00.576917  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:00.526391  5869 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.580987  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:00.582267  1566 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:00.583329  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:00.552670  5884 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:00.598196  1643 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:03:00.569348  5999 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:00.597325  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:00.607538  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.410043  6038 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:00.606498  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:00.534210  5870 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:03:00.609118  1644 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:00.609508  1636 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 10: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:00.609510  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:00.610548  1640 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:03:00.594651  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.580904  5745 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:00.590075  5979 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:00.587112  5867 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:00.561628  6034 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:03:00.587898  5724 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:00.590763  5841 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:00.552562  6059 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.558218  6052 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:03:00.605183  6073 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:00.604902  6062 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.623006  6077 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:03:00.617241  5967 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:00.613718  5859 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.591315  6027 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:03:00.651269  6057 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.608188  5999 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:00.651964  6057 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:00.652645  5999 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:00.612008  5884 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:00.654428  5884 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:00.642989  6075 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:03:00.616493  5978 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:00.624641  5979 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.595021  5869 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.619597  5604 consensus_queue.cc:1059] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:03:00.640244  5841 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:00.646625  5867 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:00.667057  6057 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:00.624599  5745 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:00.685474  6062 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:00.738127  6073 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:00.753116  6046 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.658s	user 0.012s	sys 0.000s
I20260709 13:03:00.775597  6038 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:00.775490  6046 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.776602  5749 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:00.778405  6046 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:00.782014  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.740610  5979 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:00.740957  5875 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.816869  2301 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:00.644631  5724 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.770720  6073 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:00.767670  6062 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:00.751784  5841 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:00.736159  5859 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:00.809734  5867 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:00.894443  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:00.895061  1711 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:00.896080  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:00.902050  1711 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:00.896034  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:00.905275  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:00.907155  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:00.908042  1718 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:00.908927  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:00.909865  1712 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:00.910251  1710 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:03:00.920320  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:00.921327  1713 raft_consensus.cc:2410] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 13:03:00.906945  6011 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.000s
I20260709 13:03:00.839318  5875 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:00.912042  5937 consensus_meta.cc:287] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.500s	user 0.011s	sys 0.000s
W20260709 13:03:00.927448  6039 consensus_meta.cc:287] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.645s	user 0.012s	sys 0.000s
I20260709 13:03:00.893172  6073 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:03:00.944018  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1212) took 1186 ms. Trace:
I20260709 13:03:00.910324  6034 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:03:00.947014  2298 rpcz_store.cc:276] 0709 13:02:59.757187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.757391 (+   204us) service_pool.cc:224] Handling call
0709 13:03:00.943828 (+1186437us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b5400368b54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:00.943954 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110510848}
I20260709 13:03:00.925675  6059 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:00.962590  1712 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:00.965266  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:00.952935  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 343502us
W20260709 13:03:00.978204  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.978694  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:00.983826  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:00.984547  1710 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:00.984943  1712 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:03:00.950196  6073 consensus_queue.cc:1243] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:03:00.960361  5934 consensus_meta.cc:287] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.004s
I20260709 13:03:00.997292  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:00.997965  1713 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:00.999192  1710 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:00.999264  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:01.000017  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 1001) took 1633 ms. Trace:
I20260709 13:03:01.003587  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:01.003964  6077 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:01.004496  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:00.995671  6034 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:01.005988  6034 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:01.006343  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:01.006961  6034 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:01.007953  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:01.008250  6034 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:01.011296  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:01.014904  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:01.016213  5869 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:01.017078  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:01.017581  6082 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:03:01.023238  5934 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:01.025640  5934 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:01.016271  5745 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:00.955319  5867 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:01.002429  6075 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:01.039027  6075 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:01.043876  5993 raft_consensus.cc:1064] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
W20260709 13:03:01.044236  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.053416  1713 raft_consensus.cc:2410] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:01.000460  2299 rpcz_store.cc:276] 0709 13:02:59.366159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.366330 (+   171us) service_pool.cc:224] Handling call
0709 13:02:59.384600 (+ 18270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.384867 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:59.403325 (+ 18458us) log.cc:844] Serialized 14 byte log entry
0709 13:03:00.531009 (+1127684us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b1800385420. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb1455 00007fd7a1c99060 00007fd7a1cab716 00007fd7a1cbc287
0709 13:03:00.735730 (+204721us) log.cc:844] Serialized 14 byte log entry
0709 13:03:00.999695 (+263965us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:00.999747 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:00.999789 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.999857 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:00.999966 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71908,"prepare.queue_time_us":107791,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":3728,"raft.queue_time_us":132948,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":20531,"spinlock_wait_cycles":30613248,"thread_start_us":381610,"threads_started":3}
I20260709 13:03:01.057331  2268 leader_election.cc:400] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11.
I20260709 13:03:01.059444  2268 leader_election.cc:403] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:01.102005  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.148536  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 72486.6us
W20260709 13:03:01.149220  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 247024us
W20260709 13:03:01.150717  1607 leader_election.cc:336] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.985s (SENT)
W20260709 13:03:01.151906  1607 leader_election.cc:336] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.231s (SENT)
I20260709 13:03:00.947728  6039 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:01.183410  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:01.218092  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 68421.6us
I20260709 13:03:01.288015  1641 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:03:01.290598  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:01.154130  5838 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11.
I20260709 13:03:01.297748  1641 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:03:01.297402  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:01.149371  5858 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:01.152393  6056 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:01.217264  5999 consensus_meta.cc:287] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.000s
W20260709 13:03:01.392531  5979 consensus_meta.cc:287] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.550s	user 0.005s	sys 0.004s
I20260709 13:03:01.329422  6085 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.147s
I20260709 13:03:01.366325  5724 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:01.295408  5934 consensus_queue.cc:1059] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.213s
I20260709 13:03:01.046875  5993 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:01.482998  6056 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:01.499953  5999 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:01.580499  6056 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:01.316725  5841 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:01.614696  5979 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:01.626111  6034 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.002s
W20260709 13:03:01.637878  5867 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.002s	sys 0.022s
W20260709 13:03:01.638639  5867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.002s	sys 0.022s
W20260709 13:03:01.751761  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.766609  6110 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:01.770548  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2256) took 2073 ms. Trace:
I20260709 13:03:01.771328  1564 rpcz_store.cc:276] 0709 13:02:59.696930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.697799 (+   869us) service_pool.cc:224] Handling call
0709 13:02:59.698062 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:59.926787 (+228725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.769579 (+1842792us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54002977d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:01.770285 (+   706us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":524594432}
I20260709 13:03:01.774071  6110 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:01.779502  6110 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:01.793304  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.794104  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 642131us
W20260709 13:03:01.809257  1637 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.022s	sys 0.047s
W20260709 13:03:01.811257  1637 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.023s	sys 0.047s
I20260709 13:03:01.813777  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:01.814085  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:01.857937  5724 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.860467  5724 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:01.870525  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.879608  1641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60390
I20260709 13:03:01.880193  6109 consensus_queue.cc:1059] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:01.880820  1641 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:01.881381  1641 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:01.881462  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1166) took 1741 ms. Trace:
W20260709 13:03:01.885334  2296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.241s	user 0.002s	sys 0.014s
I20260709 13:03:01.885532  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:01.885915  2296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.276s	user 0.002s	sys 0.014s
I20260709 13:03:01.887601  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:01.887925   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:01.888576  1644 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:01.889595  5867 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:01.891039  5867 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:01.891669  5724 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:01.869388  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 103081us
I20260709 13:03:01.895728  6110 consensus_queue.cc:1059] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:01.882088  2305 rpcz_store.cc:276] 0709 13:03:00.139915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.140055 (+   140us) service_pool.cc:224] Handling call
0709 13:03:00.140413 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.223316 (+ 82903us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.223347 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:00.223390 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:00.223421 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.880847 (+1657426us) spinlock_profiling.cc:243] Waited 82.7 ms on lock 0x7b5400642154. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:01.880915 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.881186 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190171392}
I20260709 13:03:01.907207  6111 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.908126  6111 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:01.908506  5867 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:01.896133  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:01.911306  6088 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:01.911695  1637 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:01.911887  6088 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:01.912339  6088 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:01.879649  1716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.002s	sys 0.014s
W20260709 13:03:01.915704  1716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.009s	user 0.002s	sys 0.014s
I20260709 13:03:01.917277  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2237) took 1022 ms. Trace:
W20260709 13:03:01.920430  1682 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.001s	sys 0.008s
W20260709 13:03:01.920938  1682 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.002s	sys 0.009s
W20260709 13:03:01.921425  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 701724us
I20260709 13:03:01.923875  1644 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:01.924429  5884 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:01.927927  6109 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:01.928849  6109 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:01.930343  1644 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 13:03:01.930889  1644 pending_rounds.cc:85] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 10
I20260709 13:03:01.931247  1644 pending_rounds.cc:107] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 13:03:01.931897  6110 raft_consensus.cc:3055] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:01.932332  1644 raft_consensus.cc:2889] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:01.932492  6110 raft_consensus.cc:740] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:01.935806  1644 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:01.939180  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1182) took 1544 ms. Trace:
I20260709 13:03:01.944319  2296 rpcz_store.cc:276] 0709 13:03:00.394994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.396459 (+  1465us) service_pool.cc:224] Handling call
0709 13:03:00.396927 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.397241 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:00.499889 (+102648us) log.cc:844] Serialized 14 byte log entry
0709 13:03:00.609315 (+109426us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.479392 (+870077us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b18004300c0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb4522 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:03:01.886537 (+407145us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:01.886663 (+   126us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:01.886798 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.893227 (+  6429us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.939102 (+ 45875us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3733,"prepare.queue_time_us":140665,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":25848,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":21239,"spinlock_wait_cycles":4039936,"thread_start_us":1380546,"threads_started":2,"wal-append.queue_time_us":367258}
I20260709 13:03:01.917695  1716 rpcz_store.cc:276] 0709 13:03:00.895030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.895143 (+   113us) service_pool.cc:224] Handling call
0709 13:03:00.895438 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.905490 (+ 10052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.905521 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:00.905984 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.916496 (+1010512us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:01.916620 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.916751 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.916831 (+    80us) raft_consensus.cc:1685] finished
0709 13:03:01.916843 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.917084 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21427,"raft.queue_time_us":10314,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":8433,"thread_start_us":1009846,"threads_started":1,"wal-append.queue_time_us":680473}
I20260709 13:03:01.952411  5993 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.952749  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 766) took 1691 ms. Trace:
I20260709 13:03:01.953509  1567 rpcz_store.cc:276] 0709 13:03:00.260648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.266159 (+  5511us) service_pool.cc:224] Handling call
0709 13:03:00.266456 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.380457 (+114001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.951737 (+1571280us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400217354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:01.952454 (+   717us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238651264}
I20260709 13:03:01.934445  6110 consensus_queue.cc:260] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:01.958741  6110 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:01.953794  5993 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:01.969117  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:01.964927  1641 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:01.976442  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 932) took 1088 ms. Trace:
I20260709 13:03:01.976992  1643 rpcz_store.cc:276] 0709 13:03:00.886328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.886566 (+   238us) service_pool.cc:224] Handling call
0709 13:03:00.887046 (+   480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.892230 (+  5184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.892280 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:00.892867 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.161019 (+268152us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:01.970885 (+809866us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.974469 (+  3584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.974567 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.974642 (+    75us) raft_consensus.cc:1685] finished
0709 13:03:01.974656 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.974895 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16870,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":9623,"thread_start_us":341565,"threads_started":2,"wal-append.queue_time_us":646747}
I20260709 13:03:01.980310  1643 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:03:01.988427  1641 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:01.988942  5993 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:01.989586  1637 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:02.000149  1636 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:03:02.002667  6007 raft_consensus.cc:3055] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:02.003209  6007 raft_consensus.cc:740] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:02.004794  6007 consensus_queue.cc:260] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:02.009713  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.010555  1641 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
W20260709 13:03:01.998662  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.013442  1644 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:02.006788  6007 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:02.019330  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.020279  1637 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:02.020316  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.021303  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:02.021624  1643 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:02.024147  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.024232  5999 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.027458  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.028329  1643 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:02.028400  1641 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:03:02.029605  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.029659  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:01.971819  6111 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:02.031446  1561 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.005s	sys 0.006s
I20260709 13:03:02.026723  5689 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:03:02.031991  1561 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.049s	user 0.006s	sys 0.006s
I20260709 13:03:02.032218  5689 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.032642  6034 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:02.033620  6034 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.036010  1636 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:02.045709  6109 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.047312  6053 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:02.051047  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2260) took 1505 ms. Trace:
I20260709 13:03:02.052585  6034 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.051700  1561 rpcz_store.cc:276] 0709 13:03:00.545629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.546206 (+   577us) service_pool.cc:224] Handling call
0709 13:03:00.546507 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.546705 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:03:00.981997 (+435292us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.050040 (+1068043us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.050345 (+   305us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:02.050506 (+   161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.050698 (+   192us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.050957 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17186,"prepare.queue_time_us":77264,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"thread_start_us":1460524,"threads_started":2,"wal-append.queue_time_us":927048}
I20260709 13:03:02.055514  1641 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:02.060544  5993 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.034713  5689 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:02.068918  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:02.071835  1637 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:02.080400  6112 raft_consensus.cc:3055] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:02.079993  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:02.081202  6112 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:02.086455  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.088805  6028 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:02.089918  6028 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.092681  6028 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.093338  5995 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:02.086966  6111 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.115399  6111 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:02.117915  6111 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.075603  5993 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.119822  5875 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.121071  5993 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:02.104468  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 135664us
I20260709 13:03:02.082693  6112 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.108212  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.127885  1641 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:03:02.128257  6112 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.118811  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 924) took 2033 ms. Trace:
I20260709 13:03:02.109812  5859 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:02.112731  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.130612  1638 rpcz_store.cc:276] 0709 13:03:00.084955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.113815 (+ 28860us) service_pool.cc:224] Handling call
0709 13:03:00.421589 (+307774us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.422438 (+   849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.118380 (+1695942us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b5400247a50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:02.118618 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626777984}
W20260709 13:03:02.139756  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.139681  6088 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:02.142136  6088 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.138469  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:02.147713  5724 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:03:02.148308  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 52778.4us
I20260709 13:03:02.148588  5724 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:02.149894  5839 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.150856  5839 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.153883  5841 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.154419  5839 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.155303  5839 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:02.155578  6039 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:02.150846  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 74859us
I20260709 13:03:02.142782  6088 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.158211  1637 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:02.156373  2298 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:02.166561  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.166693  2304 raft_consensus.cc:1240] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:02.167685  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.167519  6088 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:02.169310  2304 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:02.169926  6088 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.170689  1643 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: [9.10-9.10]
W20260709 13:03:02.171424  6114 log.cc:927] Time spent T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:03:02.171306  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.172124  1641 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:02.173007  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:02.174032  6113 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:02.174863  6113 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:02.175555  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.177816  6113 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.178754  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.179741  1639 raft_consensus.cc:2393] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 12: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:02.170697  5724 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.188712  2299 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:02.189628  5461 raft_consensus.cc:3055] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:02.190236  5461 raft_consensus.cc:740] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:02.191362  1607 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:02.193123  6117 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:02.194325  6117 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:02.195091  6039 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:02.196434  1534 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:02.196097  6039 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.197558  6117 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.197690  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:02.199262  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.199426  6116 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:02.200031  1638 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:02.133136  6053 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:02.202188  6053 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.203259  6110 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:02.203827  6110 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.204357  6110 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.204885  6053 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.206730  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.207370  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.208067  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.208312  1643 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:02.209021  6028 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.209360  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.209481  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:02.209960  6028 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.211472  5942 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:02.212174  5942 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.213363  5243 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.214119  5243 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:02.215307  6007 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:02.216147  6007 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.217013  5243 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:02.217721  5243 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:02.151079  5875 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:02.200078  6116 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.198393  6117 consensus_queue.cc:1059] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:03:02.210659  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.194629  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:02.188861  1639 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 13:03:02.220057  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.230581  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:02.226172  2304 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:03:02.225307  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:02.238588  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 925) took 1638 ms. Trace:
I20260709 13:03:02.238739  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:02.239146  1635 rpcz_store.cc:276] 0709 13:03:00.595600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.595740 (+   140us) service_pool.cc:224] Handling call
0709 13:03:00.614844 (+ 19104us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:00.981182 (+366338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.981214 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:00.993748 (+ 12534us) log.cc:844] Serialized 52 byte log entry
0709 13:03:02.074577 (+1080829us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:02.091423 (+ 16846us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.091961 (+   538us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.092121 (+   160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.092240 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.180731 (+ 88491us) raft_consensus.cc:1685] finished
0709 13:03:02.180761 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.234257 (+ 53496us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1928,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":69,"raft.queue_time_us":173092,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":17599,"thread_start_us":613186,"threads_started":2,"wal-append.queue_time_us":866078}
I20260709 13:03:02.240578  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:02.221370  5942 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.228673  6048 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.229841  5838 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.221143  5714 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.220165  5875 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.241640  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.247360  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:02.225131  6069 log.cc:927] Time spent T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 1.939s	user 0.009s	sys 0.045s
I20260709 13:03:02.223814  5993 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:03:02.255159  5885 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:02.218895  5438 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:02.224005  6059 consensus_queue.cc:1059] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:03:02.257064  5438 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.257931  6059 raft_consensus.cc:3055] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:02.258368  6059 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:02.258710  5894 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.221874  6028 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.262240  1639 raft_consensus.cc:2410] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:03:02.156419  5839 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:03:02.223639  5995 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:02.218835  6007 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.234735  6039 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.220767  6116 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.221709  5243 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.241518  5897 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:02.280681  6053 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:02.281010  5979 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.281497  6053 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:02.282049  5243 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.273408  5995 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.287118  5915 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.309799  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.259526  6059 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:02.244550  5838 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.284436  6113 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:02.243748  6048 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.291522  5995 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.297479  5875 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.284613  6028 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:02.346764  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.329125  6007 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.347694  1643 raft_consensus.cc:2368] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11.
W20260709 13:03:02.346062  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51501.1us
I20260709 13:03:02.341041  6117 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.355664  5438 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.356101  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.357340  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:02.358192  5838 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.359114  6007 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:02.361824  6007 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:02.363178  6029 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.295s	user 0.016s	sys 0.041s
W20260709 13:03:02.363646  6029 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.410s	user 0.016s	sys 0.041s
I20260709 13:03:02.366123  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1208) took 2876 ms. Trace:
I20260709 13:03:02.355497  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:02.349633  5915 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:02.367614  1637 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:02.366544  2302 rpcz_store.cc:276] 0709 13:02:59.409740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.410432 (+   692us) service_pool.cc:224] Handling call
0709 13:02:59.420889 (+ 10457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:59.511951 (+ 91062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:59.512003 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:00.064636 (+552633us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b2c00047a38. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:03:00.065207 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:03:00.203992 (+138785us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:00.204029 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:00.204208 (+   179us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.224048 (+2019840us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:02.260986 (+ 36938us) raft_consensus.cc:1685] finished
0709 13:03:02.261007 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.285670 (+ 24663us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11086,"raft.queue_time_us":134869,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":35830,"spinlock_wait_cycles":35927168,"thread_start_us":223351,"threads_started":2,"wal-append.queue_time_us":219860}
I20260709 13:03:02.369040  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:02.369707  5915 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.363651  1533 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:02.371276  6118 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:02.372044  6118 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.370673  5915 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:02.351653  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 213367us
I20260709 13:03:02.358821  1643 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:03:02.384260  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 143783us
I20260709 13:03:02.370311  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:02.393253  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:02.406071  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.426367  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:02.391440  5838 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.391145  6029 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:02.431427  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:02.436851  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:02.438403  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:02.439980  1717 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:02.440049  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:02.441402  1683 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:02.442314  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:02.414804  6017 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:02.380668  6048 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.443536  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:02.444478  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:02.445515  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:02.430419  1533 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:02.463660  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 53907.1us
I20260709 13:03:02.446727  6028 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.413002  5875 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:02.448916  5940 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:02.450659  2302 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.374421  5915 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:02.446532  5886 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:02.374820  6118 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.375255  6003 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.466281  1643 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 7.
I20260709 13:03:02.515205  1640 raft_consensus.cc:2393] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 11: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:02.422976  6007 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:02.419761  5647 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:02.408475  5438 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.532779  6053 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.533951  1561 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:02.517077  5838 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.536540  5838 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:02.538205  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 114862us
I20260709 13:03:02.538945  1640 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:02.539284  6058 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:02.539438  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:02.540185  1565 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:03:02.540237  2269 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:02.540511  1640 pending_rounds.cc:85] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 7
I20260709 13:03:02.540906  1640 pending_rounds.cc:107] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:03:02.495471  5839 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:02.528682  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:02.370682  6113 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:02.544040  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:02.544795  1565 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:02.509927  6017 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.540522  5978 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:02.541589  5894 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:02.545579  5978 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:02.545881  5894 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.546976  6017 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.547467  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:02.547988  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:02.548463  1713 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:02.548583  5894 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.548686  1568 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:03:02.532665  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:02.549865  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:02.546801  5978 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.551672  5885 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:02.551569  1644 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:02.522156  5886 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:02.541181  1640 raft_consensus.cc:2889] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:02.532684  5438 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.544845  1569 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:02.518023  5689 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.555075  5689 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.555614  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:02.556267  5438 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.556587  1569 raft_consensus.cc:2410] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:02.556946  1637 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:03:02.557296  5689 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:02.557513  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:02.532948  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:02.551540  5978 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.558434  1564 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.553663  5886 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:02.528770  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:02.446537  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:02.526650  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:02.543017  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.560717  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.564910  2298 raft_consensus.cc:2393] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:02.570173  1569 raft_consensus.cc:2410] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:03:02.530157  5647 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:02.496411  6116 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:02.571795  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 130560us
I20260709 13:03:02.451323  6117 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:02.622707  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 192433us
I20260709 13:03:02.625116  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:02.628506  1643 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:02.629287  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 215761us
I20260709 13:03:02.574996  6034 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:02.696946  1715 raft_consensus.cc:2410] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:02.697463  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:02.585673  5886 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:02.698525  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 110995us
I20260709 13:03:02.697199  1561 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:03:02.584322  5647 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:02.568271  6029 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.726635  1565 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:02.699076  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:02.635969  1560 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:02.760677  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:02.768613  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.583210  5438 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:02.574797  6007 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.584992  6110 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.786307  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:02.583374  5839 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.803184  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:02.849658  2298 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:02.633832  6034 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:02.819074  6110 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:02.819280  6122 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:03:02.819799  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.885067  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.819171  6027 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.240s
I20260709 13:03:02.819056  6064 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:02.820825  1562 raft_consensus.cc:2393] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:02.582317  5999 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.189s
W20260709 13:03:02.820977  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.821637  1563 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:02.822899  6007 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.862879  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:02.871142  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:03.000577  1715 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:02.871742  6130 log.cc:927] Time spent T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:03:02.873648  5915 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.570428  5875 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:02.886864  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:02.978950  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 206272us
I20260709 13:03:02.979748  6113 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:02.980549  6028 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.980644  5839 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.983052  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:02.984413  6112 consensus_meta.cc:287] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.856s	user 0.012s	sys 0.000s
I20260709 13:03:02.986003  6029 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:02.986474  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 193667us
I20260709 13:03:03.076273  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:03.077358  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2630) took 1002 ms. Trace:
I20260709 13:03:03.077805  1636 rpcz_store.cc:276] 0709 13:03:02.075917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.279255 (+203338us) service_pool.cc:224] Handling call
0709 13:03:02.291141 (+ 11886us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.332340 (+ 41199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.332385 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.362524 (+ 30139us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.412658 (+ 50134us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:02.412731 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.413635 (+   904us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.077132 (+663497us) raft_consensus.cc:1685] finished
0709 13:03:03.077153 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.077293 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1389,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":2216,"thread_start_us":49673,"threads_started":1,"wal-append.queue_time_us":159364}
I20260709 13:03:03.079406  1533 leader_election.cc:400] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11.
I20260709 13:03:03.079727  1533 leader_election.cc:403] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:02.582477  6003 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:03.081743  6003 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:02.988145  5885 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:03.082587  5885 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:03.085167  5885 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:03.085862  6110 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:03.087198  6110 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:03:03.087486  5978 consensus_meta.cc:287] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.000s
I20260709 13:03:03.091116  5915 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:03.092900  5915 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:02.991154  6064 raft_consensus.cc:697] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:02.991995  6117 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:03.094843  6064 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:02.998636  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.014705  1712 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:03.096403  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:03.097062  1561 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:03.015039  1714 raft_consensus.cc:1275] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:03:03.097298  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.026799  1568 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:03.044898  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:03.065455  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.095609  2300 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:03:03.097178  1682 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 2.432s (SENT) [suppressed 23 similar messages]
I20260709 13:03:03.097640  1714 pending_rounds.cc:85] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 3
I20260709 13:03:03.076267  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.100888  1714 pending_rounds.cc:107] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:03:03.101207  1714 raft_consensus.cc:2889] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:02.889339  6034 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:03.027719  6058 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:03.112124  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:03.110790  1564 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.552s	user 0.005s	sys 0.004s
I20260709 13:03:03.116624  1643 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:03.113164  1712 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 7.
I20260709 13:03:03.137912  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.139830  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:03.141024  2302 consensus_meta.cc:287] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.614s	user 0.011s	sys 0.001s
I20260709 13:03:03.141613  2302 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:03.111896  6009 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:03.144976  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 358785us
W20260709 13:03:03.122648  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 343369us
W20260709 13:03:03.147574  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.091166  6116 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:03.153190  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.152906  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.154412  1644 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:03.154320  1715 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:03.154594  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.152822  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.156625  1638 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:03.156765  1636 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:03:03.149665  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.162024  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.124836  5839 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.164987  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:03.165522  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.165997  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.166326  1643 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:03:03.167403  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.170297  1637 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:03:03.166203  1644 raft_consensus.cc:2376] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11. Re-sending same reply.
W20260709 13:03:03.174285  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.175933  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:03.173342  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.007s	sys 0.019s
W20260709 13:03:03.208868  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.007s	sys 0.019s
W20260709 13:03:03.209337  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 669172us
I20260709 13:03:03.108520  5438 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:03.215927  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.216615  1636 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:03.232671  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:03.231992  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 134829us
W20260709 13:03:03.233489  1640 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.016s	sys 0.006s
I20260709 13:03:03.217089  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:03.233807  1640 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.017s	sys 0.006s
I20260709 13:03:03.234375  1638 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
W20260709 13:03:03.235209  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.232309  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:03.237134  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:03.253664  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.256155  1715 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:03:03.257289  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.261255  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.262023  1564 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
W20260709 13:03:03.235903  1532 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.006s	sys 0.019s
W20260709 13:03:03.262895  1532 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.006s	sys 0.019s
W20260709 13:03:03.263319  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 640476us
I20260709 13:03:03.279850  1711 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:03.283571  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.283677  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.291851  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:03.299299  1644 raft_consensus.cc:2393] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 12: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:03.297672  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.293455  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.277281  5886 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:03.306874  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.307796  1715 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:03.159185  6009 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:03.324982  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.326201  1637 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 13: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:03.327145  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.328576  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1185) took 2659 ms. Trace:
I20260709 13:03:03.331089  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.332178  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.333155  2300 raft_consensus.cc:2410] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:03:03.338972  5937 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.340265  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2236) took 2445 ms. Trace:
I20260709 13:03:03.339941  5937 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:03.340687  1709 rpcz_store.cc:276] 0709 13:03:00.894692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.894812 (+   120us) service_pool.cc:224] Handling call
0709 13:03:02.514603 (+1619791us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b58001e4598. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:03:02.533463 (+ 18860us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.534169 (+   706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.534204 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.534707 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.630351 (+ 95644us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:02.630376 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.630423 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.339780 (+709357us) raft_consensus.cc:1685] finished
0709 13:03:03.339800 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.340069 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6957,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":19343,"spinlock_wait_cycles":13406080,"thread_start_us":71171,"threads_started":1,"wal-append.queue_time_us":802784}
I20260709 13:03:03.343686  6075 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:03.345793  5438 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:03.347743  5875 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:03.351425  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 44421us
W20260709 13:03:03.351819  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 189161us
I20260709 13:03:03.352357  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.353178  1635 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:03.353727  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:03.354180  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:03.355855  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.356700  1710 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
W20260709 13:03:03.357231  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2254) took 2353 ms (client timeout 1998 ms). Trace:
W20260709 13:03:03.357620  1710 rpcz_store.cc:269] 0709 13:03:01.003693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.004290 (+   597us) service_pool.cc:224] Handling call
0709 13:03:03.355525 (+2351235us) spinlock_profiling.cc:243] Waited 957 us on lock 0x7b58001e4598. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:03:03.357138 (+  1613us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201984}
I20260709 13:03:03.343322  6136 consensus_queue.cc:1059] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:03.360047  6136 raft_consensus.cc:3055] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:03.360450  6136 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:03.360556  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.361836  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.361536  6136 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:03.266108  6126 consensus_queue.cc:1059] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 13:03:03.363727  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1184) took 2799 ms. Trace:
I20260709 13:03:03.364295  6126 raft_consensus.cc:3055] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:03.364125  2301 rpcz_store.cc:276] 0709 13:03:00.564336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.564557 (+   221us) service_pool.cc:224] Handling call
0709 13:03:00.594472 (+ 29915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.821406 (+226934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.363236 (+2541830us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b5400544cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:03.363549 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497912064}
I20260709 13:03:03.364830  6126 raft_consensus.cc:740] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:03.365473  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.365839  6126 consensus_queue.cc:260] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:03.368050  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.368921  6126 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:03.369256  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.331321  2303 rpcz_store.cc:276] 0709 13:03:00.601425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.601546 (+   121us) service_pool.cc:224] Handling call
0709 13:03:00.601816 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.769359 (+167543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.769390 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:00.769431 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:00.769463 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.159567 (+2390104us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54006077d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:03.166871 (+  7304us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.261126 (+ 94255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384825088}
I20260709 13:03:03.370383  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.371259  2303 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:03:03.371764  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:03.372927  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.373481  1639 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:03.239146  5839 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:03.374260  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.375578  5839 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:03.376190  5839 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:03.371862  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 1011) took 1084 ms. Trace:
I20260709 13:03:03.379084  2303 rpcz_store.cc:276] 0709 13:03:02.289064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.370649 (+1081585us) service_pool.cc:224] Handling call
0709 13:03:03.370903 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.371630 (+   727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.371812 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:03.380376  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.380230  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.383033  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1190) took 1036 ms. Trace:
I20260709 13:03:03.352965  1533 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:03.366679  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.383486  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 1021) took 1001 ms. Trace:
I20260709 13:03:03.354754  1635 raft_consensus.cc:2376] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10. Re-sending same reply.
I20260709 13:03:03.384771  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.386724  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.392870  1710 raft_consensus.cc:2410] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:03:03.362584  1562 raft_consensus.cc:2393] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:03.389431  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.384248  1639 raft_consensus.cc:2410] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:03:03.389380  6059 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:03.394001  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.395931  1637 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:03.390070  2303 rpcz_store.cc:276] 0709 13:03:02.346272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.382119 (+1035847us) service_pool.cc:224] Handling call
0709 13:03:03.382449 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.382663 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.382690 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:03.382731 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:03.382784 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.382834 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.382964 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:03.389323  1711 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:03.391172  2304 rpcz_store.cc:276] 0709 13:03:02.381704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.370169 (+988465us) service_pool.cc:224] Handling call
0709 13:03:03.383425 (+ 13256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:03.398828  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.391283  1533 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:03.397153  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 1016) took 1016 ms. Trace:
I20260709 13:03:03.396381  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.399906  2300 rpcz_store.cc:276] 0709 13:03:02.380239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.303527 (+923288us) service_pool.cc:224] Handling call
0709 13:03:03.397084 (+ 93557us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:03.410173  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.410425  1715 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 10: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:03.411123  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:03.411690  1713 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:03.402863  1533 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:03.416988  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.363727  6136 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:03.415874  1533 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:03.422271  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.423134  1564 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:03.424491  1639 raft_consensus.cc:1240] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:03.420868  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:03.427878  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 74304.6us
I20260709 13:03:03.417718  1562 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:03.432814  1635 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:03.439633  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.414350  5915 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:03.460822  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:03.439527  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 273813us
I20260709 13:03:03.456640  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.448297  1568 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:03.454442  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 55349.9us
I20260709 13:03:03.465513  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.409713  6054 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:03.417649  6112 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:03.423558  6110 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11.
I20260709 13:03:03.481333  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.482221  2269 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:03.424384  5937 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:03.440671  5942 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.504462  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.504458  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:03.480568  5995 consensus_meta.cc:287] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.127s	user 0.013s	sys 0.000s
I20260709 13:03:03.499037  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.511066  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.432515  6058 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:03.461788  5915 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:03.482895  2303 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:03.514209  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:03.514521  2301 raft_consensus.cc:2410] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:03:03.479957  6112 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:03.521890  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.522773  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 1014) took 1143 ms. Trace:
I20260709 13:03:03.523267  2303 rpcz_store.cc:276] 0709 13:03:02.379642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.397124 (+1017482us) service_pool.cc:224] Handling call
0709 13:03:03.414201 (+ 17077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.518070 (+103869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.522712 (+  4642us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:03.526618  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.535255  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.516521  6058 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.534206  1564 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:03.534281  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.520911  6112 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:03.541894  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 934) took 1278 ms. Trace:
I20260709 13:03:03.542754  6139 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:03.543036  5944 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.543378  6139 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.543844  6139 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:03.543740  5944 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:03.486336  6003 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:03:03.519589  5915 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:03.505012  6137 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:03:03.538324  2304 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:03.548494  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.552847  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:03.553434  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 43521.7us
I20260709 13:03:03.553393  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:03.529217  1567 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.008s	sys 0.000s
W20260709 13:03:03.557961  1567 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.009s	sys 0.000s
I20260709 13:03:03.558256  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.559589  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.559983  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.559634  2301 raft_consensus.cc:2393] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:03.560424  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.560772  2304 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:03:03.561365  1710 raft_consensus.cc:2368] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11.
I20260709 13:03:03.561601  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.561918  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.560458  1635 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:03.562309  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.562852  2304 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:03:03.563498  2301 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:03.564451  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.564532  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:03.584384  2304 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:03:03.584484  2301 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:03.542366  1640 rpcz_store.cc:276] 0709 13:03:02.263795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.539814 (+276019us) service_pool.cc:224] Handling call
0709 13:03:02.540212 (+   398us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:02.567439 (+ 27227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.567480 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:02.570494 (+  3014us) log.cc:844] Serialized 52 byte log entry
0709 13:03:03.245501 (+675007us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:03.245601 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.245716 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.541653 (+295937us) raft_consensus.cc:1685] finished
0709 13:03:03.541674 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.541835 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13445,"raft.queue_time_us":16024,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":10230,"thread_start_us":537557,"threads_started":1,"wal-append.queue_time_us":938960}
W20260709 13:03:03.558369  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 129731us
I20260709 13:03:03.556979  2303 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:03:03.565557  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.565768  6137 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:03:03.566519  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 52401.1us
I20260709 13:03:03.589175  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.589867  5629 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:03.593765  6019 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.594470  6019 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:03.595373  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2132) took 1179 ms. Trace:
I20260709 13:03:03.595867  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:03.596689  2297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1170) took 3427 ms (client timeout 1558 ms). Trace:
I20260709 13:03:03.597379  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.598317  6019 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:03.528455  6072 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:03.602489  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:03.597118  2297 rpcz_store.cc:269] 0709 13:03:00.169152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.169602 (+   450us) service_pool.cc:224] Handling call
0709 13:03:03.596295 (+3426693us) spinlock_profiling.cc:243] Waited 2.19 s on lock 0x7b54001aa754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:03.596467 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5046305536}
I20260709 13:03:03.603224  6072 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:03.603868  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.605301  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:03.605688  6072 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:03.605806  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.582552  6009 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:03.607141  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.590555  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.577768  6123 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:03.609548  1566 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:03.615721  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.610230  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:03.555173  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 93157.4us
I20260709 13:03:03.584880  2304 pending_rounds.cc:85] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 10
I20260709 13:03:03.588485  6035 raft_consensus.cc:3055] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:03.588555  6137 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:03.703974  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.594993  5689 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.571674  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2267) took 1033 ms. Trace:
I20260709 13:03:03.539534  6058 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:03.519336  5995 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:03.606513  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:03.571998  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2268) took 1033 ms. Trace:
I20260709 13:03:03.598610  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.595818  1716 rpcz_store.cc:276] 0709 13:03:02.415699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.428200 (+ 12501us) service_pool.cc:224] Handling call
0709 13:03:02.449751 (+ 21551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.577296 (+127545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.577328 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.582007 (+  4679us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:02.582045 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.595070 (+1013025us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54001356d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:03.595144 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.595286 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253286272}
I20260709 13:03:03.499181  1562 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:03.619140  1640 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
W20260709 13:03:03.708190  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.713634  2304 pending_rounds.cc:107] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:03:03.531255  5942 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:03.728016  1710 raft_consensus.cc:2393] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:03:03.728339  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 108629us
I20260709 13:03:03.705626  1569 rpcz_store.cc:276] 0709 13:03:02.538619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.557270 (+ 18651us) service_pool.cc:224] Handling call
0709 13:03:03.571405 (+1014135us) spinlock_profiling.cc:243] Waited 8.25 ms on lock 0x7b5400200654. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:03.571587 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18966912}
I20260709 13:03:03.739780  2304 raft_consensus.cc:2889] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:03.732861  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:03.733721  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 322640us
I20260709 13:03:03.733969  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.759619  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:03.764037  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:03.757562  5689 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:03.765271  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.766610  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.615928  5886 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:03.769742  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:03.758635  1683 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:03.779904  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.699213  6035 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:03.771540  1568 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:03.768556  6138 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:03:03.735634  1567 rpcz_store.cc:276] 0709 13:03:02.538906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.578213 (+ 39307us) service_pool.cc:224] Handling call
0709 13:03:02.750259 (+172046us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:03.015623 (+265364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.015659 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:03.016355 (+   696us) log.cc:844] Serialized 52 byte log entry
0709 13:03:03.558444 (+542089us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:03.568274 (+  9830us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.570305 (+  2031us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.571489 (+  1184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.571589 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.571644 (+    55us) raft_consensus.cc:1685] finished
0709 13:03:03.571661 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.571930 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93363,"prepare.queue_time_us":9768,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":83,"raft.queue_time_us":217286,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":15061,"thread_start_us":541662,"threads_started":1,"wal-append.queue_time_us":523927}
I20260709 13:03:03.792953  1566 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 13:03:03.770200  6064 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:03:03.794759  5689 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:03.795027  6157 log.cc:927] Time spent T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:03:03.795773  1569 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:03.796986  1569 raft_consensus.cc:1217] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Deduplicated request from leader. Original: 7.3->[8.4-8.4]   Dedup: 8.4->[]
I20260709 13:03:03.797436  1569 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:03:03.764075  5942 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:03.798779  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:03.803380  1562 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:03.804179  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2265) took 1266 ms. Trace:
I20260709 13:03:03.801406  1638 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:03.804684  1565 rpcz_store.cc:276] 0709 13:03:02.537999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.575693 (+ 37694us) service_pool.cc:224] Handling call
0709 13:03:02.589517 (+ 13824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.763470 (+173953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.803720 (+1040250us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54001ff754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:03.803927 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255245056}
I20260709 13:03:03.807150  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1192) took 1308 ms. Trace:
I20260709 13:03:03.807722  2304 rpcz_store.cc:276] 0709 13:03:02.498748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.583450 (+1084702us) service_pool.cc:224] Handling call
0709 13:03:03.583807 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.803601 (+219794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.807070 (+  3469us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:03.811776  1564 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:03:03.811128  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2201) took 1607 ms. Trace:
I20260709 13:03:03.815135  1564 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:03:03.819147  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.773265  5438 consensus_queue.cc:1059] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:03:03.793313  5944 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:03.786682  5886 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:03.798801  6123 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.090351  1640 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:03:04.064221  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 476861us
W20260709 13:03:03.842736  1561 consensus_meta.cc:287] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.745s	user 0.009s	sys 0.005s
I20260709 13:03:03.803278  5604 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:03:04.089479  2298 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 1022) took 1705 ms (client timeout 1785 ms). Trace:
I20260709 13:03:03.872737  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:04.039414  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:04.054663  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 45212.1us
I20260709 13:03:03.877537  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:04.043497  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:04.104274  2298 rpcz_store.cc:269] 0709 13:03:02.381961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.579828 (+197867us) service_pool.cc:224] Handling call
0709 13:03:04.086845 (+1507017us) spinlock_profiling.cc:243] Waited 95.5 ms on lock 0x7b5400643f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:04.086983 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219677440}
W20260709 13:03:03.808866  5841 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.176s	user 0.010s	sys 0.013s
I20260709 13:03:04.094465  1561 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:04.108538  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2655) took 1123 ms. Trace:
I20260709 13:03:03.883961  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 770) took 1442 ms. Trace:
I20260709 13:03:03.889194  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2140) took 1316 ms. Trace:
W20260709 13:03:03.830960  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:03.835472  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.830904  1683 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:03.966486  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.069888  1713 consensus_meta.cc:287] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.614s	user 0.011s	sys 0.000s
I20260709 13:03:03.893292  1717 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:04.111413  1713 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:04.035595  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:04.111953  1563 consensus_meta.cc:287] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.000s
I20260709 13:03:04.106043  1641 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:03:04.036590  1609 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:04.055250  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 135624us
I20260709 13:03:04.109166  1561 rpcz_store.cc:276] 0709 13:03:02.985236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.096127 (+110891us) service_pool.cc:224] Handling call
0709 13:03:04.108461 (+1012334us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:04.114840  6047 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:04.116816  1566 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:04.121609  6112 consensus_meta.cc:287] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.000s
I20260709 13:03:04.121919  1637 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:03:04.121569  1568 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
W20260709 13:03:04.064522  2300 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.021s	sys 0.017s
W20260709 13:03:04.016074  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 117755us
I20260709 13:03:04.123191  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 868) took 1033 ms. Trace:
W20260709 13:03:04.123152  2300 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.024s	sys 0.018s
I20260709 13:03:04.112268  1643 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:04.124818  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2157) took 1048 ms. Trace:
I20260709 13:03:04.124006  1713 rpcz_store.cc:276] 0709 13:03:03.078866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.324191 (+245325us) service_pool.cc:224] Handling call
0709 13:03:04.112064 (+787873us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:03.822954  6034 consensus_meta.cc:287] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.000s
I20260709 13:03:04.109215  1560 rpcz_store.cc:276] 0709 13:03:02.441799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.444087 (+  2288us) service_pool.cc:224] Handling call
0709 13:03:03.883680 (+1439593us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54002986d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:03.883843 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315798400}
W20260709 13:03:04.131258  1641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2633) took 1890 ms (client timeout 1647 ms). Trace:
I20260709 13:03:03.829430  5942 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:03:04.079278  6059 consensus_meta.cc:287] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.664s	user 0.006s	sys 0.005s
W20260709 13:03:04.126139  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 295317us
I20260709 13:03:04.125260  1715 rpcz_store.cc:276] 0709 13:03:03.076041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.881437 (+805396us) service_pool.cc:224] Handling call
0709 13:03:04.124760 (+243323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:04.109607  1711 rpcz_store.cc:276] 0709 13:03:02.572609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.572751 (+   142us) service_pool.cc:224] Handling call
0709 13:03:03.888929 (+1316178us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54003d9fd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:03.889082 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1218937216}
I20260709 13:03:04.138723  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:04.128301  1567 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:04.097450  5629 consensus_meta.cc:287] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.002s
I20260709 13:03:04.129488  6145 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:04.129647  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:04.140312  1569 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:04.130365  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:04.141459  1564 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:04.130301  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:04.114156  1563 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:04.133517  1714 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
W20260709 13:03:04.114130  5841 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.486s	user 0.010s	sys 0.014s
I20260709 13:03:04.136004  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:04.144368  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:04.137166  5589 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:04.146354  5243 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.146505  5885 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:04.146405  5905 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:03.878252  5892 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:04.127962  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 91473.3us
I20260709 13:03:04.157105  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:04.139017  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:04.134626  5942 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:04.147982  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:04.143525  1636 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:04.160440  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 877) took 1079 ms. Trace:
I20260709 13:03:04.160883  1715 rpcz_store.cc:276] 0709 13:03:03.081394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.145027 (+1063633us) service_pool.cc:224] Handling call
0709 13:03:04.160368 (+ 15341us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:04.148392  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:04.131771  1641 rpcz_store.cc:269] 0709 13:03:02.223999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.224414 (+   415us) service_pool.cc:224] Handling call
0709 13:03:03.877325 (+1652911us) spinlock_profiling.cc:243] Waited 5.81 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:03:04.114370 (+237045us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13368704}
W20260709 13:03:04.134342  6139 consensus_meta.cc:287] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.590s	user 0.008s	sys 0.005s
W20260709 13:03:04.150417  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.152935  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.052543  5915 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:04.155241  5905 consensus_queue.cc:1243] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:03:04.147133  5243 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:04.157191  6136 consensus_meta.cc:287] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.724s	user 0.009s	sys 0.004s
I20260709 13:03:04.156603  5629 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:04.165614  5912 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:04.147253  5885 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.170701  5942 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:04.168468  6093 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.134511  5604 raft_consensus.cc:697] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:04.146777  5589 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:04.172392  1568 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:04.172727  2268 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:04.173249  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:04.186020  1681 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:04.122167  6112 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:04.155544  6151 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:03:04.191030  1715 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:04.193429  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 882) took 1110 ms. Trace:
I20260709 13:03:04.190629  1683 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:04.193778  1715 rpcz_store.cc:276] 0709 13:03:03.082842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.162095 (+1079253us) service_pool.cc:224] Handling call
0709 13:03:04.193370 (+ 31275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:03.901211  6137 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.177680  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:04.175628  1532 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:04.197377  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:04.184198  5629 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:02.993820   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:04.151427  6145 raft_consensus.cc:697] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:03:04.167678  6144 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:04.186097  1565 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:04.139964  6034 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.213542  5885 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:04.184682  6003 consensus_queue.cc:1059] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:03:04.190136  6059 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:03:04.217634  6112 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:04.219740  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:04.221203  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:04.174875  6139 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:03.919284  6035 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.173681  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:04.194764  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:04.226701  1715 raft_consensus.cc:2410] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
W20260709 13:03:04.175664  1635 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.026s	sys 0.029s
W20260709 13:03:04.227171  1635 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.026s	sys 0.029s
I20260709 13:03:04.201815  5905 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:04.177763  5995 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:04.183964  1711 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:04.205350  1532 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:04.193152  5589 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:04.210512  1561 raft_consensus.cc:1240] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:04.228197  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2220) took 1058 ms. Trace:
W20260709 13:03:04.240672  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 68018.8us
I20260709 13:03:04.242717  1636 rpcz_store.cc:276] 0709 13:03:03.172686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.258991 (+ 86305us) service_pool.cc:224] Handling call
0709 13:03:04.227978 (+968987us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b540032c3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:04.228122 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1968763264}
I20260709 13:03:04.182394  6149 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:03:04.230764  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:04.237855  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:04.238732  2302 raft_consensus.cc:2393] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 8: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:04.206966  5629 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:04.243750  2268 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:04.239789  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:04.325186  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 85440.4us
I20260709 13:03:04.326035  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:04.193346  6093 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:04.181625  5438 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:04.210005  5912 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:03:04.247283  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:04.335177  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 112654us
I20260709 13:03:04.244499  2269 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:04.249782  6010 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:04.249948  1718 raft_consensus.cc:2410] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:03:04.245368  6112 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:04.229657  5905 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.241671  5689 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:03:04.240568  6039 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:03:04.253211  5461 consensus_queue.cc:260] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.253996  6034 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:04.336859  6058 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.561s	user 0.010s	sys 0.004s
I20260709 13:03:04.256363  6035 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:04.256520  6162 log.cc:927] Time spent T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:03:04.184826  5892 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:04.257684  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:04.311102  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 135487us
I20260709 13:03:04.249022  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:04.361924  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 112979us
I20260709 13:03:04.363601  1532 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:04.337127  6064 raft_consensus.cc:3055] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:04.275662  6046 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:04.219683  5841 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:04.269315  5604 consensus_queue.cc:237] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.256870  5903 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:04.226219  6151 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:04.384624  6151 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:04.384754  5903 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:04.385113  6151 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:04.257861  6145 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.327318  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2658) took 1341 ms. Trace:
I20260709 13:03:04.328823  2296 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:04.331398  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 87706.2us
I20260709 13:03:04.333043  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2224) took 1035 ms. Trace:
I20260709 13:03:04.333804  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 1017) took 1951 ms. Trace:
I20260709 13:03:04.445187  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:04.336357  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 91913.3us
I20260709 13:03:04.341811  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:04.446250  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2654) took 1461 ms. Trace:
I20260709 13:03:04.444633  1644 rpcz_store.cc:276] 0709 13:03:03.297174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.428220 (+131046us) service_pool.cc:224] Handling call
0709 13:03:03.464715 (+ 36495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:03.501627 (+ 36912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:03.501676 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:03.513849 (+ 12173us) log.cc:844] Serialized 26 byte log entry
0709 13:03:03.919427 (+405578us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:03.919488 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.919603 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:04.332816 (+413213us) raft_consensus.cc:1685] finished
0709 13:03:04.332841 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.332979 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":171,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":20642,"thread_start_us":321600,"threads_started":1,"wal-append.queue_time_us":615173}
I20260709 13:03:04.343497  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:04.340354  5905 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:04.255220  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:04.371773  6112 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:04.445900  2300 rpcz_store.cc:276] 0709 13:03:02.380521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.408411 (+1027890us) service_pool.cc:224] Handling call
0709 13:03:03.427253 (+ 18842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.464482 (+ 37229us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:04.124802 (+660320us) log.cc:844] Serialized 14 byte log entry
0709 13:03:04.331467 (+206665us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:04.331519 (+    52us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:04.331563 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.331624 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.331731 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26588,"prepare.queue_time_us":590439,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":1165,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":2941,"thread_start_us":781848,"threads_started":2}
I20260709 13:03:04.443805  1563 rpcz_store.cc:276] 0709 13:03:02.986094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.153487 (+167393us) service_pool.cc:224] Handling call
0709 13:03:04.268494 (+1115007us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b54001ce154. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:04.327255 (+ 58761us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810835712}
I20260709 13:03:04.374758  6116 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:04.344659  5869 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:03.814896  1642 rpcz_store.cc:276] 0709 13:03:02.203927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.531758 (+327831us) service_pool.cc:224] Handling call
0709 13:03:02.569691 (+ 37933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.576046 (+  6355us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:02.631470 (+ 55424us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.349525 (+718055us) spinlock_profiling.cc:243] Waited 59.5 ms on lock 0x7b1800461740. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a 00007fd7a1caba2a
0709 13:03:03.809848 (+460323us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:03.810835 (+   987us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:03.810880 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.810965 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.811070 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45744,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":725,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4554,"spinlock_wait_cycles":136860928,"thread_start_us":459441,"threads_started":1}
I20260709 13:03:04.323467  1641 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:04.376802  5886 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:04.337838  6010 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.334625  5912 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:04.341528  6149 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.373145  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:04.340932  5978 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:04.367941  1681 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:04.348918  6007 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:03:04.383427  2302 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 1018) took 2002 ms (client timeout 2040 ms). Trace:
I20260709 13:03:04.536417  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:04.536701  2302 rpcz_store.cc:269] 0709 13:03:02.380795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.149341 (+768546us) service_pool.cc:224] Handling call
0709 13:03:04.383218 (+1233877us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400168354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:04.383365 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2479527168}
I20260709 13:03:04.537336  1635 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:04.383994  5461 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:04.200580  5897 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:03:04.385936  2299 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: []
W20260709 13:03:04.386502  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.430372  1709 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:03:04.351063  6123 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:04.354264  5894 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:04.448798  1710 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:04.369500  6064 raft_consensus.cc:740] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:03:04.331733  5647 consensus_meta.cc:287] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.328s	user 0.013s	sys 0.004s
I20260709 13:03:04.513424  6010 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:04.532016  5912 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:04.446877  1567 rpcz_store.cc:276] 0709 13:03:02.984932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.789582 (+804650us) service_pool.cc:224] Handling call
0709 13:03:04.123121 (+333539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.139552 (+ 16431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.445985 (+306433us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b54000a78d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:04.446169 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":738030848}
I20260709 13:03:04.529320  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 867) took 1450 ms. Trace:
W20260709 13:03:04.532992  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 159955us
I20260709 13:03:04.367842  6058 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:04.535755  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 167972us
I20260709 13:03:04.454869  5915 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:04.550055  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 214209us
I20260709 13:03:04.458587  5869 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.512439  6003 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:04.549018  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2145) took 1561 ms. Trace:
I20260709 13:03:04.544884  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2666) took 1215 ms. Trace:
W20260709 13:03:04.331578  6126 consensus_meta.cc:287] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.962s	user 0.018s	sys 0.000s
I20260709 13:03:04.330848  5438 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:04.552024  5894 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:04.557485  1714 rpcz_store.cc:276] 0709 13:03:03.078594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.872344 (+793750us) service_pool.cc:224] Handling call
0709 13:03:04.460136 (+587792us) spinlock_profiling.cc:243] Waited 27.7 ms on lock 0x7b54001473d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:04.529201 (+ 69065us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63696640}
I20260709 13:03:04.554183  6007 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:04.560437  6064 consensus_queue.cc:260] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.561658  5907 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:04.562361  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2264) took 1719 ms. Trace:
I20260709 13:03:04.556936  5647 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:04.567227  6077 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:04.570832  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1235) took 1051 ms. Trace:
W20260709 13:03:04.571712  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2253) took 3572 ms (client timeout 4632 ms). Trace:
I20260709 13:03:04.573408  6093 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:04.577934  6046 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:04.578262  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1195) took 1452 ms. Trace:
I20260709 13:03:04.553198  6112 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:03:04.535167  5978 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.588282  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:04.586645  1681 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:04.539242  6072 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:04.590338  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.592057  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.635763  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.545606  5602 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:04.594735  1709 rpcz_store.cc:276] 0709 13:03:02.988024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.333262 (+1345238us) service_pool.cc:224] Handling call
0709 13:03:04.359365 (+ 26103us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.547401 (+188036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.548950 (+  1549us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:04.595273  1565 rpcz_store.cc:276] 0709 13:03:03.329685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.814611 (+484926us) service_pool.cc:224] Handling call
0709 13:03:03.814930 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.210313 (+395383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.544523 (+334210us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b5400200654. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:04.544786 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":853228416}
W20260709 13:03:04.598361  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 272371us
W20260709 13:03:04.598837  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 153717us
W20260709 13:03:04.593963  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 401708us
I20260709 13:03:04.621943  1566 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:04.540973  6149 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:04.584064  6139 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:04.605494  5894 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:04.622776  1717 rpcz_store.cc:276] 0709 13:03:02.843718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.862397 (+ 18679us) service_pool.cc:224] Handling call
0709 13:03:03.893080 (+1030683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.111785 (+218705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.562132 (+450347us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54003d9fd0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:04.562286 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2369693568}
I20260709 13:03:04.594357  5869 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:04.629537  2305 rpcz_store.cc:276] 0709 13:03:03.125954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.585229 (+459275us) service_pool.cc:224] Handling call
0709 13:03:04.087053 (+501824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.087335 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.087375 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:04.087430 (+    55us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:04.087466 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.087536 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.578083 (+490547us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b5400643f50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:04.578199 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1153222016}
I20260709 13:03:04.580276  5892 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:04.626873  6077 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:04.627935  1718 rpcz_store.cc:269] 0709 13:03:00.998880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.003342 (+  4462us) service_pool.cc:224] Handling call
0709 13:03:04.571461 (+3568119us) spinlock_profiling.cc:243] Waited 3.18 s on lock 0x7b54003390d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:04.571642 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7324613888}
I20260709 13:03:04.627040  2299 rpcz_store.cc:276] 0709 13:03:03.519598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.587312 (+ 67714us) service_pool.cc:224] Handling call
0709 13:03:04.385650 (+798338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.551512 (+165862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.570556 (+ 19044us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b5400168350. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:04.570757 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1835193728}
I20260709 13:03:04.602253  5438 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:04.594146  6003 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:04.683166  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:04.640041  5602 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:03:04.688371  6163 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.354s	user 0.000s	sys 0.001s
W20260709 13:03:04.644658  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.620195  6116 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:04.654358  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 302253us
I20260709 13:03:04.657506  1569 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:04.651284  1609 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:04.649772  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 952) took 1055 ms. Trace:
I20260709 13:03:04.699631  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:04.662086  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 75556.5us
I20260709 13:03:04.689304  5602 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:04.702520  5602 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:04.630527  1714 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:03:04.703087  1714 pending_rounds.cc:85] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 7
I20260709 13:03:04.703367  1714 pending_rounds.cc:107] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:03:04.665242  2268 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:04.664670  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 76481.4us
I20260709 13:03:04.707146  6064 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:04.708082  1532 leader_election.cc:400] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11.
I20260709 13:03:04.709023  1532 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:04.709568  5647 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:04.709807  5689 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:04.710961  1717 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:04.711671  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 865) took 1633 ms. Trace:
I20260709 13:03:04.711539  5589 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.712142  1717 rpcz_store.cc:276] 0709 13:03:03.078038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.661082 (+1583044us) service_pool.cc:224] Handling call
0709 13:03:04.681890 (+ 20808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.711385 (+ 29495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.711586 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:04.714452  5589 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:04.716051  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:04.719038  2302 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:03:04.638165  5978 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:04.721207  5905 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.722553  1713 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:04.670365  6149 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:04.659538  6047 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.544s	user 0.008s	sys 0.004s
I20260709 13:03:04.724767  6124 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.725299  6124 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:04.682025  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 90122.3us
I20260709 13:03:04.682288  2300 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:03:04.688031  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.687819  6126 consensus_queue.cc:1059] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:04.664646  6118 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:04.703624  1714 raft_consensus.cc:2889] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:04.707544  6077 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.678977  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:04.706650  2268 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:04.724779  1681 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:04.726653  1712 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:04.700058  1640 rpcz_store.cc:276] 0709 13:03:03.594474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.127817 (+533343us) service_pool.cc:224] Handling call
0709 13:03:04.144995 (+ 17178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.211091 (+ 66096us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:04.246202 (+ 35111us) log.cc:844] Serialized 14 byte log entry
0709 13:03:04.536173 (+289971us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:04.536227 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:04.536271 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.649502 (+113231us) spinlock_profiling.cc:243] Waited 58.1 ms on lock 0x7b54003188d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:04.649562 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.649697 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9721,"prepare.queue_time_us":39047,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":2349,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":14147,"spinlock_wait_cycles":202084992,"thread_start_us":197319,"threads_started":1,"wal-append.queue_time_us":285235}
I20260709 13:03:04.750993  1683 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:04.730011  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:04.792505  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 67828.2us
I20260709 13:03:04.792795  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:04.698129  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 866) took 1619 ms. Trace:
I20260709 13:03:04.727388  1709 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:04.726557  5905 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:04.804884  5995 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:04.805518  5995 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:04.806013  5995 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:04.809355  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 85366.8us
I20260709 13:03:04.809124  5978 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.809909  6123 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:04.810396  6123 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:04.810859  6123 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:04.782881  6077 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:04.783488  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 863) took 1706 ms. Trace:
I20260709 13:03:04.815112  5647 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:04.815676  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 136947us
I20260709 13:03:04.817132  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:04.685631  6003 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:04.671340  6028 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:04.762424  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.802350  6019 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:04.782119  5894 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:04.804278  6165 log.cc:927] Time spent T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.460s	user 0.001s	sys 0.000s
I20260709 13:03:04.794323  6047 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:04.805038  6046 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.815102  1714 rpcz_store.cc:276] 0709 13:03:03.077384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.605758 (+1528374us) service_pool.cc:224] Handling call
0709 13:03:04.630238 (+ 24480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.783172 (+152934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.783415 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:04.785432  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 134238us
I20260709 13:03:04.865556  1609 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:04.797679  1635 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:03:04.739880  6139 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:04.798103  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 91518.4us
I20260709 13:03:04.793937  1718 rpcz_store.cc:276] 0709 13:03:03.078310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.672503 (+1594193us) service_pool.cc:224] Handling call
0709 13:03:04.677343 (+  4840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.685068 (+  7725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.685100 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:04.697795 (+ 12695us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:04.697853 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.697921 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.698059 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:04.868036  2268 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:04.832520  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 123586us
I20260709 13:03:04.760834  1710 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:04.792994  2302 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:03:04.869629  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 876) took 1788 ms. Trace:
I20260709 13:03:04.870035  1710 rpcz_store.cc:276] 0709 13:03:03.081125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.256162 (+1175037us) service_pool.cc:224] Handling call
0709 13:03:04.869563 (+613401us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:04.749977  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:04.780483  6167 log.cc:927] Time spent T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:03:04.739775  5243 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:04.836957  1716 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
W20260709 13:03:04.736393  5942 consensus_meta.cc:287] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.002s
I20260709 13:03:04.881019  5942 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.882066  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 864) took 1804 ms. Trace:
I20260709 13:03:04.882493  1709 rpcz_store.cc:276] 0709 13:03:03.077696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.640652 (+1562956us) service_pool.cc:224] Handling call
0709 13:03:04.665508 (+ 24856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.818701 (+153193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.881823 (+ 63122us) spinlock_profiling.cc:243] Waited 61.7 ms on lock 0x7b54003395d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:04.882003 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141795072}
I20260709 13:03:04.820837  1713 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:03:04.884250  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 870) took 1804 ms. Trace:
I20260709 13:03:04.884346  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 873) took 1804 ms. Trace:
I20260709 13:03:04.831570  6059 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:04.884917  1714 rpcz_store.cc:276] 0709 13:03:03.080297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.883386 (+1803089us) service_pool.cc:224] Handling call
0709 13:03:04.883738 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.883957 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.883981 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:04.884031 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:04.884064 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.884116 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.884253 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:04.886107  5942 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:04.735894  6058 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:04.784984  6116 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.884691  1713 rpcz_store.cc:276] 0709 13:03:03.079440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.816947 (+1737507us) service_pool.cc:224] Handling call
0709 13:03:04.817288 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.883965 (+ 66677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.884188 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:04.889596  6168 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.354s	user 0.004s	sys 0.004s
I20260709 13:03:04.892691  6046 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:04.896974  1712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 872) took 1817 ms (client timeout 1785 ms). Trace:
W20260709 13:03:04.897454  1712 rpcz_store.cc:269] 0709 13:03:03.080018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.326716 (+246698us) service_pool.cc:224] Handling call
0709 13:03:04.896759 (+1570043us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400337cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:04.896902 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3065704192}
I20260709 13:03:04.831354  5589 raft_consensus.cc:3055] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:04.864095  6047 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.898947  5589 raft_consensus.cc:740] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:04.899382  6047 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:04.899940  5589 consensus_queue.cc:260] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.898974  1715 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:04.903532  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2146) took 1915 ms. Trace:
I20260709 13:03:04.903954  1715 rpcz_store.cc:276] 0709 13:03:02.988331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.332000 (+1343669us) service_pool.cc:224] Handling call
0709 13:03:04.898578 (+566578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.902367 (+  3789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.903261 (+   894us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b5400337cd0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:04.903460 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1302423424}
I20260709 13:03:04.909322  5978 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:04.910351  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 117644us
I20260709 13:03:04.912307  6124 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:04.913316  6035 consensus_meta.cc:287] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.000s
I20260709 13:03:04.914464  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 949) took 1320 ms. Trace:
I20260709 13:03:04.914897  1639 rpcz_store.cc:276] 0709 13:03:03.593649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.095503 (+501854us) service_pool.cc:224] Handling call
0709 13:03:04.122751 (+ 27248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:04.129687 (+  6936us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.129720 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:04.132069 (+  2349us) log.cc:844] Serialized 26 byte log entry
0709 13:03:04.334423 (+202354us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:04.334479 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.334572 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:04.888942 (+554370us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:04.914149 (+ 25207us) raft_consensus.cc:1685] finished
0709 13:03:04.914178 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.914402 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5029,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":13728,"spinlock_wait_cycles":70446848,"thread_start_us":156885,"threads_started":1,"wal-append.queue_time_us":282921}
W20260709 13:03:04.916973  5629 consensus_meta.cc:287] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.580s	user 0.002s	sys 0.007s
I20260709 13:03:04.917528  5629 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.820771  5885 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:04.919261  1682 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:04.731766  6007 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:04.921988  6010 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:04.923332  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 57883.4us
I20260709 13:03:04.924706  5912 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.925524  5886 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:04.926009  1560 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 8: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:04.928284  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:04.854810  6028 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.928767  5894 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:04.929260  6028 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:04.929977  6169 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:04.930439  1607 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:04.930877  6169 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:04.931507  6093 raft_consensus.cc:3055] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:04.931923  6093 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:04.932833  6093 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.934152  6093 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:04.926795  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:04.837466  6176 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.854705  6019 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:04.870338  6173 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:04.878801  5869 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.873126  5604 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:04.885326  1465 catalog_manager.cc:5697] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 5 to 8, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:03:04.936436  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:04.876735  1532 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:04.994308  2269 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:04.901571  5589 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:04.944882  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:04.942364  2299 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:04.945793  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:04.947420  6151 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:04.947746  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:04.946692  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:04.948632  1714 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:04.836174  6039 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:04.994599  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 55355.6us
W20260709 13:03:04.954394  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 874) took 1873 ms (client timeout 2165 ms). Trace:
I20260709 13:03:05.000720  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:04.994659  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 54273.7us
W20260709 13:03:04.998466  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 121819us
W20260709 13:03:05.000670  1716 rpcz_store.cc:269] 0709 13:03:03.080570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.763767 (+683197us) service_pool.cc:224] Handling call
0709 13:03:04.953709 (+1189942us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b54003386d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:04.954077 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1927469312}
I20260709 13:03:05.003142  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:05.003793  1643 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:04.984556  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.006991  1639 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:05.008474  2304 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:05.008644  1683 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:04.968125  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 59423.8us
I20260709 13:03:05.010454  1607 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:04.999836  2297 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:05.013404  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:05.013795  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2166) took 1130 ms. Trace:
I20260709 13:03:05.014163  1714 rpcz_store.cc:276] 0709 13:03:03.883567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.886519 (+1002952us) service_pool.cc:224] Handling call
0709 13:03:04.886897 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.013558 (+126661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.013745 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:05.013968  1637 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:05.016505  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:05.016582  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 159777us
I20260709 13:03:04.960418  6019 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.014355  1465 catalog_manager.cc:5697] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:03:05.025228  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:05.026142  2299 raft_consensus.cc:2368] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:05.025184  2302 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:04.830305  5915 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.132s
I20260709 13:03:05.027081  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:05.039275  1641 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.021327  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:04.968518  5886 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:04.994659  6064 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:05.008042  6059 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:03:05.029198  5892 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:05.018028  6048 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:05.018230  5897 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:05.018291  6039 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.009191  5602 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:03:05.031250  5629 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:05.042511  6116 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:05.060787  6111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.021s	sys 0.024s
I20260709 13:03:05.033756  5885 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:05.034504  1643 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:03:05.036023  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:05.043047  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:05.043689  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:05.090624  2297 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:04.967224  6072 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:05.013690  6034 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:03:05.009022  6171 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:05.055895  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 136732us
I20260709 13:03:05.090888  1465 catalog_manager.cc:5697] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 6 to 9, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:05.124224  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:05.013942  5647 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:05.058655  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 45948.2us
I20260709 13:03:05.027980  6169 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:05.070093  1637 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:04.998320  5689 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
W20260709 13:03:05.078969  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 73122.5us
I20260709 13:03:05.082163  1609 leader_election.cc:400] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:05.031988  1683 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:05.083506  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 67092.7us
I20260709 13:03:05.129684  1609 leader_election.cc:403] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:03:05.130043  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.044175  5886 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:05.090683  1640 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:05.129719  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:05.032327  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:05.130738  1715 raft_consensus.cc:2376] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14. Re-sending same reply.
I20260709 13:03:05.131206  1644 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:05.132515  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:05.133570  1560 raft_consensus.cc:2393] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:05.091573  6111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.022s	sys 0.024s
I20260709 13:03:05.143846  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:04.996528  6176 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.144675  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:05.156236  1716 raft_consensus.cc:2376] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10. Re-sending same reply.
W20260709 13:03:05.144775  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 77011.4us
I20260709 13:03:05.156715  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 862) took 2079 ms. Trace:
I20260709 13:03:05.157249  1711 rpcz_store.cc:276] 0709 13:03:03.077059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.441548 (+1364489us) service_pool.cc:224] Handling call
0709 13:03:04.534763 (+ 93215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.605460 (+ 70697us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:04.959376 (+353916us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.140575 (+181199us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.156336 (+ 15761us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:05.156412 (+    76us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:05.156461 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.156530 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.156657 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15541,"prepare.queue_time_us":58327,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":22698,"thread_start_us":396501,"threads_started":2}
W20260709 13:03:05.161957  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 130044us
W20260709 13:03:05.162896  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.155591  1569 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:05.092998  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:05.164747  1714 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:03:05.152709  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.165773  1560 raft_consensus.cc:2393] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:05.169350  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.163971  1683 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:05.172529  1641 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:05.173565  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:05.137265  1607 connection.cc:256] RPC call timeout handler was delayed by -1.2628389430000198s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
I20260709 13:03:05.089691  5885 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:05.185163  1681 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:05.184559  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:05.079384  6048 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:05.084071  6058 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 13:03:05.193694  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:05.194881  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:05.211386  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:05.198226  1711 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:05.213616  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:05.083885  5604 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 13:03:05.082973  5897 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:05.223747  2300 raft_consensus.cc:2410] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:03:05.174556  5629 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:05.198658  1465 catalog_manager.cc:5697] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 6 to 9, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 9 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:05.140974  5886 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:05.204563  5907 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:05.086900  5940 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 13:03:05.199838  5905 consensus_queue.cc:1059] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:03:05.205575  5912 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:05.215291  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.219332  1712 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 10: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:05.211539  6149 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:05.224624  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:05.173488  5942 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:05.232939  1561 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:05.233928  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:05.225966  1569 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:05.144254  6072 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.247007  6046 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:05.248188  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:05.186642  6173 consensus_queue.cc:1059] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:03:05.248070  6028 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.248612  6008 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:05.249552  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 125464us
W20260709 13:03:05.220081  5438 consensus_meta.cc:287] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.002s
W20260709 13:03:05.250159  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 52914.7us
I20260709 13:03:05.222882  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:05.252720  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:05.168558  6064 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:03:05.255371  1711 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
W20260709 13:03:05.256101  1466 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45150 (request call id 325) took 34347 ms (client timeout 15000 ms). Trace:
I20260709 13:03:05.251678  1565 raft_consensus.cc:2376] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11. Re-sending same reply.
I20260709 13:03:05.263943  1640 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:05.242189  6145 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.234460  5629 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:05.265318  6151 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:05.269850  6072 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:05.270355  2298 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:05.270888  6048 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.280630  6028 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:05.281251  1709 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:05.261209  5907 raft_consensus.cc:697] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:05.284273  5869 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:05.285851  6188 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.287397  5897 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:05.263480  6010 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:05.141192  6171 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.349722  6171 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:05.349090  5907 consensus_queue.cc:237] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:05.290333  6183 log.cc:927] Time spent T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:03:05.260244  5995 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.355188  5995 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:05.208081  6152 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:03:05.356559  6152 raft_consensus.cc:3055] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:05.356922  6152 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:05.357800  6152 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:05.293282  5885 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.294898  1639 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:05.299302  6173 raft_consensus.cc:3055] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:03:05.299372  6185 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
W20260709 13:03:05.300223  6193 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:03:05.300143  6003 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:05.300807  6189 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:03:05.302279  6176 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:05.267547  1465 catalog_manager.cc:5697] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 4 to 5, term changed from 3 to 10, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to <none>, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 10 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:03:05.308135  6179 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:03:05.309741  2297 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
W20260709 13:03:05.312971  6184 log.cc:927] Time spent T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.235s	user 0.000s	sys 0.002s
I20260709 13:03:05.313488  5886 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:05.383026  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.383463  5886 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:05.314262  6187 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:03:05.259536  5942 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.385514  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 869) took 2306 ms. Trace:
I20260709 13:03:05.327047  5438 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:05.385845  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 65303.1us
I20260709 13:03:05.386737  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:05.385552  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:05.388160  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:05.388370  2297 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.386169  1717 rpcz_store.cc:276] 0709 13:03:03.079163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.713317 (+1634154us) service_pool.cc:224] Handling call
0709 13:03:04.713733 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:04.714167 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.714202 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:04.818788 (+104586us) log.cc:844] Serialized 26 byte log entry
0709 13:03:05.069275 (+250487us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:05.079585 (+ 10310us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.083149 (+  3564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.083246 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.313507 (+230261us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:05.385205 (+ 71698us) raft_consensus.cc:1685] finished
0709 13:03:05.385321 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.385455 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28567,"prepare.queue_time_us":8293,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":38,"raft.queue_time_us":8981,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":7564,"spinlock_wait_cycles":48363392,"thread_start_us":195734,"threads_started":1,"wal-append.queue_time_us":312276}
I20260709 13:03:05.339078  6145 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:05.264144  6136 raft_consensus.cc:3055] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:05.391219  6136 raft_consensus.cc:740] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:05.391494  6010 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:05.392220  6136 consensus_queue.cc:260] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:05.393464  6136 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:05.362125  6188 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.393640  6003 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:05.332724  5629 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.363667  6173 raft_consensus.cc:740] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:05.342897  1714 raft_consensus.cc:1240] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:05.395202  1710 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:05.385915  5942 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:05.395725  6173 consensus_queue.cc:260] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:05.397006  6173 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:05.387615  5438 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:05.364542  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:05.370349  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.295064  6047 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:05.400640  6047 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:05.361972  6196 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:05.402047  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:05.404299  6064 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:05.404942  6064 raft_consensus.cc:697] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:05.411569  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:05.412197  2296 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:05.314548  1532 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:05.385418  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:05.390761  6145 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:05.414238  2297 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:05.406008  6064 consensus_queue.cc:237] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.360387  5885 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:05.328579  6008 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:05.424371  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.455494  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.456598  1560 raft_consensus.cc:2393] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:05.456743  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.457600  1565 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:05.455503  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:05.458935  2297 raft_consensus.cc:2393] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 12: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:05.425716  6019 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.426405  6151 raft_consensus.cc:3055] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:05.427101  6039 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:05.427742  6188 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:05.394681  6003 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:05.431327  6123 consensus_meta.cc:287] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.620s	user 0.011s	sys 0.000s
I20260709 13:03:05.399190  1465 catalog_manager.cc:5697] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:05.443696  1682 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:05.450208  2305 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:05.367937  6152 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.451705  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:05.424479  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 886) took 2065 ms. Trace:
I20260709 13:03:05.493134  1709 rpcz_store.cc:276] 0709 13:03:03.359352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.883389 (+1524037us) service_pool.cc:224] Handling call
0709 13:03:04.883780 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:05.338518 (+454738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.424259 (+ 85741us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b540020ba54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:05.424424 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":913721600}
I20260709 13:03:05.452132  5915 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.463640  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.497609  1561 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:05.465072  2298 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:03:05.478094  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.498333  2269 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:05.477924  2296 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:05.485073  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:05.462376  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.494561  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:05.495787  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:05.501792  2297 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:03:05.503685  1636 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:05.503937  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 421979us
I20260709 13:03:05.504663  1609 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:05.504947  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:05.506721  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 55094.8us
I20260709 13:03:05.508973  2299 raft_consensus.cc:2410] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:03:05.514602  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 200158us
I20260709 13:03:05.515407  1532 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:05.515233  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:05.479717  6118 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:05.516218  1564 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:03:05.518072  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.520108  1563 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:05.511241  1465 catalog_manager.cc:5697] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 7 to 10, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 10 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:05.518126  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.476969  6151 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:05.518204  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:05.490399  6149 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:03:05.537465  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:05.538779  1640 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:05.538789  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:05.541652  1639 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:05.542263  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:05.542732  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.542953  1636 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:05.537786  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:05.497118  6145 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:05.546320  1465 catalog_manager.cc:5697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 5 to 10. New cstate: current_term: 10 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:05.556202  6145 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:05.557338  1465 catalog_manager.cc:5697] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 4 to 9, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 9 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:05.559424  6145 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:05.516587  6007 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:05.519189  6137 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:05.565075  2302 raft_consensus.cc:1240] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:05.522327  5995 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.606523  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.522866  6047 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:05.571331  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:05.573274  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 59393.3us
I20260709 13:03:05.533891  5629 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.564065  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:05.576252  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:05.612061  1639 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:05.529487  6171 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:05.581152  2299 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:05.579216  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:05.524487  6034 raft_consensus.cc:697] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:05.582512  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 67162.2us
I20260709 13:03:05.588201  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:05.539930  6188 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:05.501143  6123 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.541133  6118 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:05.588191  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.591872  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:05.592499  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.631132  1560 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:05.594664  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 180937us
I20260709 13:03:05.516937  5915 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.533605  1533 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:05.599104  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:05.636077  2301 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:03:05.636253  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 102753us
I20260709 13:03:05.601657  1563 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:03:05.603277  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:05.595296  5892 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.002s
I20260709 13:03:05.611951  1465 catalog_manager.cc:5697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 6 to 7, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
W20260709 13:03:05.612694  1607 leader_election.cc:336] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 2.088s (SENT)
W20260709 13:03:05.644542  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 56419.2us
I20260709 13:03:05.644708  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:05.613025  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.611361  2300 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:05.624084  1532 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:05.625753  2269 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:05.632251  1682 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:05.633270  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.652127  1562 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:05.637063  1533 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:05.641646  2302 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:05.656303  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:05.658088  1711 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:05.620275  6137 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:05.660054  1532 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:05.620044  6047 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:05.664141  1465 catalog_manager.cc:5697] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:05.666584  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:05.668892  2299 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:05.663230  1533 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:05.669440  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:05.630055  6007 raft_consensus.cc:697] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:05.671994  1532 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:05.636714  6188 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:05.672901  1465 catalog_manager.cc:5697] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 4 to 9, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:05.614679  5995 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.677870  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:05.678499  2300 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:03:05.644939  5589 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:05.672506  6046 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:03:05.679984  5589 raft_consensus.cc:697] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:03:05.680181  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 48075.6us
I20260709 13:03:05.670540  5940 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:05.680696  6019 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:05.675122  6171 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:05.680919  6126 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:05.681505  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:05.681871  1533 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:05.682075  5944 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:05.682744  5944 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:05.683069  6123 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:05.683421  5647 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:05.683874  5944 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:05.684104  1465 catalog_manager.cc:5697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:05.686160  6010 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:05.685916  1465 catalog_manager.cc:5697] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 2 to 7, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 7 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:05.681505  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:05.644698  6176 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:05.656695  5892 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:05.687747  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:03:05.689929  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.689702  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:05.694530  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:05.695422  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.670686  6077 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:05.328820  1466 rpcz_store.cc:269] 0709 13:02:30.891349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.891810 (+   461us) service_pool.cc:224] Handling call
0709 13:03:05.154312 (+34262502us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:56.986054 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:56.986722 (+   668us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:56.986773 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:02:56.986806 (+    33us) tablet.cc:662] Decoding operations
0709 13:02:56.988621 (+  1815us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:56.988716 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:56.988756 (+    40us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:02:56.990617 (+  1861us) tablet.cc:701] Row locks acquired
0709 13:02:56.990693 (+    76us) write_op.cc:260] PREPARE: finished
0709 13:02:56.991208 (+   515us) write_op.cc:270] Start()
0709 13:02:56.991398 (+   190us) write_op.cc:276] Timestamp: P: 1783602176991071 usec, L: 0
0709 13:02:56.991413 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 13:02:56.992393 (+   980us) log.cc:844] Serialized 8732 byte log entry
0709 13:02:57.002724 (+ 10331us) op_driver.cc:464] REPLICATION: finished
0709 13:02:57.025475 (+ 22751us) write_op.cc:301] APPLY: starting
0709 13:02:57.025579 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:57.098513 (+ 72934us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:57.098656 (+   143us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:57.157065 (+ 58409us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:57.161620 (+  4555us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 13:02:57.161678 (+    58us) write_op.cc:312] APPLY: finished
0709 13:02:57.162309 (+   631us) log.cc:844] Serialized 196 byte log entry
0709 13:02:57.162527 (+   218us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:57.162726 (+   199us) write_op.cc:454] Released schema lock
0709 13:02:57.162808 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:57.166241 (+  3433us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:57.166270 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3214791,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":22382,"apply.run_cpu_time_us":34247,"apply.run_wall_time_us":140773,"cfile_cache_hit":86,"cfile_cache_hit_bytes":89913,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6218,"cfile_init":2,"lbm_read_time_us":1511,"lbm_reads_lt_1ms":12,"num_ops":22,"prepare.queue_time_us":1664,"prepare.run_cpu_time_us":7248,"prepare.run_wall_time_us":7682,"raft.queue_time_us":1222,"raft.run_cpu_time_us":12434,"raft.run_wall_time_us":22829,"replication_time_us":11100,"thread_start_us":30212,"threads_started":4,"wal-append.queue_time_us":1335,"wal-append.run_cpu_time_us":5300,"wal-append.run_wall_time_us":1176762}]]}
I20260709 13:03:05.668591  6035 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:05.662352  6116 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:05.615283  6151 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:05.699155  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:05.683809  6123 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:05.701656  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:05.701793  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 89132.8us
I20260709 13:03:05.703840  1641 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:03:05.707690  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.684010  5647 raft_consensus.cc:697] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:05.693208  5912 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:05.709908  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:05.713398  2304 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:05.710717  1465 catalog_manager.cc:5697] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:05.708310  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:05.698231  6112 raft_consensus.cc:3055] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:05.699847  6077 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.696324  6149 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:05.693024  6008 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:05.746091  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:05.702586  6116 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:05.744724  2297 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:05.713047  5912 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.720989  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:05.696734  6046 consensus_queue.cc:1059] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:03:05.724473  6171 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.730356  1465 catalog_manager.cc:5697] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:03:05.696213  5940 consensus_queue.cc:1059] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:03:05.711022  6010 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:05.729532  6077 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:05.745568  6201 log.cc:927] Time spent T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:03:05.707495  6064 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:03:05.726226  6112 raft_consensus.cc:740] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:05.766343  6188 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:05.771948  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:05.747203  5589 consensus_queue.cc:237] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:05.773262  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 90340.9us
W20260709 13:03:05.796417  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.777964  6007 consensus_queue.cc:237] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.757706  6126 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:05.767158  1465 catalog_manager.cc:5697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:05.787171  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:05.757916  5912 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:05.765288  6169 consensus_meta.cc:287] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.622s	user 0.011s	sys 0.000s
I20260709 13:03:05.869688  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:05.870645  2304 raft_consensus.cc:2393] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 12: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:05.872018  1568 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:03:05.872417  1568 pending_rounds.cc:85] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 8
I20260709 13:03:05.872659  1568 pending_rounds.cc:107] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:03:05.872910  1568 raft_consensus.cc:2889] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:05.759713  6116 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:05.796290  6209 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:03:05.781641  6093 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:05.723989  6118 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:05.818987  6126 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.838464  5915 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:05.812017  5647 consensus_queue.cc:237] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:05.842706  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:05.838701  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.745906  6137 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.745709  6202 consensus_queue.cc:1059] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.186s
I20260709 13:03:05.840483  6112 consensus_queue.cc:260] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.710628  6034 consensus_queue.cc:237] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:05.850802  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 78927.2us
W20260709 13:03:05.955380  6218 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:03:05.759403  6149 raft_consensus.cc:697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:05.844132  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:05.947270  6093 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.842061  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:05.856554  1565 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:05.748172  6047 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.857487  1562 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
W20260709 13:03:05.860870  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 56105.9us
I20260709 13:03:05.930955  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:05.956645  6126 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:05.951555  2300 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:05.955377  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:05.955750  6215 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:03:05.994261  6077 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:05.958348  1636 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:05.964941  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.009769  1714 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 15: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:05.871511  5629 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:05.896894  6176 raft_consensus.cc:697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:05.883330  6173 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:05.887202  6035 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:05.970603  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:05.767588  6010 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:05.980688  1642 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:05.987598  6093 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:05.981690  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.043102  1718 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:05.947088  1465 catalog_manager.cc:5697] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:05.988526  1607 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:05.990717  6216 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:03:05.991204  6194 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:03:05.993824  5942 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.597s	user 0.007s	sys 0.004s
I20260709 13:03:05.963649  6169 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:05.999665  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:05.957494  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:06.059265  2297 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:06.012141  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.062211  1532 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:06.065376  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.065524  6035 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:06.066220  5995 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:06.021035  1711 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:06.011408  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.058687  6194 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:06.067747  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.068981  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.058874  5942 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.070259  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:06.023904  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 341002us
I20260709 13:03:06.026268  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:06.031160  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:06.034142  1643 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:03:06.072990  1635 raft_consensus.cc:2393] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 12: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:05.981519  6128 raft_consensus.cc:2764] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:03:06.047241  1638 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:03:06.074482  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1251) took 1136 ms. Trace:
I20260709 13:03:06.074963  2298 rpcz_store.cc:276] 0709 13:03:04.938224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.946305 (+  8081us) service_pool.cc:224] Handling call
0709 13:03:06.074309 (+1128004us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400686ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:06.074424 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":621654912}
I20260709 13:03:06.075582  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:06.048288  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.046768  6010 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:06.052210  1640 raft_consensus.cc:1240] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:03:06.086725  1560 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:06.087029  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:06.055094  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 66648.7us
I20260709 13:03:06.057251  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:06.024017  6136 consensus_meta.cc:287] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.630s	user 0.007s	sys 0.006s
I20260709 13:03:06.022719  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:06.089259  1712 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:06.089906  1639 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:03:06.039362  6112 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:06.073515  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:06.092317  1643 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:06.074271  1644 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:06.048035  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:06.076416  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.049589  1465 catalog_manager.cc:5697] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 5 to 7, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 7 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:03:06.069065  6194 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:06.079128  6191 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.075714  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:06.096324  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.080286  1636 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:03:06.097157  1711 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:06.081816  6224 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:06.098811  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.087628  1718 raft_consensus.cc:2376] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11. Re-sending same reply.
I20260709 13:03:06.082195  1715 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:03:06.082904  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.102087  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.083277  5942 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:06.091295  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:06.082692  6176 consensus_queue.cc:237] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:06.103667  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.103463  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:06.107340  1639 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:06.115542  6194 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:06.116235  6194 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.116726  6194 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:06.123651  1607 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:06.128077  5995 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:06.128861  5995 raft_consensus.cc:697] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:06.130059  5995 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.138954  6206 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.139830  6206 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:06.143394  6206 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:06.144536  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:06.146003  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:06.079607  6205 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:03:06.147591  6224 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:06.148115  6224 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.148490  6224 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:06.150542  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:06.151635  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 52924.4us
I20260709 13:03:06.146692  1635 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:06.153075  1682 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:06.101382  1714 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:03:06.155854  2298 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:06.157433  1714 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:06.159273  1465 catalog_manager.cc:5697] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 6 to 8, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 8 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:06.162861  1639 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:06.164232  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.154716  1682 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:06.164969  1718 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:06.176584  1465 catalog_manager.cc:5697] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 4 to 10, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 10 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:03:06.188508  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:06.189491  2298 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:06.189833  1682 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:06.191947  1640 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:03:06.196111  1465 catalog_manager.cc:5697] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 6 to 8, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 8 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.211637  1682 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:06.174238  6128 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:06.171048  6136 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.171871  6171 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:06.220448  1682 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:06.232402  1682 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:06.234869  6221 log.cc:927] Time spent T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:03:06.230860  6200 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:06.163793  6219 consensus_queue.cc:1059] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:03:06.238966  5629 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:06.240157  5907 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:06.243564  1640 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:06.245321  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.246856  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.248647  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.249974  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:06.250528  1716 raft_consensus.cc:2376] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10. Re-sending same reply.
I20260709 13:03:06.251194  6224 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.254905  6224 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:06.256412  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:06.257174  1716 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:06.264405  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:06.197418  6008 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.273998  1681 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:06.266618  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:06.268404  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:06.274410  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:06.275228  1562 raft_consensus.cc:2368] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11.
I20260709 13:03:06.269083  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:06.275727  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:06.271925  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:06.277072  1710 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:03:06.277511  1710 pending_rounds.cc:85] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 6
I20260709 13:03:06.277757  1710 pending_rounds.cc:107] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:03:06.278000  1710 raft_consensus.cc:2889] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:06.278733  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:06.280143  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2234) took 1276 ms. Trace:
I20260709 13:03:06.280548  1642 rpcz_store.cc:276] 0709 13:03:05.003830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.014281 (+ 10451us) service_pool.cc:224] Handling call
0709 13:03:06.279982 (+1265701us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b5400155254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:06.280082 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1418213248}
W20260709 13:03:06.285243  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 376231us
I20260709 13:03:06.275822  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:06.319937  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:06.320006  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:06.321084  1568 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:06.322954  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:06.323657  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:06.323835  1642 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:06.226962  6128 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:06.335632  1566 raft_consensus.cc:2368] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11.
I20260709 13:03:06.342285  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:06.343899  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.344619  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.346397  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 72473.8us
W20260709 13:03:06.344954  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 70613.8us
I20260709 13:03:06.352773  1562 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:06.370246  1681 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:06.347515  2300 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:03:06.372841  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:06.366781  2268 connection.cc:256] RPC call timeout handler was delayed by -2.3327555110000731s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17820000000000003s for a response
I20260709 13:03:06.391863  1642 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:06.391739  1465 catalog_manager.cc:5697] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:06.398640  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:06.170118  6149 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.393579  2269 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:06.392041  1683 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:06.315068  5886 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:06.427776  1465 catalog_manager.cc:5697] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:06.434639  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 80693.5us
I20260709 13:03:06.441041  6008 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:03:06.442286  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 69586.4us
I20260709 13:03:06.443203  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:06.431540  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.345167  6171 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.445376  1710 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:06.449272  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:06.451462  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:06.347997  6077 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:06.352985  5629 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.404435  6200 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:06.448944  6205 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:06.461316  6191 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:06.466873  1560 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:03:06.506855  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:06.467833  1562 raft_consensus.cc:3055] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:06.513268  1562 raft_consensus.cc:740] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:06.460230  1566 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:06.515223  1712 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:06.469873  6136 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:06.477141  6128 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.460881  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:06.450299  1465 catalog_manager.cc:5697] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 7, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 7 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:06.503087  1716 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:06.518061  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:06.443931  6145 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:06.519052  1568 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
W20260709 13:03:06.520378  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 90260.1us
I20260709 13:03:06.522001  1534 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:06.526398  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 65627.1us
I20260709 13:03:06.529479  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.481737  5885 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:06.542104  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:06.523823  6200 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:06.550048  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:06.553452  1715 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:03:06.524484  6213 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:06.546195  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.588621  1716 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:06.577491  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.556232  1562 consensus_queue.cc:260] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.531765  6191 raft_consensus.cc:697] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:06.543495  6205 raft_consensus.cc:697] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:06.594384  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.540954  6136 raft_consensus.cc:697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:06.602057  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.611109  1562 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:06.619000  6176 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.050s	sys 0.056s
W20260709 13:03:06.619709  6176 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.050s	sys 0.056s
I20260709 13:03:06.629868  1562 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:06.557834  6035 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:06.637432  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:06.638324  1716 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 8.
I20260709 13:03:06.588209  6171 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:06.546376  6206 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:03:06.652925  1710 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: [7.4-9.5]
W20260709 13:03:06.650604  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 214574us
I20260709 13:03:06.557852  5629 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:06.636396  6188 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:06.648118  6152 consensus_meta.cc:287] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.150s	user 0.012s	sys 0.001s
I20260709 13:03:06.671190  1710 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:03:06.654563  5995 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.010s	sys 0.048s
I20260709 13:03:06.603282  5885 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:03:06.676385  1711 raft_consensus.cc:2368] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12.
I20260709 13:03:06.675580  1710 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:03:06.661445  6116 consensus_meta.cc:287] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.711s	user 0.010s	sys 0.002s
W20260709 13:03:06.672897  6010 consensus_meta.cc:287] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.001s
W20260709 13:03:06.669925  6194 consensus_meta.cc:287] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.002s
W20260709 13:03:06.676355  5995 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.010s	sys 0.048s
W20260709 13:03:06.681540  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 159653us
I20260709 13:03:06.688696  1714 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:06.692935  1714 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:06.690452  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:06.694880  1562 raft_consensus.cc:2410] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:03:06.697379  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:06.698755  6226 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:06.700556  6226 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:06.701534  6136 consensus_queue.cc:237] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.703665  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:06.705040  6058 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:03:06.690991  5915 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:06.702028  6226 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:06.711191  1635 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:06.693020  6010 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.717705  1710 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:06.719189  6010 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:06.706593  6194 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:06.726171  6194 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:06.709635  5915 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:06.730058  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 341228us
W20260709 13:03:06.730647  2268 connection.cc:256] RPC call timeout handler was delayed by -2.1390758880000362s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
W20260709 13:03:06.731113  2268 connection.cc:256] RPC call timeout handler was delayed by -1.9736594220000256s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2231s for a response
W20260709 13:03:06.731504  2268 connection.cc:256] RPC call timeout handler was delayed by -1.5829960420000475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
W20260709 13:03:06.731886  2268 connection.cc:256] RPC call timeout handler was delayed by -1.4228127050000694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
W20260709 13:03:06.737033  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 366944us
I20260709 13:03:06.742404  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:06.677671  6188 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:06.682384  6116 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:06.711077  6226 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:06.727995  5915 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:06.760638  5912 consensus_meta.cc:287] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.821s	user 0.010s	sys 0.000s
I20260709 13:03:06.755553  1681 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:06.729259  6191 consensus_queue.cc:237] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.751219  1562 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:03:06.763701  6116 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:06.764600  6019 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:06.764907  6188 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:06.753183  6152 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.754791  6034 consensus_queue.cc:1059] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:03:06.768731  1568 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:06.766894  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:06.747681  6077 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:06.774979  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 197601us
I20260709 13:03:06.685490  6200 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:06.685675  6205 consensus_queue.cc:237] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.741676  5604 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:03:06.779093  5912 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.817458  6194 consensus_queue.cc:1059] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:06.816319  6019 raft_consensus.cc:2764] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:03:06.820935  5912 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:06.747510  6137 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 13:03:06.786202  6093 consensus_meta.cc:287] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.724s	user 0.009s	sys 0.002s
W20260709 13:03:06.774248  6112 consensus_meta.cc:287] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.674s	user 0.007s	sys 0.005s
I20260709 13:03:06.787142  6007 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:06.788956  6010 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:06.789424  6047 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:06.789669  6064 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:06.807696  2268 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:06.815382  1568 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:03:06.819308  6194 raft_consensus.cc:3055] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:06.823035  1565 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 11: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:06.785190  6126 consensus_meta.cc:287] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.772s	user 0.005s	sys 0.005s
I20260709 13:03:06.824144  1564 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:03:06.777807  6008 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:03:06.831032  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 127434us
I20260709 13:03:06.808138  6077 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:06.836571  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:06.835424  6194 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:06.822419  6246 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:03:06.790903  6226 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.832228  6093 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:06.790890  6152 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:06.852025  1534 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:06.854238  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:06.853219  6126 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.870532  6253 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.870776  1709 raft_consensus.cc:2368] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:06.872224  1712 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 9: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:06.884370  1643 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:06.884291  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:06.885174  2304 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:06.887890  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:06.889815  1643 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:03:06.883088  1532 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:06.872112  6126 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:06.874027  5886 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.874536  6145 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.875376  1533 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:06.876237  6206 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:06.879853  5869 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:06.882364  6034 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:06.838737  6019 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:06.894973  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:06.853214  6077 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:06.865748  6039 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.896872  2268 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:06.898538  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:06.883584  6112 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:06.901960  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:06.901517  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:06.915318  1643 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:06.907478  6116 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:06.897609  5912 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:03:06.895762  5995 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:03:06.914774  6226 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:06.901973  6249 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:03:06.937871  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 46839.8us
I20260709 13:03:06.942483  6151 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:06.899686  5886 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.924289  6034 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.914463  6220 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:03:06.902493  6252 consensus_queue.cc:1059] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:03:06.896584  6253 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:06.956537  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 70743.8us
I20260709 13:03:06.906689  5869 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.904805  6145 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.965893  5886 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:06.953380  6188 raft_consensus.cc:3055] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:06.920404  6223 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:03:06.909072  6194 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:06.972862  6253 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:06.957518  6266 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:06.953248  6249 raft_consensus.cc:697] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:06.997823  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 85480.5us
I20260709 13:03:06.975160  5647 raft_consensus.cc:3055] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:06.975813  6204 raft_consensus.cc:3055] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:06.976290  6093 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:06.952806  6116 raft_consensus.cc:697] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:06.976806  6111 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:06.977118  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 975) took 1393 ms. Trace:
I20260709 13:03:07.021754  6266 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:07.029232  6266 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:06.978492  1639 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:03:06.921904  6008 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:06.983374  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:06.997941  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 101207us
I20260709 13:03:06.926541  6019 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:03:07.032975  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:07.017771  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 103003us
I20260709 13:03:07.018364  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:07.023519  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 71798.7us
I20260709 13:03:07.024158  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:06.994463  6188 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:07.030201  1639 pending_rounds.cc:85] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 9
I20260709 13:03:07.028548  1644 rpcz_store.cc:276] 0709 13:03:05.583389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.583536 (+   147us) service_pool.cc:224] Handling call
0709 13:03:05.598953 (+ 15417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.092842 (+493889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:06.976883 (+884041us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54002d8a54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:06.977059 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1092753792}
I20260709 13:03:06.961145  6034 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.035399  2297 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:07.035782  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:06.999334  6152 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:07.020071  5647 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:07.019840  6204 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:07.055846  6204 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.051297  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:07.057001  6204 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:07.057336  1563 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:07.057907  6145 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:07.039145  1639 pending_rounds.cc:107] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 13.10
I20260709 13:03:07.058754  1639 raft_consensus.cc:2917] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Aborting config change with OpId 13.10: Aborted: Op aborted by new leader
I20260709 13:03:07.059376  1639 raft_consensus.cc:2889] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:07.060596  5869 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:07.054383  6188 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.055960  6249 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.053009  6077 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.045428  6194 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:07.077713  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.090647  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 57735.3us
I20260709 13:03:07.091995  2268 leader_election.cc:400] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11.
I20260709 13:03:07.092358  2268 leader_election.cc:403] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:07.094290  6225 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:07.094877  6225 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.097658  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:07.098536  1565 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:07.103183  1563 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:07.107928  1560 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:07.108856   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:07.109419  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:07.110293  2297 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:07.111524  1609 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:07.126528  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:07.126797  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.018414  6264 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:03:07.101634  5869 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:07.146076  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 54213.4us
I20260709 13:03:07.149088  2268 leader_election.cc:400] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11.
I20260709 13:03:07.135372  6206 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:07.149479  2268 leader_election.cc:403] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:03:07.137190  6188 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:07.155086  6271 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:07.155856  6260 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:03:07.155987  6271 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:07.158015  6261 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:03:07.159847  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:07.149319  6280 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11.
I20260709 13:03:07.142371  6116 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.161518  6280 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:07.164556  6077 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:07.165548  6077 raft_consensus.cc:2764] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:03:07.165618  6265 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:07.160586  6034 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:07.133392  6269 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:07.186347  6271 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:07.188166  6222 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:03:07.096069  5647 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:07.171118  6234 log.cc:927] Time spent T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.475s	user 0.004s	sys 0.000s
I20260709 13:03:07.194152  1717 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:07.212970  1713 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:07.172331  6269 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.171769  6206 raft_consensus.cc:697] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:07.216843  6269 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:07.218724  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 57679.6us
I20260709 13:03:07.220216  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:07.227906  6262 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:03:07.229852  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2274) took 1246 ms. Trace:
I20260709 13:03:07.230288  1718 rpcz_store.cc:276] 0709 13:03:05.983506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.167155 (+183649us) service_pool.cc:224] Handling call
0709 13:03:06.180968 (+ 13813us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:06.181344 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.181376 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:06.184212 (+  2836us) log.cc:844] Serialized 26 byte log entry
0709 13:03:06.982142 (+797930us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b1800446ce0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:03:07.020460 (+ 38318us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:07.135273 (+114813us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.135532 (+   259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.135620 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.229566 (+ 93946us) raft_consensus.cc:1685] finished
0709 13:03:07.229671 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.229797 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":206538,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":800,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":15597,"spinlock_wait_cycles":25404160,"thread_start_us":127028,"threads_started":2,"wal-append.queue_time_us":1027596}
I20260709 13:03:07.235929  1718 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:07.176826  6077 raft_consensus.cc:3055] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:07.205618  6213 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.241027  5647 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:07.241291  6213 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:07.255576  1709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2279) took 1259 ms (client timeout 1620 ms). Trace:
W20260709 13:03:07.256194  1709 rpcz_store.cc:269] 0709 13:03:05.995808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.060977 (+ 65169us) service_pool.cc:224] Handling call
0709 13:03:07.255307 (+1194330us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b5400339fd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:07.255498 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1850323712}
I20260709 13:03:07.266014  2297 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 13:03:07.268066  6227 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.741s	user 0.002s	sys 0.000s
I20260709 13:03:07.271450  6268 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:07.271163  6206 consensus_queue.cc:237] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:07.272334  6273 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:07.272426  6266 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.247975  6149 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.275312  2302 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:07.275552  6149 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:07.274549  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:07.245827  6267 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.277334  6267 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:07.277652  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:07.279541  6267 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:07.280154  6267 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:07.277702  6149 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:07.281524  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:07.281579  6267 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:03:07.284188  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:07.286183  6145 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:07.287006  6145 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.287969  6267 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:07.288450  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:07.300930  1636 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:07.311544  6093 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:07.313246  6171 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:07.313728  6171 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:07.314100  6171 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:03:07.320032  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 99922us
I20260709 13:03:07.321166  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:07.324506  1714 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:07.324361  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:07.240926  6077 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:07.326236  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:07.328624  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:07.333125  6276 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.151s	user 0.000s	sys 0.003s
I20260709 13:03:07.333812  6204 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:07.335716  2268 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:07.338819  2268 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:07.342937  5604 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:07.343652  6268 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:07.346998  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:07.348618  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:07.350908  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:07.349515  1714 raft_consensus.cc:2376] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12. Re-sending same reply.
I20260709 13:03:07.351537  1709 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:07.353765  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:07.355423  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:07.326831  6267 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:03:07.355876  6266 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:07.357434  6194 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:07.358984  6169 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:07.340960  6204 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:07.340675  6280 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11.
I20260709 13:03:07.355326  6253 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:07.363554  6077 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:07.378242  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2270) took 1446 ms (client timeout 1925 ms). Trace:
W20260709 13:03:07.378708  1717 rpcz_store.cc:269] 0709 13:03:05.931814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.931982 (+   168us) service_pool.cc:224] Handling call
0709 13:03:07.378056 (+1446074us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001365d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:07.378180 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2510077312}
I20260709 13:03:07.379771  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:07.380676  1714 raft_consensus.cc:2368] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:07.383493  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:07.384191  1717 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:07.378273  6077 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:07.387701  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:07.389124  1717 raft_consensus.cc:2410] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:03:07.389348  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:07.396266  6269 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.397984  1709 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:07.398069  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:07.401448  6269 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:07.398887  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:07.408854  1709 raft_consensus.cc:2410] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:07.405411  6253 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:07.411039  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.409497  6149 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:07.414005  6264 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.414783  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:07.414593  6149 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:07.414702  6264 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.416896  6149 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:07.417456  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:07.417642  6264 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:07.418107  1643 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.412302  6145 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:07.424234  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:07.425069  1644 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:07.430080  1643 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:07.432595  1638 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:07.434804  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:07.439427  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.444702  1644 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:07.438149  1714 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:07.448082  1639 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:07.457152  5629 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:07.458796  6171 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.492547  6246 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.493257  6246 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:07.503191  6246 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:07.506361  6225 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:07.510416  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:07.511237  1533 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:07.512756  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 900) took 1287 ms. Trace:
I20260709 13:03:07.528223  1710 rpcz_store.cc:276] 0709 13:03:06.225086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.680575 (+455489us) service_pool.cc:224] Handling call
0709 13:03:06.680901 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.718226 (+ 37325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.512505 (+794279us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b54003db3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:07.512701 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84107136}
I20260709 13:03:07.498128  6171 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:07.522408  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 895) took 1303 ms. Trace:
I20260709 13:03:07.523648  1714 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:07.524631  1717 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:07.532162  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:07.531949  1715 rpcz_store.cc:276] 0709 13:03:06.219156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.577128 (+357972us) service_pool.cc:224] Handling call
0709 13:03:07.522235 (+945107us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b5400339ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:07.522351 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":533851520}
I20260709 13:03:07.534536  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:07.537628  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.537837  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:07.538560  1564 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.541814  6271 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:07.542150  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:07.543557  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:07.544243  1714 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:07.544245  1567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:03:07.545064  1567 raft_consensus.cc:606] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:07.545446  1567 raft_consensus.cc:614] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:07.545702  1714 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:03:07.545861  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 651173us
I20260709 13:03:07.544348  1710 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:07.549423  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 192643us
I20260709 13:03:07.546814  1534 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:07.563323  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:07.563809  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:07.564528  1717 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:07.565204  1532 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:07.565860  1534 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:07.572798  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:07.573547  1565 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:03:07.575271  1565 raft_consensus.cc:1217] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Deduplicated request from leader. Original: 8.9->[9.10-9.10]   Dedup: 9.10->[]
I20260709 13:03:07.575791  1565 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:03:07.577270  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:07.580200  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:07.581369  1565 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:03:07.586158  1562 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: [10.11-13.13]
I20260709 13:03:07.586750  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:07.562240  6225 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:07.583014  6112 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:07.578212  6111 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:07.608883  6111 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:07.609352  6111 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.587929  6284 consensus_queue.cc:1059] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:03:07.612212  6284 raft_consensus.cc:3055] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:03:07.612649  6284 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:07.613646  6284 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.584646  1564 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:07.603049  6273 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:07.604066  6219 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:07.592430  6077 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:03:07.583344  6204 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:03:07.561707  6191 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:07.606093  6296 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 13:03:07.623482  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.629668  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:07.623611  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:07.632205  1562 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:07.626852  1717 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:07.583222  5647 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:07.628078  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 896) took 1408 ms. Trace:
W20260709 13:03:07.635566  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.633291  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:07.633471  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:07.640805  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:07.642361  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:07.643352  1717 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:07.643611  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:07.638906  1714 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:07.635516  1716 rpcz_store.cc:276] 0709 13:03:06.219919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.670259 (+450340us) service_pool.cc:224] Handling call
0709 13:03:06.682424 (+ 12165us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:06.708957 (+ 26533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:06.708989 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:06.713876 (+  4887us) log.cc:844] Serialized 52 byte log entry
0709 13:03:07.523669 (+809793us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b1800348d00. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:03:07.541333 (+ 17664us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:07.561369 (+ 20036us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.584151 (+ 22782us) log.cc:844] Serialized 14 byte log entry
0709 13:03:07.584719 (+   568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.584801 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:07.627767 (+ 42966us) raft_consensus.cc:1685] finished
0709 13:03:07.627804 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:07.627990 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22195,"prepare.queue_time_us":405,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":82,"raft.queue_time_us":43862,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":11475,"spinlock_wait_cycles":23458688,"thread_start_us":17414,"threads_started":1,"wal-append.queue_time_us":795074}
I20260709 13:03:07.649130  1716 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:07.624331  6249 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:07.655241  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:07.654199  6206 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.658104  2302 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:07.659660  6077 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:07.660117  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 892) took 1444 ms. Trace:
I20260709 13:03:07.657045  6206 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.660617  1712 rpcz_store.cc:276] 0709 13:03:06.215693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.553767 (+338074us) service_pool.cc:224] Handling call
0709 13:03:07.659879 (+1106112us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54001473d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:07.660053 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":627044096}
I20260709 13:03:07.660275  6077 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:07.655990  2297 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.668488  6269 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:07.638191  6191 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.669338  6072 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:07.669852  6072 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:07.670094  6039 raft_consensus.cc:3055] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:07.670542  6039 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:07.671422  6039 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.588122  6058 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:03:07.676550  6039 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:07.628746  6284 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:03:07.629355  5869 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.702039  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:07.708988  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:07.707902  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:07.723767  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:07.669346  6269 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.735852  1562 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:07.738008  2297 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:07.739490  1609 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:07.711808  6246 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:07.711535  5885 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:03:07.747253  2302 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:07.729646  6137 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:07.737007  6191 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:07.745410  6297 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:07.714365  5869 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.670817  6072 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.758186  6137 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.734309  5604 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:03:07.751729  6077 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.753914  6253 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:07.769434  6264 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.799070  6253 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:07.799476  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 983) took 1646 ms. Trace:
I20260709 13:03:07.799909  1635 rpcz_store.cc:276] 0709 13:03:06.153051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.243461 (+ 90410us) service_pool.cc:224] Handling call
0709 13:03:06.243811 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:06.711653 (+467842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.799204 (+1087551us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b54001e9454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:07.799410 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1074514688}
I20260709 13:03:07.800628  5869 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:07.800175  6253 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.802847  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:07.804329  6269 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:07.805194  6264 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.807662  6264 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:07.808830  6265 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:03:07.809266  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.810094  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2188) took 1829 ms. Trace:
I20260709 13:03:07.809273  6265 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:07.810540  1713 rpcz_store.cc:276] 0709 13:03:05.980965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.987663 (+  6698us) service_pool.cc:224] Handling call
0709 13:03:05.987950 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.213645 (+1225695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.809850 (+596205us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54001360d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:07.810034 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2816943360}
I20260709 13:03:07.811265  6265 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.812791  6265 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:07.788102  6137 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.817489  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:07.818214  6111 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.820078  1607 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:07.820719  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2706) took 1081 ms. Trace:
I20260709 13:03:07.821029  6269 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:07.821478  1607 leader_election.cc:400] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:07.821103  1560 rpcz_store.cc:276] 0709 13:03:06.739476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.739613 (+   137us) service_pool.cc:224] Handling call
0709 13:03:07.099659 (+360046us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:07.108537 (+  8878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:07.820521 (+711984us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54000a7dd0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:07.820663 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":827270912}
I20260709 13:03:07.821915  6269 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:07.822718  6077 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:07.822849  6151 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.823431  6151 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.826066  6151 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:07.827536  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.827998  6299 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:03:07.828372  5589 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:07.756266  6246 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.815979  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:07.831601  2298 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:07.831830  1635 raft_consensus.cc:2393] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:07.831902  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:07.817670  6213 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:07.829697  2302 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:07.831023  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:07.821859  1607 leader_election.cc:403] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:07.832602  1713 raft_consensus.cc:2410] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:03:07.836839  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:07.837620  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:07.823134  6269 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.837663  2302 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:07.843523  2304 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:07.833354  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:07.840482  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:07.839802  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:07.833400  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:07.848598  2304 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:07.850896  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:03:07.852335  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.855762  2297 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:07.852998  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:07.862406  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.867585  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:07.836257  5629 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:07.871977  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 124932us
I20260709 13:03:07.874356  2268 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:07.857565  5885 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:07.865928  6145 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:07.875443  2302 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:07.841564  6191 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:07.846123  6171 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:07.882392  2268 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:07.860682  6169 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:07.854446  6213 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:07.912065  2305 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:07.912684  6246 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:07.914988  6297 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:07.880409  2302 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:07.884177  2297 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:03:07.878708  5629 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:07.875895  6190 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:07.875707  5885 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:07.895044  6206 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:07.895889  6111 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:07.899305  6191 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:07.903594  6266 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:07.921365  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:07.924945  2302 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:07.944322  1713 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:03:08.000022  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:08.000046  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:08.001616  1715 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:07.902606  6137 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:08.020903  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 170573us
I20260709 13:03:08.024852  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:07.925698  6190 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:07.930081  5885 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:08.038467  6039 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:08.040678  6039 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:08.042477  6263 log.cc:927] Time spent T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 1.024s	user 0.003s	sys 0.001s
I20260709 13:03:08.043499  6039 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:08.019838  6297 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:08.049322  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:08.049538  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 182062us
I20260709 13:03:08.050169  2302 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:08.052310  5629 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:08.052583  5995 raft_consensus.cc:3055] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:08.053177  5995 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:08.072022  6310 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:03:08.054291  5995 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.037719  6190 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:08.080852  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 193287us
I20260709 13:03:08.098222  6145 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:08.098366  5944 raft_consensus.cc:3055] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:08.103776  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:08.105211  1718 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:08.106334  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2275) took 2112 ms. Trace:
I20260709 13:03:08.107327  1711 rpcz_store.cc:276] 0709 13:03:05.994447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.099985 (+105538us) service_pool.cc:224] Handling call
0709 13:03:08.058402 (+1958417us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b54001482d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:08.106274 (+ 47872us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1316819200}
W20260709 13:03:08.116808  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 525383us
I20260709 13:03:08.119716  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:08.121331  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:08.122174  1636 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:08.133332  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:08.141407  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 217651us
I20260709 13:03:08.143193  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:08.145206  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:08.145974  2300 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:03:08.149745  1636 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:08.153702  1635 raft_consensus.cc:1240] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:08.118412  5629 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:03:08.155504  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 519170us
I20260709 13:03:08.167131  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:08.167368  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:08.167856  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:08.168471  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:08.169073  2296 raft_consensus.cc:2410] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 14 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:03:08.170267  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:08.171281  2299 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:08.104698  5944 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:08.140933  5915 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:08.234239  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:08.235307  2301 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:08.139642  6206 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.240262  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:08.109692  6272 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.123s
I20260709 13:03:08.148945  6225 raft_consensus.cc:697] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:08.256304  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 138853us
I20260709 13:03:08.267688  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:08.269208  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:08.272333  6273 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:08.273110  6273 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:08.270721  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:08.275224  6222 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:08.140283  6246 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 13:03:08.281495  6072 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:08.260217  6145 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:08.171865  6213 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 13:03:08.287822  6008 consensus_queue.cc:1059] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:03:08.294842  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.300853  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 276165us
I20260709 13:03:08.277388  5885 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:08.239287  6253 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:08.310204  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 167131us
I20260709 13:03:08.312916  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:08.313001  1636 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:08.316411  1635 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:03:08.318092  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:08.319244  1560 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:08.254015  6206 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.255926  6313 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:03:08.324630  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:08.329806  1636 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:03:08.330003  1560 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:08.322532  6137 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:08.341429  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 199012us
I20260709 13:03:08.314635  6188 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:03:08.328326  6008 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:08.319190  5438 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:03:08.343039  6223 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:08.344399  1607 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:08.345559  6213 raft_consensus.cc:3055] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:08.317953  6253 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.310307  5944 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.343144  6246 raft_consensus.cc:3055] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:08.356738  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:08.342819  6035 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:08.363269  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:08.363543  1565 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:08.360890  1564 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:08.362751  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:08.364768  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:08.366267  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:08.366649  1569 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:03:08.351506  6191 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:08.351466  6213 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:08.373780  6213 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.375046  6213 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:08.361106  6226 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.376102  6226 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.378239  6226 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:08.370306  1567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:03:08.354771  6320 raft_consensus.cc:3055] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:08.380385  6008 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:08.379362  5885 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:08.392460  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.380296  1567 raft_consensus.cc:606] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:08.380739  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:08.381101  1564 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:08.385351  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:08.341641  6225 consensus_queue.cc:237] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:08.370719  6035 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.358093  6246 raft_consensus.cc:740] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:08.371567  6191 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.408927  6326 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:08.443145  5944 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:08.409062  6320 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:08.487862  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:08.368634  6149 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 13:03:08.443387  6188 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:08.490743  1563 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:08.484251  6191 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:08.513808  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 164377us
I20260709 13:03:08.483310  6222 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:08.489480  6112 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.518638  1682 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:08.526978  1644 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:08.479501  6328 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:03:08.488014  6327 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:08.516201  5915 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:08.581653  6324 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:03:08.581830  6116 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:08.582284  6116 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:08.582957  6330 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:03:08.583247  6116 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:08.584517  6116 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:08.586118  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 274822us
W20260709 13:03:08.539697  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.591940  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:08.548261  6323 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:03:08.517217  6188 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:08.489719  6326 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.550036  6322 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:08.589066  6297 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.589053  6206 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:08.527128  6222 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.548545  6035 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:08.549853  5915 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.523818  6008 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.529168  6112 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:08.612140  6222 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:08.613183  6335 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:03:08.602487  6253 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:08.614598  6335 consensus_queue.cc:1059] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:03:08.617062  6335 consensus_queue.cc:1243] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:03:08.600279  6322 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:08.619549  6332 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:03:08.620478  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.592104  6320 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 7, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.622120  6320 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:08.624315  6321 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:08.626688  6149 raft_consensus.cc:3055] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:08.627097  6269 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.627987  6269 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.630468  6269 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:08.631793  6329 consensus_queue.cc:1175] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5184c101be204393adf3ac8b31f25ad6" responder_term: 11 status { last_received { term: 11 index: 10 } last_committed_idx: 7 last_received_current_leader { term: 11 index: 10 } }
I20260709 13:03:08.633689  6206 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:08.634298  6206 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:08.635736  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.637107  6206 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:08.633059  6266 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.613s	user 0.003s	sys 0.007s
I20260709 13:03:08.637892  6266 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:08.638620  6266 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.633286  5915 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:08.639175  6266 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:08.639848  6302 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.883s	user 0.001s	sys 0.004s
I20260709 13:03:08.639215  6191 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.641775  6266 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:08.642342  6266 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.643337  2298 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:03:08.643733  2298 pending_rounds.cc:85] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 9
I20260709 13:03:08.643826  5915 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.644133  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:08.643756  1636 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:08.644413  5915 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.645089  1640 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:08.644094  2298 pending_rounds.cc:107] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 13.10
I20260709 13:03:08.646592  1639 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:08.631474  6326 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:08.627259  6149 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:08.646607  2298 raft_consensus.cc:2917] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Aborting config change with OpId 13.10: Aborted: Op aborted by new leader
I20260709 13:03:08.656051  2298 raft_consensus.cc:2889] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:08.653541  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 285672us
I20260709 13:03:08.660590  2269 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:08.685405  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:08.688728  1635 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:03:08.617976  6322 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:08.642386  6191 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:08.731701  1569 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:08.710206  2269 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:08.590826  6327 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:08.646708  6266 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.735415  1563 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:08.698863  6271 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:08.604758  6297 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:08.743366  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 184301us
I20260709 13:03:08.747912  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:08.749626  1561 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:03:08.749912  1569 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:08.532894  6246 consensus_queue.cc:260] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.751713  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:08.726996  6326 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:08.773470  6266 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:08.783279  6327 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:08.755961  6191 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.787495  6191 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.789844  6191 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:08.791198  6188 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:08.732463  6280 consensus_queue.cc:1059] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:03:08.732134  6137 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:03:08.743937  6271 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:08.797348  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 209576us
I20260709 13:03:08.755402  6204 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:08.801617  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:08.803119  1639 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:08.804199  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:08.804896  1641 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
W20260709 13:03:08.805855  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.756935  6213 consensus_queue.cc:1059] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:03:08.796343  5915 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:08.779953  6326 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.802165  6271 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:08.739483  6149 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:08.798437  6334 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:03:08.811933  1683 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:08.828207  1561 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:03:08.829696  6190 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.712s	user 0.009s	sys 0.000s
I20260709 13:03:08.829608  5915 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.824821  6329 consensus_queue.cc:1059] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:03:08.830504  5915 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:08.831709  6223 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:08.832278  6223 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:08.833585  5915 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:08.839883  6338 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:08.840270  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:08.840441  6338 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.840925  6338 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:08.851660  1534 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:08.854210  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1274) took 1024 ms. Trace:
I20260709 13:03:08.854843  2305 rpcz_store.cc:276] 0709 13:03:07.830095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.830778 (+   683us) service_pool.cc:224] Handling call
0709 13:03:08.853882 (+1023104us) spinlock_profiling.cc:243] Waited 68.9 ms on lock 0x7b5400600254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:08.854126 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158567808}
I20260709 13:03:08.858170  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2194) took 2158 ms. Trace:
I20260709 13:03:08.859087  6339 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:03:08.870033  6149 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:08.875816  5629 consensus_queue.cc:1175] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3f64b76180a54492bb015e405caeacec" responder_term: 12 status { last_received { term: 10 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:08.810509  6008 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:08.879925  6188 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:08.880326  6008 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.880981  6008 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.884531  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:08.875890  1681 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:08.858749  1718 rpcz_store.cc:276] 0709 13:03:06.699360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.246494 (+547134us) service_pool.cc:224] Handling call
0709 13:03:08.104781 (+858287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.105905 (+  1124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.857897 (+751992us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b54001482d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:08.858093 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1856058112}
I20260709 13:03:08.883769  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:08.898250  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:08.899223  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 47702.5us
I20260709 13:03:08.830375  6190 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.901482  1683 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:08.974036  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 64222.8us
W20260709 13:03:08.974313  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 72953.2us
I20260709 13:03:08.900198  6169 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:08.979491  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:08.988155  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 80598.3us
I20260709 13:03:08.988415  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:09.000988  1643 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:09.001747  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:09.003793  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:09.003082  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 293003us
I20260709 13:03:09.004796  2300 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:08.903776  6339 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:08.903966  6297 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:09.009720  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:08.989818  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:09.034127  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:09.035140  1562 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:08.903759  6225 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 13:03:09.051571  2300 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:09.053740  6151 consensus_queue.cc:1059] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:09.055076  6151 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:09.057265  6266 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:09.003460  6335 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:03:09.003615  6310 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.061302  6310 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:09.064575  6190 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:09.065474  6310 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:09.000170  6280 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:09.067135  2303 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:09.066618  6280 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:09.071612  6280 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:09.081760  6335 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:09.082690  6335 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:09.085960  6335 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:09.089025  6335 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.089864  6335 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:09.000801  5589 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:09.093109  5589 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:09.093407  6225 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:09.094514  6335 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:09.094027  6225 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:08.972994  6204 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:08.988318  6194 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.094154  5589 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:09.124071  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 134361us
I20260709 13:03:09.007340  5915 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:08.989101  6169 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:09.104562  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:08.910214  6327 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.140s
I20260709 13:03:08.973379  6035 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.124s
I20260709 13:03:08.988023  6219 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.135s
I20260709 13:03:09.050537  6297 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:09.140128  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.188342  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:09.200434  1607 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.726s (SENT) [suppressed 38 similar messages]
I20260709 13:03:09.120609  6145 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:09.013756  6340 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.139s
I20260709 13:03:09.134588  5915 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:09.227378  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:09.228648  2300 raft_consensus.cc:2410] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:03:09.136971  6169 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:09.232259  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:09.233162  2300 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 10: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:09.129858  6194 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:09.230013  6322 consensus_meta.cc:287] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.002s
I20260709 13:03:09.212080  6326 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:09.129628  6204 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:09.256269  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 504651us
I20260709 13:03:09.204145  6008 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:09.209334  6206 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:09.204422  6093 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:09.185907  6222 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 13:03:09.236212  5944 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.741s	user 0.006s	sys 0.006s
I20260709 13:03:09.276086  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:09.230844  5915 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:09.337839  2300 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:09.279138  1682 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:09.220364  6145 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:09.355729  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 76700.2us
I20260709 13:03:09.357774  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:09.353015  6320 consensus_meta.cc:287] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.716s	user 0.008s	sys 0.006s
W20260709 13:03:09.371665  2297 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.000s	sys 0.029s
W20260709 13:03:09.377048  2297 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.000s	sys 0.030s
I20260709 13:03:09.372941  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:09.317451  6297 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.276855  6322 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:09.390985  2300 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:09.392932  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:09.393940  2296 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:09.395431  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:09.396245  2296 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:09.360869  6008 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:09.371872  6335 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:09.404913  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 403254us
W20260709 13:03:09.409756  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 404599us
I20260709 13:03:09.316591  6206 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:09.411988  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 421614us
I20260709 13:03:09.411625  2269 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:09.410624  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:09.336730  6093 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:09.417208  2300 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:09.418826  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:09.420591  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:09.417749  6137 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:09.418798  6223 consensus_meta.cc:287] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.584s	user 0.013s	sys 0.000s
I20260709 13:03:09.401695  6297 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:09.409467  5438 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.000s	sys 0.022s
W20260709 13:03:09.452502  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.458022  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:09.457270  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:09.426776  6345 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:09.485275  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:09.492071  1643 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:09.502597  1609 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:09.454994  5438 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.001s	sys 0.022s
I20260709 13:03:09.506242  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:09.511133  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 310753us
I20260709 13:03:09.457619  6191 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.423398  6008 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:03:09.546089  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:09.561977  6269 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:09.563194  6269 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.563807  6269 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:09.566861  6269 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:09.554459  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:09.578574  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:09.579421  2300 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:09.580483  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:09.583045  6359 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.457846  6190 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 13:03:09.592730  6297 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:09.509436  6188 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:09.457134  6223 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:09.642330  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:09.681128  1637 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 15: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:09.642742  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:09.681363  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:09.681768  1638 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:09.643240  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:09.644585  1560 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:09.645394  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:09.683334  1564 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:09.684276  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:09.645929  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:09.663833  1642 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:09.690223  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:09.690908  1560 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:09.691763  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:09.693454  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:09.695061  1569 raft_consensus.cc:2410] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:03:09.692312  1560 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
W20260709 13:03:09.663772  6116 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.012s	sys 0.009s
I20260709 13:03:09.713896  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:09.665597  6194 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:09.683990  6225 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:09.552644  6191 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:09.735044  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:09.716537  6116 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.013s	sys 0.009s
I20260709 13:03:09.698947  6340 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.688092  6269 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:09.732259  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:09.747233  1681 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:09.759727  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:09.705729  6204 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:09.762307  1569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:03:09.715494  6145 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:09.767030  1569 raft_consensus.cc:606] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:09.772878  1569 raft_consensus.cc:614] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:09.772563  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 270059us
I20260709 13:03:09.723326  6334 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:09.727398  6363 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:09.744109  6322 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:09.651993  6359 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:09.770969  6206 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:09.838759  6363 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:09.865231  6194 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.869725  6219 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:09.792147  6347 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:09.792618  6190 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.770244  6269 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:09.919767   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:03:09.784488  5589 consensus_queue.cc:1059] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.132s
I20260709 13:03:09.793111  6222 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 13:03:09.934974   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:09.816797  6008 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:09.770777  6340 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:09.923105  6204 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:09.955930  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:09.920146  6358 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:09.793615  6334 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:09.983338  6351 log.cc:927] Time spent T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.430s	user 0.000s	sys 0.001s
I20260709 13:03:09.999611  6223 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:10.001113  6191 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:10.001672  6366 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:03:09.923691  6219 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:10.004122  6361 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
I20260709 13:03:10.004741  6219 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:09.919180  6363 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:10.007339  6363 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:09.995456  6352 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 13:03:09.997848  6359 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:09.981487  6358 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.008482  6358 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:10.009091  6338 consensus_meta.cc:287] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.168s	user 0.014s	sys 0.005s
I20260709 13:03:09.921213  6194 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:10.011107  6334 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:09.987437  6354 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.348s	user 0.001s	sys 0.000s
I20260709 13:03:10.011473  6194 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:10.012579  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1288) took 1162 ms. Trace:
I20260709 13:03:10.013108  2298 rpcz_store.cc:276] 0709 13:03:08.850399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.850598 (+   199us) service_pool.cc:224] Handling call
0709 13:03:08.850999 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.851374 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.851409 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.851907 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 13:03:09.290718 (+438811us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:09.290785 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.298426 (+  7641us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.012279 (+713853us) raft_consensus.cc:1685] finished
0709 13:03:10.012309 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.012500 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11850,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"thread_start_us":438302,"threads_started":1,"wal-append.queue_time_us":613189}
I20260709 13:03:10.015136  2300 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:03:10.015379  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1289) took 1162 ms. Trace:
I20260709 13:03:10.015942  2297 rpcz_store.cc:276] 0709 13:03:08.853252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.853481 (+   229us) service_pool.cc:224] Handling call
0709 13:03:08.853839 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.854254 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.854296 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.854857 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:03:09.377433 (+522576us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:09.615754 (+238321us) log.cc:844] Serialized 14 byte log entry
0709 13:03:09.616057 (+   303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.616159 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.015098 (+398939us) raft_consensus.cc:1685] finished
0709 13:03:10.015125 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.015305 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":214732,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":77420,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":12722,"thread_start_us":735860,"threads_started":2,"wal-append.queue_time_us":697191}
I20260709 13:03:09.978634  6204 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:09.971287  5944 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:09.942071  6190 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:09.979605  6008 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.027827  6385 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.024034  6204 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:10.027782  6363 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:10.059018  6271 consensus_meta.cc:287] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.233s	user 0.011s	sys 0.004s
I20260709 13:03:10.072374  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 1067) took 1060 ms. Trace:
I20260709 13:03:10.068143  6222 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:10.072811  2296 rpcz_store.cc:276] 0709 13:03:09.010556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.397031 (+386475us) service_pool.cc:224] Handling call
0709 13:03:09.397384 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.397631 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:09.748258 (+350627us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.070356 (+322098us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.070473 (+   117us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:10.070540 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.070658 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.070827 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9702,"prepare.queue_time_us":194732,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"thread_start_us":644440,"threads_started":3}
I20260709 13:03:10.075807  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 1060) took 1195 ms. Trace:
I20260709 13:03:10.062968  6364 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.009523  6338 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.048199  6219 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:10.039556  5944 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.073485  6310 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.076303  2302 rpcz_store.cc:276] 0709 13:03:08.880745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.882450 (+  1705us) service_pool.cc:224] Handling call
0709 13:03:08.882788 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:08.883050 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:09.292480 (+409430us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.047372 (+754892us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.075442 (+ 28070us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:10.075502 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.075589 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.075738 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40567,"prepare.queue_time_us":569931,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":13472,"thread_start_us":1010780,"threads_started":3,"wal-append.queue_time_us":627207}
W20260709 13:03:10.076117  6169 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.821s	user 0.012s	sys 0.000s
W20260709 13:03:10.097747  6362 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 13:03:10.073769  6222 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.075356  6363 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.082185  6271 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.123641  5995 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:10.124666  6190 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:10.093609  6364 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:10.127753  6402 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.123730  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1293) took 1226 ms. Trace:
I20260709 13:03:10.128377  6402 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:10.127904  6360 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:03:10.129096  6364 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:10.128638  2304 rpcz_store.cc:276] 0709 13:03:08.897068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.897227 (+   159us) service_pool.cc:224] Handling call
0709 13:03:08.897573 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.910193 (+ 12620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.910225 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.987599 (+ 77374us) log.cc:844] Serialized 26 byte log entry
0709 13:03:09.424015 (+436416us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:09.424070 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.424167 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.123472 (+699305us) raft_consensus.cc:1685] finished
0709 13:03:10.123494 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.123670 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9880,"raft.queue_time_us":73082,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":19660,"thread_start_us":302370,"threads_started":1,"wal-append.queue_time_us":700269}
I20260709 13:03:10.140719  6269 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:10.141610  6338 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:10.142040  6249 raft_consensus.cc:993] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:03:10.098060  6219 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.130646  6402 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:10.072196  6385 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.152087  2299 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:10.148916  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.013s	sys 0.037s
I20260709 13:03:10.116761  6365 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:10.155225  6365 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:10.157871  6365 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:10.159471  6402 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.160099  6402 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.162384  6402 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:10.110755  6310 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.164878  6112 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:10.165145  6310 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:10.142712  6249 raft_consensus.cc:1081] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:03:10.165915  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 799) took 2596 ms. Trace:
I20260709 13:03:10.166302  1568 rpcz_store.cc:276] 0709 13:03:07.569552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.569715 (+   163us) service_pool.cc:224] Handling call
0709 13:03:07.570199 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:07.570503 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:07.570541 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:07.571137 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 13:03:07.715316 (+144179us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:07.743876 (+ 28560us) log.cc:844] Serialized 14 byte log entry
0709 13:03:08.245083 (+501207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:08.245193 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.162408 (+1917215us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b180043cde0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:03:10.165587 (+  3179us) raft_consensus.cc:1685] finished
0709 13:03:10.165693 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.165821 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":500836,"prepare.queue_time_us":21688,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":501328,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":6583,"spinlock_wait_cycles":11910400,"thread_start_us":141886,"threads_started":1,"wal-append.queue_time_us":188172}
I20260709 13:03:10.172875  6389 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.173197  1643 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:10.173504  6389 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.175346  6008 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:10.175956  6389 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:10.176265  6194 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.176862  6194 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.179497  6385 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:10.180183  6385 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.180727  6385 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.181809  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.182562  6359 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:10.183109  6359 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.183636  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:10.184641  5915 consensus_meta.cc:287] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.808s	user 0.012s	sys 0.000s
W20260709 13:03:10.184890  6335 consensus_meta.cc:287] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.772s	user 0.008s	sys 0.003s
I20260709 13:03:10.186147  6335 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:10.186686  6246 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:10.140331  6116 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:10.188016  6116 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:10.189086  6116 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:10.114351  6169 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.191380  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.191959  5604 raft_consensus.cc:3055] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:10.192494  1609 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:10.192818  6169 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:10.193434  6169 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:10.194059  1609 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:10.197038  6340 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:10.197592  6266 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.198244  6266 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.199105  6222 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:10.200737  6266 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:10.154060  6149 consensus_meta.cc:287] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.283s	user 0.015s	sys 0.000s
I20260709 13:03:10.192497  5604 raft_consensus.cc:740] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:10.203871  5604 consensus_queue.cc:260] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.204902  5604 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:10.194479  6378 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:10.192049  2302 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:10.207489  6378 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.207937  6378 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:10.208425  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:10.209365  6334 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:10.209745  6334 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.210081  6334 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:10.211236  6269 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:10.211941  6269 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:10.212926  6269 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.185253  6359 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:10.196386  6219 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:10.215270  2304 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:10.215370  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.216617  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.208737  6271 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:10.217687  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.218240  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:10.206490  6358 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.219983  2296 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:10.220683  6093 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:10.221045  6358 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:10.190623  6116 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:10.217712  6385 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:10.225306  6334 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.233596  6112 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:10.218809  2304 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:10.185115  5915 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:10.234344  6112 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.235486  1563 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:10.236652  5915 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:10.237282  5915 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.237919  5915 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.232352  6329 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:10.232594  6320 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:10.247306  6320 consensus_queue.cc:1243] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 13.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:10.247959  6320 raft_consensus.cc:3055] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:10.230134  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.248611  6320 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:10.249670  6320 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.250187  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.251129  6320 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:10.239879  6334 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:10.228375  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.254640  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.248080  5915 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:10.256698  1637 raft_consensus.cc:1240] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:10.238597  6112 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:10.153055  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.014s	sys 0.037s
I20260709 13:03:10.257664  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:10.262233  2302 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:10.275035  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.264163  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.265753  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.265900  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.266450  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.276731  1715 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:10.276978  1714 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 9: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:10.261771  6220 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:10.270213  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.270510  6374 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:10.262014  5885 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.271060  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.271302  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.270329  6226 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:10.286999  6374 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:10.270283  6297 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:03:10.289424  6374 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:10.194613  6169 raft_consensus.cc:3055] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:10.290043  6169 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:10.290524  6220 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:10.291029  6169 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.277760  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.293242  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.285212  1712 raft_consensus.cc:2393] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:10.278656  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:10.307898  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:10.281463  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:10.286198  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.308490  2302 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:10.308638  1713 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:10.286291  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:10.308913  1709 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:10.286482  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:10.287067  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.309724  1718 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:10.330025  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 190902us
W20260709 13:03:10.343509  2305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.007s	sys 0.000s
I20260709 13:03:10.262089  6334 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:10.344798  2305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.007s	sys 0.000s
I20260709 13:03:10.346148  1713 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:10.350520  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:10.351397  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 77368.7us
I20260709 13:03:10.351565  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:10.352171  1640 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:10.346787  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.349135  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:10.353080  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:10.272408  6213 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.356287  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:10.356824  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.353664  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.262287  6264 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:10.358361  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.359412  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:10.364579  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.359949  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:10.275979  6335 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:10.361656  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:10.367762  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.362092  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.302103  5915 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:10.285715  5995 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:10.372109  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:10.366817  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.368522  1714 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:03:10.368954  1713 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:10.363036  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1290) took 1507 ms. Trace:
I20260709 13:03:10.292903  6220 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:10.371917  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.374279  1709 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:03:10.373478  2305 rpcz_store.cc:276] 0709 13:03:08.856152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.902562 (+ 46410us) service_pool.cc:224] Handling call
0709 13:03:08.903043 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.976002 (+ 72959us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.976056 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.976673 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 13:03:09.459995 (+483322us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:09.647795 (+187800us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.245947 (+598152us) spinlock_profiling.cc:243] Waited 7.04 ms on lock 0x7b18003e1b20. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:03:10.350513 (+104566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.350639 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.350686 (+    47us) raft_consensus.cc:1685] finished
0709 13:03:10.362803 (+ 12117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.362975 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97495,"prepare.queue_time_us":132740,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":136035,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":98,"spinlock_wait_cycles":30728192,"thread_start_us":1192076,"threads_started":3,"wal-append.queue_time_us":692449}
I20260709 13:03:10.375857  1709 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:03:10.376479  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.377364  2305 raft_consensus.cc:2410] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 14 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:03:10.278138  2304 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:03:10.378206  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:10.378752  2305 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:10.380771  6350 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.931s	user 0.004s	sys 0.008s
I20260709 13:03:10.382043  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2735) took 1549 ms. Trace:
I20260709 13:03:10.373950  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.382290  2304 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:10.338297  5885 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.349901  6112 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:03:10.272240  6378 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.411077  1711 raft_consensus.cc:2410] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:03:10.415657  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:10.415692  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:10.425374  2269 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:10.435349  1639 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 8.
I20260709 13:03:10.425788  1718 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:10.382465  1561 rpcz_store.cc:276] 0709 13:03:08.833086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.833404 (+   318us) service_pool.cc:224] Handling call
0709 13:03:08.833769 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:08.834126 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:08.834155 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:08.834654 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:03:09.254886 (+420232us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:09.254977 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:09.255089 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.380188 (+1125099us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:10.381705 (+  1517us) raft_consensus.cc:1685] finished
0709 13:03:10.381734 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.381979 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1238,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":204,"spinlock_wait_cycles":10107136,"thread_start_us":419754,"threads_started":1,"wal-append.queue_time_us":567591}
I20260709 13:03:10.436335  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:10.437613  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:10.386443  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.409380  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.426891  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.433816  2302 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:10.440795  1714 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:10.441154  1534 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:10.423731  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:10.350682  6072 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:10.409024  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.422748  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:10.447988  1712 raft_consensus.cc:2410] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:03:10.424244  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:10.442173  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 94324.2us
I20260709 13:03:10.448408  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:10.447266  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:10.449739  1640 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:10.450160  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:10.442971  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:10.368165  6222 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:10.367767  6169 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:10.473548  1635 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:10.426079  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1239) took 1006 ms. Trace:
I20260709 13:03:10.448699  1644 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:10.475863  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:10.474694  2297 rpcz_store.cc:276] 0709 13:03:09.419276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.043504 (+624228us) service_pool.cc:224] Handling call
0709 13:03:10.064841 (+ 21337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:10.065182 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.065210 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:10.065692 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 13:03:10.152024 (+ 86332us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:10.274794 (+122770us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.345638 (+ 70844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.345714 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:10.425858 (+ 80144us) raft_consensus.cc:1685] finished
0709 13:03:10.425878 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.426022 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45959,"prepare.queue_time_us":123454,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":132841,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":23112,"thread_start_us":85645,"threads_started":1,"wal-append.queue_time_us":353685}
I20260709 13:03:10.451874  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.445201  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:10.412945  6370 log.cc:927] Time spent T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 13:03:10.419229  5944 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:10.487653  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1232) took 1070 ms. Trace:
I20260709 13:03:10.453855  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.464184  2305 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:10.467717  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:10.471897  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.471592  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.518611  2296 raft_consensus.cc:2368] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11.
I20260709 13:03:10.370819  5915 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.521854  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.473120  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:10.481031  1641 raft_consensus.cc:2368] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14.
I20260709 13:03:10.472191  1683 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:10.483016  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:10.523697  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 50700us
W20260709 13:03:10.413426  6368 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
I20260709 13:03:10.483578  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:10.535784  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 52281.6us
I20260709 13:03:10.447417  6128 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:10.501430  1609 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:10.442742  6406 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:10.471810  6272 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:10.363512  6334 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.448300  5604 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:10.472779  6375 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:10.368347  6213 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.513996  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.489971  2301 rpcz_store.cc:276] 0709 13:03:09.416833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.788107 (+371274us) service_pool.cc:224] Handling call
0709 13:03:09.788387 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.788578 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:09.914096 (+125518us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.104664 (+190568us) log.cc:844] Serialized 14 byte log entry
0709 13:03:10.384451 (+279787us) spinlock_profiling.cc:243] Waited 8.11 ms on lock 0x7b18003a6840. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb1455 00007fd7a1c99060 00007fd7a1cab716 00007fd7a1cbc287 00007fd7a1cbc208
0709 13:03:10.486582 (+102131us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:10.486645 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:10.486703 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.486796 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:10.487585 (+   789us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8584,"prepare.queue_time_us":124800,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":2382,"raft.queue_time_us":61857,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"spinlock_wait_cycles":18702848,"thread_start_us":274244,"threads_started":2,"wal-append.queue_time_us":283616}
I20260709 13:03:10.385022  6335 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.403792  6093 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:03:10.519431  1640 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:10.589015  1640 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:10.539582  6246 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:10.523674  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 56089.8us
I20260709 13:03:10.524818  1682 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:10.418504  6204 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.533953  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.486105  6379 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:10.385835  6264 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.539813  1534 leader_election.cc:400] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11.
I20260709 13:03:10.407176  5995 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.577200  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.622983  1716 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:10.471452  6072 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.623903  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.580534  1712 raft_consensus.cc:2410] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:03:10.544679  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.533501  5915 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:10.588864  1715 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:03:10.484864  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:10.523677  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51581.8us
I20260709 13:03:10.591261  2269 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:10.591707  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.593693  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 69009.4us
I20260709 13:03:10.541776  6019 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:10.541185  6384 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:10.614655  1714 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:10.593459  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:10.546147  5604 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.620976  1534 leader_election.cc:403] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:10.544495  6269 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:10.620621  6379 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:10.581152  6191 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:10.629521  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.634137  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.650060  1712 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:10.631865  6093 consensus_queue.cc:1243] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:03:10.637625  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:10.637637  1682 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:10.538522  6406 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.592686  6402 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:03:10.641218  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 47858.1us
W20260709 13:03:10.641625  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 50463.3us
I20260709 13:03:10.620752  6338 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:10.643180  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:10.775576  1710 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:10.639247  6019 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:10.648025  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 108349us
I20260709 13:03:10.651088  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.786343  1712 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:10.635684  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.655014  6384 raft_consensus.cc:697] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:10.658959  1718 raft_consensus.cc:2410] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
W20260709 13:03:10.626327  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 134899us
W20260709 13:03:10.689389  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:10.785133  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 147563us
I20260709 13:03:10.642509  5604 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:10.776798  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:10.650573  6417 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:10.796037  1682 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:10.800416  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:10.793227  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.801942  1716 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:10.811523  1718 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:10.652245  6379 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:10.658036  6397 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:03:10.819491  1716 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:03:10.805122  6151 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:10.775223  6322 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:10.830870  1716 pending_rounds.cc:85] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 5
I20260709 13:03:10.831231  1716 pending_rounds.cc:107] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:03:10.831491  1716 raft_consensus.cc:2889] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:10.832957  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:10.833961  1716 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:10.792027  6019 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:10.783927  5885 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:10.849568  1710 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:10.850672  1712 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:03:10.807152  6093 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:10.794848  6191 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:10.857419  1716 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:03:10.790259  6402 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.851202  1710 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:10.834779  5589 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:10.839327  6137 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.834666  6417 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:10.822206  6008 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
W20260709 13:03:10.874259  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 125930us
I20260709 13:03:10.836251  6151 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:10.839016  6112 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:10.902354  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:10.902879  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:10.872495  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 152362us
I20260709 13:03:10.857537  6322 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:10.883457  5995 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:10.853909  6378 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:10.961256  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:10.887831  6204 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:10.888862  6191 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:10.891112  6384 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:10.907315  6412 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 13:03:10.897500  1713 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:10.899474  6345 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:10.855523  6093 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:10.907261  6407 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
I20260709 13:03:10.902325  6151 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:11.028131  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:10.914973  6424 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:10.954154  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:10.962484  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:11.032735  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:11.033002  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 70649.5us
W20260709 13:03:11.033222  6425 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
W20260709 13:03:11.033571  6416 log.cc:927] Time spent T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.257s	user 0.000s	sys 0.000s
I20260709 13:03:10.985481  6112 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:11.034556  6415 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:03:11.035146  6204 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:10.985981  1710 raft_consensus.cc:1240] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:10.907310  6394 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.235s	user 0.001s	sys 0.001s
I20260709 13:03:10.894678  6406 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:11.049737  6035 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:11.050272  6035 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.050638  6035 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.017479   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:10.902081  6137 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.025024  6345 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:10.902455  6116 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.681s	user 0.011s	sys 0.000s
W20260709 13:03:10.902288  6320 consensus_meta.cc:287] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.651s	user 0.009s	sys 0.002s
I20260709 13:03:10.961187  6358 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:10.933002  6375 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:11.033486  2301 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:03:10.986805  6322 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.037220  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.037525  6424 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.026297  6093 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.027674  6297 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:11.021754  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.091157  1713 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:03:11.038604  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 84600.9us
I20260709 13:03:11.072623  1717 raft_consensus.cc:2410] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:03:11.023881  6213 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.074494  1683 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:11.036641  6335 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:11.079947  6419 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.383s	user 0.001s	sys 0.000s
I20260709 13:03:11.103360  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:11.115334  2305 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:03:11.096310  1532 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:11.086956  6112 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.124081  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2746) took 1360 ms. Trace:
I20260709 13:03:11.124608  1560 rpcz_store.cc:276] 0709 13:03:09.763882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.764114 (+   232us) service_pool.cc:224] Handling call
0709 13:03:09.764457 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:09.764799 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:09.764828 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:09.765297 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 13:03:10.149407 (+384110us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:10.149447 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.149515 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:11.123815 (+974300us) raft_consensus.cc:1685] finished
0709 13:03:11.123840 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.124008 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7896,"raft.queue_time_us":731,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":11621,"thread_start_us":332165,"threads_started":1,"wal-append.queue_time_us":703152}
I20260709 13:03:11.122581  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:11.074095  6375 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:11.122717  6392 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:11.132109  6392 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.096927  6297 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.079074  6424 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:11.159229  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:03:11.167932  2303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 1065) took 2187 ms (client timeout 2193 ms). Trace:
W20260709 13:03:11.168527  2303 rpcz_store.cc:269] 0709 13:03:08.981010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.981162 (+   152us) service_pool.cc:224] Handling call
0709 13:03:11.167722 (+2186560us) spinlock_profiling.cc:243] Waited 46 ms on lock 0x7b540068d3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:11.167859 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105847040}
I20260709 13:03:11.076212  6358 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:11.178998  1714 consensus_meta.cc:287] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.538s	user 0.009s	sys 0.002s
I20260709 13:03:11.179409  1714 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:03:11.184789  6316 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:11.185714  6316 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.143523  6392 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:11.178154  6112 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:11.144101  6297 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:11.178391  6272 consensus_meta.cc:287] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.634s	user 0.011s	sys 0.000s
I20260709 13:03:11.196631  6379 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.181398  6320 raft_consensus.cc:2764] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:03:11.199744  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 403790us
I20260709 13:03:11.183746  6116 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:11.208356  1682 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:11.202780  6272 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:11.217963  1682 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:11.220252  6151 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.221832  1710 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:11.222750  6280 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.203642  5604 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.223420  6280 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.225507  6280 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:11.226007  5604 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.227962  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:11.228790  6213 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:11.222765  6151 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:11.229290  6213 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:11.229724  6213 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:11.230149  6151 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:11.231508  1682 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:11.233582  6339 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:11.234280  6339 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:11.235548  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:11.204104  6264 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.238094  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:11.206524  6378 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:11.240484  1639 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:11.208159  6116 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:11.243727  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:11.247114  6272 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:11.247885  6272 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:11.252785  6019 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.235946  5589 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:11.236579  6316 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.266830  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:11.269086  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:11.271274  1642 raft_consensus.cc:2393] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 9: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:11.273834  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:11.199802  6375 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.240237  5995 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:11.257194  6116 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:11.285594  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:11.286550  1642 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:11.273053  5915 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.000s
I20260709 13:03:11.202461  6417 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.281461  5995 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.292990  5995 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:11.290728  5915 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.257714  6378 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.295310  5915 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.296892  6378 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.257175  6137 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.267977  5589 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:11.275586  5629 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.268921  6264 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:11.280287  6316 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:11.280346  6339 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.279762  6432 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:11.316588  1642 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:11.279887  6384 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:11.312974  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 131071us
I20260709 13:03:11.313197  6112 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:11.298345  6379 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.281471  5885 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:03:11.307354  5589 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:11.314642  6375 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:11.355571  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 927) took 1090 ms. Trace:
I20260709 13:03:11.359025  1711 rpcz_store.cc:276] 0709 13:03:10.264727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.270789 (+  6062us) service_pool.cc:224] Handling call
0709 13:03:11.355370 (+1084581us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b5400136ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:11.355511 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":289261824}
I20260709 13:03:11.360656  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.362074  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:11.319974  6128 consensus_meta.cc:287] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.779s	user 0.008s	sys 0.004s
I20260709 13:03:11.314944  6358 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:11.298518  6421 consensus_queue.cc:1175] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" responder_term: 13 status { last_received { term: 9 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:11.372946  6093 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.317550  6272 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:11.363924  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:11.291059  6431 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:03:11.332894  6432 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:11.388401  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:11.389272  2305 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:03:11.391703  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.410401  2305 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:11.413645  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.319360  6264 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:11.416587  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:11.417286  1711 raft_consensus.cc:2376] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13. Re-sending same reply.
I20260709 13:03:11.353665  6384 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.418337  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:11.419632  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.421799  1682 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:11.423702  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:11.332352  5629 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.424342  1711 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:11.385099  6249 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.396879  6432 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:11.398432  6149 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.348205  6339 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.415371  6128 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:11.385283  6272 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:11.457016  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 108314us
I20260709 13:03:11.452095  6112 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.523479  1532 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:11.432512  6358 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:11.465750  6436 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:03:11.446740  6338 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:11.536096  6338 raft_consensus.cc:697] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:03:11.476943  6435 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:03:11.478880  6264 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.460841  6128 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:11.546989  6439 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:03:11.436610  6443 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.120s
I20260709 13:03:11.550011  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:11.551427  6128 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.552263  6194 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.542163  6272 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:11.553874  6397 consensus_meta.cc:287] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.697s	user 0.011s	sys 0.000s
I20260709 13:03:11.554379  6397 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11.
I20260709 13:03:11.480723  6035 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.557271  6035 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.446529  6149 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.557986  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2215) took 1106 ms. Trace:
I20260709 13:03:11.558715  6093 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:11.559367  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 926) took 1294 ms (client timeout 1630 ms). Trace:
W20260709 13:03:11.559803  1717 rpcz_store.cc:269] 0709 13:03:10.264474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.265535 (+  1061us) service_pool.cc:224] Handling call
0709 13:03:11.559181 (+1293646us) spinlock_profiling.cc:243] Waited 796 ms on lock 0x7b540020ba54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:11.559309 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1830342528}
I20260709 13:03:11.561028  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.561745  1717 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:11.469681  5629 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.470140  6019 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.467746  5995 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:11.558385  1710 rpcz_store.cc:276] 0709 13:03:10.451953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.073261 (+621308us) service_pool.cc:224] Handling call
0709 13:03:11.073577 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:11.222256 (+148679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.557746 (+335490us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54003db8d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:11.557926 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340535040}
I20260709 13:03:11.563747  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:11.602344  2301 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:11.564527  1532 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:11.562803  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:11.603181  1717 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:11.533138  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2206) took 1264 ms. Trace:
I20260709 13:03:11.565869  1642 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:11.592417  6345 consensus_meta.cc:287] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.000s
I20260709 13:03:11.552282  6384 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.567260  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:11.606959  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.608610  2304 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:11.573808  6363 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.592304  6402 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:11.469583  6339 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.530710  6451 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:11.449478  5944 consensus_queue.cc:1059] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:03:11.580641  5589 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.581476  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2333) took 1170 ms. Trace:
I20260709 13:03:11.628656  1714 rpcz_store.cc:276] 0709 13:03:10.410994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.442676 (+ 31682us) service_pool.cc:224] Handling call
0709 13:03:11.581286 (+1138610us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54003386d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:11.581413 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375622528}
I20260709 13:03:11.629724  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.537426  6338 consensus_queue.cc:237] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.631312  1717 raft_consensus.cc:2410] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:03:11.632133  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:11.632642  1717 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.583634  6213 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.559808  6149 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:11.585639  6169 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.100s	user 0.013s	sys 0.000s
I20260709 13:03:11.591698  6424 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.599258  2302 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:03:11.595050  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1004) took 1341 ms. Trace:
I20260709 13:03:11.569999  2305 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:11.442426  6249 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.605230  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:11.535305  6358 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:11.609467  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.589313  6432 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.649108  1710 raft_consensus.cc:2410] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:03:11.649964  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.650034  6424 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.650542  1710 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:11.651619  6358 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.604395  1709 rpcz_store.cc:276] 0709 13:03:10.268313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.377354 (+109041us) service_pool.cc:224] Handling call
0709 13:03:10.377735 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:10.442359 (+ 64624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.442394 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:10.453818 (+ 11424us) log.cc:844] Serialized 26 byte log entry
0709 13:03:11.169799 (+715981us) spinlock_profiling.cc:243] Waited 4.78 ms on lock 0x7b180023ef60. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:03:11.355868 (+186069us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:03:11.458010 (+102142us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.525083 (+ 67073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.525215 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:11.525270 (+    55us) raft_consensus.cc:1685] finished
0709 13:03:11.532639 (+  7369us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.533045 (+   406us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124792,"prepare.queue_time_us":123889,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":2224,"raft.queue_time_us":139706,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":51389,"spinlock_wait_cycles":28397696,"thread_start_us":260850,"threads_started":2,"wal-append.queue_time_us":990262}
I20260709 13:03:11.630812  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.615201  6345 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.606748  6456 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:11.650633  6432 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.656931  6345 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.627135  1532 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:11.658226  6456 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:11.660488  6116 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.661159  1532 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:11.594189  6297 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.641424  1643 rpcz_store.cc:276] 0709 13:03:10.253648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.258217 (+  4569us) service_pool.cc:224] Handling call
0709 13:03:10.306097 (+ 47880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.640367 (+334270us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:11.089466 (+449099us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.412916 (+323450us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:11.412972 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:11.413013 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.594693 (+181680us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b5400247a54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:11.594763 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.594969 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29601,"prepare.queue_time_us":149924,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":32831,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":25113,"spinlock_wait_cycles":768284416,"thread_start_us":617708,"threads_started":3,"wal-append.queue_time_us":331572}
I20260709 13:03:11.664561  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.668083  1713 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:11.677752  1637 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:11.617501  6392 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.685508  5944 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:11.686220  5944 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.687394  6188 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:11.687914  6188 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:11.688603  5944 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.676684  6456 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:11.639964  6444 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:03:11.692068  5589 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.694000  6297 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.694892  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:11.697875  1717 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:11.694345  6116 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.699115  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:11.700990  1717 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:11.650055  5995 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.703369  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:11.705888  2303 raft_consensus.cc:2368] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13.
I20260709 13:03:11.708107  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:11.708947  1713 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:11.692122  6392 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.718998  1717 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:11.707683  5629 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.740031  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:11.740919  1717 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:11.741851  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.748989  1713 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:11.750379  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.726799  6213 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.723875  5629 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.783509  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2315) took 1506 ms. Trace:
I20260709 13:03:11.784649  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:11.784204  1712 rpcz_store.cc:276] 0709 13:03:10.276897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.808833 (+531936us) service_pool.cc:224] Handling call
0709 13:03:10.850432 (+ 41599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:10.851098 (+   666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.783209 (+932111us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b54001365d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:11.783420 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94749952}
I20260709 13:03:11.787468  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:11.779114  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:11.759366  6456 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:11.761284  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.766315  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:11.727275  6392 consensus_queue.cc:1059] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:03:11.771008  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:11.771386  2304 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:11.794977  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.758164  6188 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:11.790608  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 47906.8us
I20260709 13:03:11.791031  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:11.792018  2301 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:11.798141  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:11.798743  2297 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:11.799911  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:11.802240  2298 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:11.803107  1717 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:11.804615  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:11.806319  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.792541  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:11.795598  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:11.815482  1717 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:03:11.815579  2304 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:11.791931  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:11.816342  1712 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:11.808832  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:11.809540  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:11.813217  1609 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:11.813975  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:11.796857  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:11.818022  1642 raft_consensus.cc:2368] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14.
I20260709 13:03:11.821076  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:11.823096  2298 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:11.823845  6375 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:11.825984  6424 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:11.826265  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:11.826498  6424 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.828966  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:11.839818  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:11.840571  2303 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:11.840680  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:11.821813  6363 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:11.841410  6432 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:11.841861  6363 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:11.841975  6432 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:11.829437  6431 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:11.810216  6462 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:03:11.843953  1712 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:11.803920  6456 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.790545  6338 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:11.833916  6188 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.821594  5995 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.014s
W20260709 13:03:11.837955  6429 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.779s	user 0.003s	sys 0.005s
I20260709 13:03:11.838586  5629 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.858526  2297 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:11.856088  1609 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:11.842013  6112 raft_consensus.cc:3055] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:11.861253  1609 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:11.849592  6424 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:11.856169  6363 raft_consensus.cc:697] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:11.857089  6431 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:11.860674  6462 raft_consensus.cc:697] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:11.864705  6194 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:11.850239  6093 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:03:11.862891  6379 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:11.863799  6112 raft_consensus.cc:740] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:11.866127  6331 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:11.858625  6338 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.884404  6272 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:11.901726  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1021) took 1490 ms. Trace:
I20260709 13:03:11.905171  1644 rpcz_store.cc:276] 0709 13:03:10.411555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.487041 (+ 75486us) service_pool.cc:224] Handling call
0709 13:03:10.487456 (+   415us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:10.599106 (+111650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.599138 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:10.620421 (+ 21283us) log.cc:844] Serialized 52 byte log entry
0709 13:03:10.852657 (+232236us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:10.852701 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:10.852805 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:11.837212 (+984407us) spinlock_profiling.cc:243] Waited 8.29 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:11.896866 (+ 59654us) raft_consensus.cc:1685] finished
0709 13:03:11.896905 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.901617 (+  4712us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25733,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":17115,"spinlock_wait_cycles":19073408,"thread_start_us":312015,"threads_started":2,"wal-append.queue_time_us":348036}
I20260709 13:03:11.889688  6331 raft_consensus.cc:697] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:11.910674  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:11.889012  6432 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:11.908160  6272 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.888052  6379 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:11.924031  6472 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:03:11.925395  6470 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:03:11.910857  6456 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.910054  6112 consensus_queue.cc:260] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.942801  6112 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:11.919127  6462 consensus_queue.cc:237] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.928539  6194 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:11.908377  6363 consensus_queue.cc:237] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.947556  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1013) took 1585 ms. Trace:
I20260709 13:03:11.949903  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1231) took 2533 ms. Trace:
I20260709 13:03:11.950327  2300 rpcz_store.cc:276] 0709 13:03:09.416585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.657877 (+241292us) service_pool.cc:224] Handling call
0709 13:03:09.658280 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.015662 (+357382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.949638 (+1933976us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54004998d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:11.949833 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":615668608}
I20260709 13:03:11.947999  1638 rpcz_store.cc:276] 0709 13:03:10.362347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.362478 (+   131us) service_pool.cc:224] Handling call
0709 13:03:10.362806 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:10.370956 (+  8150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:10.370987 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:10.407781 (+ 36794us) log.cc:844] Serialized 52 byte log entry
0709 13:03:11.757130 (+1349349us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b1800400540. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:03:11.867695 (+110565us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:11.876035 (+  8340us) log.cc:844] Serialized 14 byte log entry
0709 13:03:11.883910 (+  7875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:11.883994 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:11.945364 (+ 61370us) raft_consensus.cc:1685] finished
0709 13:03:11.947319 (+  1955us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:11.947490 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3731,"prepare.queue_time_us":6542,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":7472,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":22852,"spinlock_wait_cycles":63963264,"thread_start_us":110340,"threads_started":1,"wal-append.queue_time_us":1493356}
I20260709 13:03:11.956315  6338 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.917191  6431 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:11.959352  6331 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:11.964316  6072 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:11.980296  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:11.955668  6008 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:12.009312  6379 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:11.989933  6008 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:12.021221  6008 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:12.021063  6424 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:12.021763  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.028412  1637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 1007) took 1752 ms (client timeout 1627 ms). Trace:
W20260709 13:03:12.028834  1637 rpcz_store.cc:269] 0709 13:03:10.276339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.278356 (+  2017us) service_pool.cc:224] Handling call
0709 13:03:12.028228 (+1749872us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400461ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:12.028344 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3074478336}
I20260709 13:03:12.029799  6424 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:12.032716  6325 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.033275  6325 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:12.035715  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2230) took 1173 ms. Trace:
I20260709 13:03:12.036108  1716 rpcz_store.cc:276] 0709 13:03:10.862429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.901924 (+ 39495us) service_pool.cc:224] Handling call
0709 13:03:12.035540 (+1133616us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b540033aed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:12.035660 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":863419264}
W20260709 13:03:12.024199  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.025207  6456 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.025741  6320 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:12.040055  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.041520  1566 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:12.042155  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:12.043813  6320 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:12.044324  6320 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:12.044749  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.045069  6320 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:12.045939  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:12.046344  6320 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:12.043735  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.046424  1561 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:12.052083  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:12.052110  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1294) took 3153 ms. Trace:
I20260709 13:03:12.053239  2299 rpcz_store.cc:276] 0709 13:03:08.899137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:08.899311 (+   174us) service_pool.cc:224] Handling call
0709 13:03:08.899714 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.152555 (+1252841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.051872 (+1899317us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54005ceed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:12.052036 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2880006528}
I20260709 13:03:12.055673  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:12.040481  6456 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:12.057132  1713 raft_consensus.cc:1240] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:03:12.060241  6325 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:12.070322  6334 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:12.075335  6313 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:12.075812  6313 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:12.076428  6272 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:12.075834  6456 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:12.076766  6313 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:12.077957  6313 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:12.087710  6375 raft_consensus.cc:3055] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:12.088403  6375 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:12.089520  6375 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:12.099249  6035 raft_consensus.cc:3055] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:12.099965  6035 raft_consensus.cc:740] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:12.100841  6035 consensus_queue.cc:260] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:12.101917  6035 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:12.097092  6375 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:12.108611  6249 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:12.151928  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:12.153246  1532 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:12.160497  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.161249  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.160518  1565 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:12.165227  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.166679  1560 raft_consensus.cc:2410] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:03:12.166616  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.167061  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.168527  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.171062  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.172439  1564 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:12.177361  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.178336  1565 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:03:12.177659  1532 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:12.179634  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.181164  1563 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:12.181895  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:12.182245  1561 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:03:12.185536  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.182863  1565 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:12.183837  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.186796  1563 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:12.187165  1560 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:12.187415  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:12.184754  1532 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:12.194146  1532 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:12.197829  1565 raft_consensus.cc:2410] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:03:12.199666  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.200637  1562 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:12.202157  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:12.203027  1562 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:12.205197  1566 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
W20260709 13:03:12.207167  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.216382  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:12.217607  1565 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:12.231701  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.233268  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:12.235845  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:12.236560  6116 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:12.237525  6116 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:12.236980  5915 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:12.239311  5944 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:12.240478  5944 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:12.240823  2300 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:12.242283  6226 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:12.231910  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.242841  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:12.237038  2297 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:03:12.214757  1562 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:12.250198  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.256633  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:12.258143  1713 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 16: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:12.259774  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:12.262127  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:12.263556  1709 raft_consensus.cc:2410] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:03:12.240908  6116 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:12.267432  1566 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:12.275388  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:12.285753  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 56275.3us
W20260709 13:03:12.286744  1715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 935) took 2019 ms (client timeout 1604 ms). Trace:
I20260709 13:03:12.287675  1683 leader_election.cc:400] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13.
I20260709 13:03:12.287969  6433 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:12.292199  1717 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:03:12.292869  6188 consensus_queue.cc:1059] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:03:12.294338  6188 raft_consensus.cc:3055] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:12.294875  6188 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:12.296001  1560 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:03:12.296437  6188 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:12.299081  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:12.301862  1561 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:12.302462  6271 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:12.285998  6329 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:03:12.307363  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.308176  1566 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:03:12.308717  6188 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:12.309270  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.309546  6151 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:12.310068  1566 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
W20260709 13:03:12.287259  1715 rpcz_store.cc:269] 0709 13:03:10.267415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.277493 (+ 10078us) service_pool.cc:224] Handling call
0709 13:03:12.286314 (+2008821us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b54001473d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:12.286501 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":647614720}
I20260709 13:03:12.317836  1569 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:12.319320  2269 leader_election.cc:400] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14.
W20260709 13:03:12.319494  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 377864us
I20260709 13:03:12.320777  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:12.324273  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:12.324862  1565 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:12.327168  1562 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 9: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:12.326743  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.288532  1683 leader_election.cc:403] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:03:12.247655  5944 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:12.330711  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.336128  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.337689  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.339105  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:12.339272  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.339954  1563 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:12.340664  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.342161  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:12.344246  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 130696us
I20260709 13:03:12.344947  1561 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:12.336519  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:12.344430  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.310074  6151 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:12.343876  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.346904  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.347076  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.348641  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.348723  1560 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:12.357959  1568 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:12.359162  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:12.360430  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.361208  1568 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:12.362475  6271 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:12.363570  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:12.365314  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.365911  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:12.367134  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.367275  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.368721  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:12.369593  2297 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:12.369768  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:12.370536  1638 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:12.372604  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.374169  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.375658  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:12.377982  6151 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:12.365460  6271 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:12.380849  6151 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:12.380882  1636 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:12.391181  1713 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:12.392685  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:12.393260  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.381237  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:12.393700  1636 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:12.394984  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.366401  1642 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:12.396746  2269 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:12.397884  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.365363  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:12.399712  1565 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:12.400144  6359 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:12.399712  2300 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:12.401537  1564 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:12.383721  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:12.403024  1710 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:03:12.404626  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:12.379901  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.379853  1641 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
W20260709 13:03:12.385087  1567 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.640s	user 0.009s	sys 0.032s
I20260709 13:03:12.383646  1637 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:12.396788  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:12.409874  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 122534us
I20260709 13:03:12.406428  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.414769  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:12.418673  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:12.418835  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:12.419592  1567 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.675s	user 0.010s	sys 0.033s
I20260709 13:03:12.420228  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:12.420674  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 815) took 2706 ms. Trace:
I20260709 13:03:12.421437  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:12.422039  1569 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:12.417007  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:12.423161  2269 leader_election.cc:400] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14.
I20260709 13:03:12.425882  2269 leader_election.cc:403] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:03:12.426124  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.427583  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.427599  6271 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14.
I20260709 13:03:12.428719  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:12.430799  1642 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:03:12.431874  1562 raft_consensus.cc:2410] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 13:03:12.431286  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.430557  1644 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:12.428488  1641 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:03:12.435051  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.434365  6393 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:12.439220  1644 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 16. Ops: []
I20260709 13:03:12.440798  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:12.441623  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.443537  1532 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:12.447343  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 112140us
W20260709 13:03:12.451138  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.453429  6191 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:12.421092  1567 rpcz_store.cc:276] 0709 13:03:09.713934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:09.714072 (+   138us) service_pool.cc:224] Handling call
0709 13:03:09.714401 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:09.714651 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:09.744233 (+ 29582us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.420181 (+2675948us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:12.420273 (+    92us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:12.420347 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.420434 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:12.420593 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27994,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":1938,"thread_start_us":2675494,"threads_started":1,"wal-append.queue_time_us":698843,"wal-append.run_cpu_time_us":2499,"wal-append.run_wall_time_us":1591010}
W20260709 13:03:12.456456  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 1014) took 2084 ms (client timeout 2097 ms). Trace:
I20260709 13:03:12.458264  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:12.459408  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2313) took 2185 ms. Trace:
I20260709 13:03:12.459554  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:12.459795  1718 rpcz_store.cc:276] 0709 13:03:10.273967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.795289 (+521322us) service_pool.cc:224] Handling call
0709 13:03:10.805791 (+ 10502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:10.811991 (+  6200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.409647 (+1597656us) spinlock_profiling.cc:243] Waited 5.36 ms on lock 0x7b5400147dd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:12.459219 (+ 49572us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12330368}
I20260709 13:03:12.460631  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:12.461361  1718 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:12.454632  6191 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:12.462366  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.464088  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.465726  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:12.467804  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 1019) took 2056 ms (client timeout 2038 ms). Trace:
W20260709 13:03:12.468230  1639 rpcz_store.cc:269] 0709 13:03:10.411031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.444649 (+ 33618us) service_pool.cc:224] Handling call
0709 13:03:12.467471 (+2022822us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b5400247f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:12.467633 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1740321536}
I20260709 13:03:12.469297  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.472083  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:12.472838  1640 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:12.473376  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.474684  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:12.479158  1569 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:12.479645  1713 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:12.469399  6191 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:12.481902  1569 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:12.418700  6337 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:12.485050  6325 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:12.491377  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.497277  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:12.492039  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 48619.6us
I20260709 13:03:12.486820  2269 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:12.458765  1567 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:12.493542  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:12.456861  1635 rpcz_store.cc:269] 0709 13:03:10.371723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:10.371897 (+   174us) service_pool.cc:224] Handling call
0709 13:03:12.456080 (+2084183us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400126454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:12.456251 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231579008}
I20260709 13:03:12.495851  1561 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:12.496289  1564 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:12.499734  1560 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:12.496542  1710 raft_consensus.cc:2368] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:12.474660  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2696) took 1204 ms. Trace:
I20260709 13:03:12.500214  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.500231  1640 rpcz_store.cc:276] 0709 13:03:11.270103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.270404 (+   301us) service_pool.cc:224] Handling call
0709 13:03:12.472281 (+1201877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.473215 (+   934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.474379 (+  1164us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400247f50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:12.474521 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2763471232}
I20260709 13:03:12.507678  1638 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
W20260709 13:03:12.523685  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 98091.1us
I20260709 13:03:12.518163  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.525068  1636 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:12.533974  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.539862  1560 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:12.541993  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.552580  1562 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:12.567589  1609 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:12.570113  1609 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:12.570827  1641 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:12.571053  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:12.572840  6072 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:12.574572  1683 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:12.492571  1642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60378
I20260709 13:03:12.584074  1642 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:12.591989  1642 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:12.590832  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.595181  1643 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:12.596474  6431 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:12.597802  6264 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:12.602640  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:12.593469  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 52069.8us
W20260709 13:03:12.605016  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 79136.2us
I20260709 13:03:12.587463  6072 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.588284  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:12.613571  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:12.590540  6191 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:03:12.611524  6072 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:12.631155  6385 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:12.617058  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:12.648937  2269 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:12.648938  1563 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
W20260709 13:03:12.652906  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:12.662313  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 74129.3us
I20260709 13:03:12.666473  1709 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:03:12.669942  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:12.683825  1607 leader_election.cc:400] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:12.684243  1607 leader_election.cc:403] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:03:12.688441  1640 raft_consensus.cc:2368] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13.
I20260709 13:03:12.665057  6431 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:12.687427  6385 raft_consensus.cc:697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:12.721138  6385 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:12.723655  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:12.731374  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 68129.2us
I20260709 13:03:12.710726  6320 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:12.732072  6320 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.732494  6320 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:12.698397  1681 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:12.736207  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 48890.1us
I20260709 13:03:12.736768  1709 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:03:12.705593  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 135634us
I20260709 13:03:12.703692  6431 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:12.724624  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:12.745325  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:12.746074  2268 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:12.719753  6169 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Leader election lost for term 15. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:12.752115  1682 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:12.752849  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.757916  5885 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:12.758960  6424 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:12.759799  6424 consensus_queue.cc:1243] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 10.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:03:12.759975  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 57396.1us
I20260709 13:03:12.757957  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:12.762082  1683 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:12.764240  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:12.764570  6375 consensus_meta.cc:287] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.657s	user 0.012s	sys 0.006s
I20260709 13:03:12.765295  6331 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:03:12.765661  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.786036  1566 raft_consensus.cc:2410] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:12.781384  1644 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:12.784092  1710 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:12.796275  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:12.780207  1682 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:12.782256  1681 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:12.823860  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 99320.7us
W20260709 13:03:12.769256  6493 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:03:12.779578  6328 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:12.766883  6491 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:12.820533  1717 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:12.773541  6329 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:12.845966  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 65996.2us
I20260709 13:03:12.780195  6374 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:12.855001  1640 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
W20260709 13:03:12.855640  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 240317us
W20260709 13:03:12.860066  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 77995us
I20260709 13:03:12.860512  1533 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:12.861461  6313 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:12.862051  6313 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.862517  6313 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:12.771649  6340 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 13:03:12.780470  5438 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
W20260709 13:03:12.876343  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:12.782526  5885 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:03:12.835794  6328 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.856684  6271 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:12.901444  6271 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.901930  6271 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:12.895545  2297 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:03:12.891505  1642 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:12.865479  6337 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13.
I20260709 13:03:12.871308  6433 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:12.907454  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:12.909566  2296 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:12.924136  1636 raft_consensus.cc:2368] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11.
I20260709 13:03:12.895237  6188 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:12.938957  6505 consensus_queue.cc:1059] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 13:03:12.940920  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 80616.2us
I20260709 13:03:12.941103  6375 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:12.942292  6505 raft_consensus.cc:3055] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:03:12.942773  6505 raft_consensus.cc:740] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:12.944859  6417 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:12.949898  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 94583.2us
I20260709 13:03:12.952203  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:12.955391  6337 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:12.956038  6337 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:12.956532  6337 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:12.942709  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:12.907982  6328 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:12.982239  1642 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:12.984563  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 86295.4us
W20260709 13:03:12.987663  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 239103us
I20260709 13:03:13.002148  6505 consensus_queue.cc:260] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.002444  6431 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.003605  6505 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:13.004801  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:13.006359  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:12.981433  6331 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:12.908552  6335 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 13:03:12.981881  6393 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:13.018515  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.027236  1561 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:03:13.023136  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:13.035750  6507 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:13.036707  6507 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:13.037760  1712 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:13.037961  6320 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.016798  6359 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:13.045142  6359 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.045346  1642 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:13.045584  6359 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:13.047693  5438 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:13.048135  5438 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:13.049026  5438 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:13.050278  5438 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:13.050686  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 108112us
I20260709 13:03:13.044668  6334 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:03:13.054523  6393 consensus_queue.cc:1059] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:03:13.060351  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.061152  1561 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:13.061069  6393 raft_consensus.cc:3055] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:13.061579  6393 raft_consensus.cc:740] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:13.062036  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.062310  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.063045  6393 consensus_queue.cc:260] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:13.063953  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.064388  6393 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:13.064880  1564 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:13.062037  6334 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:13.068238  6334 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:13.069258  6334 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.070876  6334 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:13.071517  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 119523us
I20260709 13:03:13.072772  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:13.073863  1562 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:13.083133  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.085602  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.012943  6072 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.088781  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.091352  1533 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:13.095135  1560 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:13.098453  1643 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:03:13.099437  6462 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:13.100244  6491 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:13.100698  6491 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:13.101073  6491 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:13.105785  6072 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:13.115468  1637 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:13.120005  6320 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:13.056133  6507 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:03:13.130688  1562 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
W20260709 13:03:13.133772  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 61145us
I20260709 13:03:13.134830  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:13.137414  6169 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:13.142354  1563 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:13.142938  1560 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:13.148583  1563 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:13.130234  6507 consensus_queue.cc:1243] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:03:13.163968  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:13.164832  1560 raft_consensus.cc:2376] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11. Re-sending same reply.
W20260709 13:03:13.166502  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 90436.6us
I20260709 13:03:13.180783  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.181699  1560 raft_consensus.cc:2376] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11. Re-sending same reply.
I20260709 13:03:13.198244  1562 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:03:13.201293  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 50177.3us
I20260709 13:03:13.203003  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.208324  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:13.208998  1563 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:13.210862  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 318206us
I20260709 13:03:13.212687  1562 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:13.219008  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:13.247802  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:13.197119  6337 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.250556  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:13.266144  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.253332  6226 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:13.272858  6339 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.273305  1564 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:13.273861  6339 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:13.279103  6454 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:13.283624  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 250070us
I20260709 13:03:13.292634  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:13.298801  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 98868.7us
I20260709 13:03:13.303030  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.293733  6072 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
W20260709 13:03:13.307427  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 93854.6us
I20260709 13:03:13.308650  1563 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
W20260709 13:03:13.312668  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.318764  1560 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:03:13.320972  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:13.323199  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:13.323774  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.325057  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.326416  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.327997  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.325165  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.329998  1568 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:13.332297  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.335282  6517 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:13.335804  6517 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:13.336229  6517 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.342036  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2339) took 1041 ms. Trace:
I20260709 13:03:13.342516  1566 rpcz_store.cc:276] 0709 13:03:12.300859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.331321 (+ 30462us) service_pool.cc:224] Handling call
0709 13:03:13.341780 (+1010459us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54003e8354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:13.341924 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1031694208}
I20260709 13:03:13.343394  1563 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:13.347479  1568 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:13.350800  1533 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:13.352607  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:13.354619  1682 leader_election.cc:400] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13.
I20260709 13:03:13.354748  6393 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:13.355674  6393 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:13.356884  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:13.358538  6433 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:13.359171  6431 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:13.359650  6454 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:13.360515  6431 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:13.361790  1533 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:13.363875  1533 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:13.364817  6225 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:13.365168  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:13.365674  6225 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:13.367127  6325 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:13.367676  6337 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:13.371367  1533 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:13.371431  6337 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:13.378595  6116 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:13.379369  1533 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:13.381541  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:13.383577  1533 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:13.385463  6517 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:13.386406  6492 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.828s	user 0.000s	sys 0.003s
I20260709 13:03:13.386938  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:13.387905  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.390889  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:13.376870  6188 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:13.395465  2297 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:13.436836  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:13.437785  1562 raft_consensus.cc:2376] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14. Re-sending same reply.
I20260709 13:03:13.374643  6328 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.447531  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:13.397773  6226 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:03:13.462303  6313 consensus_meta.cc:287] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.599s	user 0.009s	sys 0.005s
I20260709 13:03:13.462875  6313 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.366633  6359 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.468894  6359 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:13.470289  1713 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:13.447224  6188 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:13.475812  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:13.476794  1562 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.479761  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1257) took 1170 ms. Trace:
I20260709 13:03:13.480295  2302 rpcz_store.cc:276] 0709 13:03:12.309447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.309833 (+   386us) service_pool.cc:224] Handling call
0709 13:03:12.310294 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:12.310817 (+   523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:12.310853 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:12.311706 (+   853us) log.cc:844] Serialized 26 byte log entry
0709 13:03:12.433219 (+121513us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:12.665678 (+232459us) log.cc:844] Serialized 14 byte log entry
0709 13:03:12.789813 (+124135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:12.789890 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:13.385633 (+595743us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:13.474491 (+ 88858us) raft_consensus.cc:1685] finished
0709 13:03:13.479402 (+  4911us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.479570 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":594880,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":2026,"raft.queue_time_us":134646,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":20474,"spinlock_wait_cycles":43410688,"thread_start_us":382630,"threads_started":3,"wal-append.queue_time_us":239693}
I20260709 13:03:13.479890  6339 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:13.478634  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:13.503096  2300 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 13: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:13.470184  6375 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:03:13.510113  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 112894us
I20260709 13:03:13.485610  6313 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:13.523741  1715 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:03:13.512913  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 143601us
I20260709 13:03:13.528754  1681 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:13.515130  6375 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:13.532944  6431 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:03:13.545096  6431 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:13.546649  1682 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:13.548584  6191 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:13.549057  6191 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:13.549180  1682 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:13.549791  6320 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:13.492118  6491 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.525211  6151 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.559027  1562 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:13.567152  5438 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:13.567778  5438 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:13.568228  5438 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.539803  6375 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.546761  6225 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:13.574388  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 190898us
I20260709 13:03:13.577855  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:13.579413  2300 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:13.576812  1562 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:03:13.572481  6188 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:13.580657  1562 pending_rounds.cc:85] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 6
I20260709 13:03:13.571110  6313 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:03:13.585664  6522 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:03:13.549978  6191 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:13.586647  6188 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.588904  6188 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:13.575240  6151 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.593346  6151 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:13.594493  2298 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:03:13.595357  6188 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:13.596349  6188 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.595399  2305 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEARNER]: Advancing to term 16
I20260709 13:03:13.597656  1564 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:03:13.598628  6271 consensus_meta.cc:287] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.696s	user 0.003s	sys 0.007s
I20260709 13:03:13.600206  6491 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:13.597199  6225 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:13.595577  6328 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:13.602764  1682 leader_election.cc:400] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11.
I20260709 13:03:13.603858  1682 leader_election.cc:403] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:13.606148  6188 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:13.569355  6517 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:13.573390  6334 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
W20260709 13:03:13.610525  6527 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:03:13.611634  6331 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.612180  6331 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.612572  2301 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:13.613135  6507 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:13.615870  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:13.617321  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 946) took 1327 ms. Trace:
I20260709 13:03:13.617669  6264 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.617715  1712 rpcz_store.cc:276] 0709 13:03:12.290231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.460420 (+170189us) service_pool.cc:224] Handling call
0709 13:03:13.617168 (+1156748us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b5400135bd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:13.617270 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1171367168}
I20260709 13:03:13.618352  6264 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.620476  1642 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:13.621850  6264 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:13.622371  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2286) took 1252 ms. Trace:
I20260709 13:03:13.618782  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:13.624668  1638 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:13.618363  1643 pending_rounds.cc:85] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 10
I20260709 13:03:13.581393  1562 pending_rounds.cc:107] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:03:13.627499  1643 pending_rounds.cc:107] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:03:13.627628  1562 raft_consensus.cc:2889] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:13.627763  1643 raft_consensus.cc:2889] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:13.628140  2305 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:13.628316  1635 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:03:13.618741  1717 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:13.622766  1637 rpcz_store.cc:276] 0709 13:03:12.369965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.417691 (+ 47726us) service_pool.cc:224] Handling call
0709 13:03:13.622214 (+1204523us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b5400196ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:13.622320 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":842746112}
I20260709 13:03:13.631253  6507 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:03:13.635342  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 51126.2us
I20260709 13:03:13.635679  6340 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:03:13.630240  2297 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:13.640681  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2305) took 1197 ms. Trace:
I20260709 13:03:13.658973  1638 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:13.661260  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:13.657826  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:13.664865  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:13.679031  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 150362us
I20260709 13:03:13.680444  1681 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:13.615008  6331 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:13.642444  1639 rpcz_store.cc:276] 0709 13:03:12.443932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.503754 (+ 59822us) service_pool.cc:224] Handling call
0709 13:03:12.532303 (+ 28549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.575334 (+ 43031us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:03:12.593599 (+ 18265us) log.cc:844] Serialized 14 byte log entry
0709 13:03:13.639816 (+1046217us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b180045a5a0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb2ec0a 00007fd7a826e219 00007fd79ebb6a40 00007fd79ebb14e4 00007fd79ebb1455 00007fd7a56993c7 00007fd7a1cab9a7 00007fd7a1cbb0bf 00007fd7a1cbb038
0709 13:03:13.640086 (+   270us) log.cc:844] Serialized 14 byte log entry
0709 13:03:13.640292 (+   206us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:13.640351 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:13.640399 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.640475 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.640621 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19724,"prepare.queue_time_us":17917,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":38,"raft.queue_time_us":16837,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":6717,"spinlock_wait_cycles":4622720,"thread_start_us":71790,"threads_started":1,"wal-append.queue_time_us":442403}
I20260709 13:03:13.660969  6454 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:03:13.640641  6271 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.711601  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:13.678721  6359 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.719087  6339 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:13.718959  6359 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.721673  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:13.721915  6359 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:13.722479  1566 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 13: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:13.723428  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:13.724012  1566 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:13.727281  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:13.727552  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:13.728195  1564 raft_consensus.cc:2410] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:03:13.729233  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:13.730057  1639 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 13: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:13.730688  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:13.706082  6517 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:13.737335  5438 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.707484  6331 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.739622  5629 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.703706  6328 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:03:13.739859  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:13.743323  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:13.664043  6462 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:03:13.676302  6432 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:03:13.752807  2299 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:03:13.778970  1566 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
W20260709 13:03:13.779635  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 117465us
W20260709 13:03:13.780506  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 248004us
W20260709 13:03:13.753383  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 408650us
W20260709 13:03:13.796465  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:13.798348  6359 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:13.799090  6359 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.790077  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:13.801348  6359 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:13.732396  6536 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 13:03:13.814133  5438 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:13.784924  5629 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:13.819043  6329 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:13.819758  6329 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.821007  5629 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:13.822106  6329 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:13.822338  2297 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:13.794981  6517 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:03:13.833525  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 255754us
I20260709 13:03:13.836670  6329 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.837486  6329 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:13.841923  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2312) took 1395 ms. Trace:
I20260709 13:03:13.842425  1635 rpcz_store.cc:276] 0709 13:03:12.446419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.515263 (+ 68844us) service_pool.cc:224] Handling call
0709 13:03:13.623469 (+1108206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.628703 (+  5234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.841685 (+212982us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400196ed0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:13.841851 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2548028288}
I20260709 13:03:13.844388  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:13.845458  2304 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:13.855572  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:13.857239  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:13.755883  6375 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.851874  1683 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:13.786199  6331 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.906818  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:13.878382  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:13.851164  6329 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:13.911566  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 121586us
I20260709 13:03:13.910743  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:13.914673  2298 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 9.
I20260709 13:03:13.925504  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2713) took 1178 ms. Trace:
I20260709 13:03:13.911701  6335 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:13.908530  6151 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.929464  2268 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:13.935468  6507 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:13.936048  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:13.936975  6359 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:13.937484  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:13.939163  6530 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.939440  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:13.939770  6530 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:13.941236  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:13.923743  6537 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:13.944736  6269 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:13.925935  1641 rpcz_store.cc:276] 0709 13:03:12.747381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.779299 (+ 31918us) service_pool.cc:224] Handling call
0709 13:03:12.779774 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:12.780191 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:13.640185 (+859994us) log.cc:844] Serialized 14 byte log entry
0709 13:03:13.924993 (+284808us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:13.925055 (+    62us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:13.925107 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:13.925184 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:13.925440 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":859597,"prepare.queue_time_us":893907,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":63,"raft.queue_time_us":5859,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":281999,"threads_started":2,"wal-append.queue_time_us":187632}
I20260709 13:03:13.956275  6375 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:13.959532  6271 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:13.961825  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:13.963002  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:13.965381  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:13.939581  6504 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:13.981773  6340 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11.
I20260709 13:03:13.967691  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:13.982628  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 43795us
I20260709 13:03:13.982900  1642 raft_consensus.cc:2410] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:13.967547  6517 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.983642  2268 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:13.985762  6507 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:13.968159  6530 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:13.986325  1642 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:13.971077  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:13.972232  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:13.942852  6151 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:13.986231  6507 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:13.999043  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:13.971535  6269 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:14.001080  6507 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:14.014067  6513 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:03:14.014555  6151 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.015352  6513 raft_consensus.cc:3055] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:14.015754  6513 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:14.016681  6513 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.017941  6513 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:14.023242  6508 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:14.023800  6508 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:14.024197  6508 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:14.028143  6151 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.028776  6151 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:14.039374  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 74119.1us
I20260709 13:03:14.041560  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:14.052321  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 68775us
I20260709 13:03:14.062711  6552 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:14.063474  6552 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:14.063989  6552 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:14.071692  6151 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.073645  6331 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.076408  6507 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.083519  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:14.084197  2296 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:14.086628  1683 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:14.053233  6269 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:14.088124  6269 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:14.090212  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:14.093235  1683 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:14.098585  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:14.108114  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:14.106761  1683 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:14.095018  6507 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:14.116767  2268 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:14.129522  6340 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:14.130066  6340 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.130411  6340 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:14.105150  6337 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:14.101423  6328 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:14.135488  6328 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.137668  6328 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:14.138223  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 96775.4us
I20260709 13:03:14.139551  6375 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:14.140257  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:14.141095  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.142527  6551 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:14.143860  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:14.145087  6504 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:14.145610  6504 raft_consensus.cc:697] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:14.146541  6359 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:14.147035  6359 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:14.147449  6359 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:14.147776  6264 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.148272  6264 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.151069  6151 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:14.151628  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:14.151722  6151 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.152388  2303 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:03:14.152758  6191 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:14.154227  6151 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.155548  1609 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:14.168262  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:14.172287  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1337) took 1363 ms. Trace:
I20260709 13:03:14.172714  2299 rpcz_store.cc:276] 0709 13:03:12.808713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.808892 (+   179us) service_pool.cc:224] Handling call
0709 13:03:12.841998 (+ 33106us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:13.795197 (+953199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.172018 (+376821us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b5400600254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:14.172224 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2094167936}
I20260709 13:03:14.173928  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2767) took 1095 ms. Trace:
I20260709 13:03:14.146564  6504 consensus_queue.cc:237] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.174400  1569 rpcz_store.cc:276] 0709 13:03:13.078046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:13.078195 (+   149us) service_pool.cc:224] Handling call
0709 13:03:13.078616 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:13.078853 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:13.331130 (+252277us) log.cc:844] Serialized 14 byte log entry
0709 13:03:13.795781 (+464651us) log.cc:844] Serialized 14 byte log entry
0709 13:03:14.173181 (+377400us) spinlock_profiling.cc:243] Waited 7.21 ms on lock 0x7b18003157c0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb2ec0a 00007fd7a826e219 00007fd79ebb6a40 00007fd79ebb14e4 00007fd79ebb1455 00007fd7a1c99060 00007fd7a1cab716 00007fd7a1cbc287 00007fd7a1cbc208
0709 13:03:14.173309 (+   128us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:14.173366 (+    57us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:14.173450 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:14.173688 (+   238us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:14.173837 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8,"prepare.queue_time_us":287342,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":85,"spinlock_wait_cycles":16588032,"thread_start_us":614150,"threads_started":2,"wal-append.queue_time_us":475862}
I20260709 13:03:14.176898  6533 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:14.167127  6375 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:14.179277  6375 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.179693  6375 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:14.183094  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 380611us
W20260709 13:03:14.183223  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 348377us
I20260709 13:03:14.184979  6538 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.185559  6538 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:14.186025  1681 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:14.129426   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:14.187300  6325 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:14.187330  6539 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:14.188946  6539 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.189456  1463 catalog_manager.cc:5697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 7 to 8, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 8 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } }
I20260709 13:03:14.190304  6538 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:14.178599  6116 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:14.192492  6225 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.192893  6539 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.193040  6225 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.195864  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:14.197177  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:14.199990  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:14.214764  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:14.230324  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:14.231262  1569 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:14.235448  1560 raft_consensus.cc:2368] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:14.237563  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:14.238204  1560 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:14.240669  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 56376.8us
I20260709 13:03:14.241127  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:14.241930  1568 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:14.241863  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:14.242563  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:14.242835  1565 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
W20260709 13:03:14.237855  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 53640.3us
I20260709 13:03:14.243165  1567 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:14.251089  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:14.243937  6564 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:03:14.261854  1566 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: [7.4-11.6]
I20260709 13:03:14.227545  6334 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:14.185039  6550 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:03:14.236304  6329 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:14.275051  5438 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:14.227803  6264 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.268822  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:14.279592  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:14.280090  1566 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:14.280344  1561 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:03:14.271029  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:14.233773  6151 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:14.278137  5438 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.281186  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:14.227430  5885 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:03:14.283077  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:14.282310  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:14.245071  6331 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:03:14.288043  1715 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:14.295951  1569 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:14.269596  6385 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:14.269151  6517 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:03:14.307297  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 56302us
I20260709 13:03:14.308784  1710 raft_consensus.cc:2410] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:03:14.317163  1566 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:03:14.318169  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 49256.4us
I20260709 13:03:14.319553  1681 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:14.322593  1607 leader_election.cc:400] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:14.323134  1607 leader_election.cc:403] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:14.274281  6225 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.332726  5438 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.276322  5629 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:03:14.330729  6552 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:14.340571  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.341727  6543 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.115s
I20260709 13:03:14.342571  6552 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:14.335136  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:14.347429  6562 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:03:14.354154  6491 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:14.354836  6563 log.cc:927] Time spent T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 13:03:14.375067  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 55617.1us
I20260709 13:03:14.298300  6539 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:03:14.382617  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:14.391605  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:14.379323  6385 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:14.392695  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:14.394654  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:14.395462  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:14.399510  6151 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:14.393663  6340 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.400031  6151 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:14.400970  6337 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:14.401682  1710 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:14.401634  6337 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:14.402727  6573 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:14.379755  6462 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:03:14.403446  6359 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.404101  6337 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:14.405014  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 80530.1us
I20260709 13:03:14.379256  6431 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:14.406358  6431 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.408492  6431 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.409823  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:14.405792  6359 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:14.412259  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:14.405821  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:14.401312  6151 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.328869  6517 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:14.413580  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 136046us
W20260709 13:03:14.415026  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.413245  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:14.420193  2301 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:14.421023  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:14.421393  1715 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:14.421411  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:14.457118  2299 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:03:14.459949  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 137569us
I20260709 13:03:14.464550  6551 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:14.465232  6568 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:14.465994  6560 log.cc:927] Time spent T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.168s	user 0.003s	sys 0.000s
I20260709 13:03:14.465930  6568 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.468540  6568 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.472846  1560 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:14.429363  6582 raft_consensus.cc:3055] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:14.477982  1710 raft_consensus.cc:2410] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:03:14.483702  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:14.484009  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 70841.2us
I20260709 13:03:14.412281  6340 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.484786  1681 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:14.496587  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 90851.5us
I20260709 13:03:14.485329  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:14.497797  1713 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
W20260709 13:03:14.483139  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51123us
I20260709 13:03:14.459661  6151 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:14.487128  1715 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:14.491214  6508 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.500417  6508 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:14.494885  2299 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:14.476336  6359 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:14.480255  6582 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:14.515597  6574 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.512475  1683 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:14.491206  6576 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:14.502367  5629 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:14.501748  6517 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:14.506839  6329 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:14.531471  6359 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:14.540606  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:14.556562  1713 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:14.562003  6582 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:14.537024  5629 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.563125  6359 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.563791  5629 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:14.533519  6576 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.568061  6576 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:14.559321  6375 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.573096  6375 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.574156  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:14.574582  6329 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.575398  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:14.576159  1563 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:14.574272  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:14.580835  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:14.591261  1564 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 13: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:14.630708  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:14.636746  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 62687.5us
I20260709 13:03:14.637102  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:14.637782  1564 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:14.541327  6574 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:14.549997  6568 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.640071  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:14.647152  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:14.644390  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:14.647982  1642 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:14.644944  6537 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:14.649027  1563 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:14.652262  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:14.651546  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 143143us
W20260709 13:03:14.640872  6328 consensus_meta.cc:287] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.000s
I20260709 13:03:14.715024  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:14.716799  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 218539us
I20260709 13:03:14.740612  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:14.650859  6269 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:14.741137  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:14.769155  2303 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:14.783169  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:14.739213  6116 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.738734  6536 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:14.787848  6340 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:14.789978  5438 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:14.787578  6572 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:14.751597  6507 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:14.792313  2299 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:14.652139  6568 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:14.799261  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:14.797161  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 82267.4us
W20260709 13:03:14.804503  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 53741.8us
I20260709 13:03:14.748805  6375 raft_consensus.cc:476] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Not starting pre-election -- already a leader
I20260709 13:03:14.753165  6269 raft_consensus.cc:697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:14.767913  6551 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.738328  6328 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.818550  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:14.826334  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 227591us
I20260709 13:03:14.818252  1609 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:14.820604  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:14.781096  6571 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.658099  6537 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.832799  1682 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:14.831897  1532 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:03:14.993173  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 161394us
I20260709 13:03:14.863503  1463 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 8 to 10, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:14.871803  1710 raft_consensus.cc:1240] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:03:14.871008  6340 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:03:14.874881  6556 log.cc:927] Time spent T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:03:14.876119  6530 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:14.999243  6530 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:15.000083  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 49554.2us
I20260709 13:03:14.999467  6340 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:15.001631  6530 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:14.878082  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 59619.7us
I20260709 13:03:15.004055  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:15.005362  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:14.827406  6536 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:15.007053  6269 consensus_queue.cc:237] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:15.008066  6271 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:15.008762  6536 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:15.011037  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 178368us
I20260709 13:03:14.879338  6328 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.880522  6537 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.869201  6571 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:15.006577  2268 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:15.015061  6571 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:14.869498  6375 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:14.891470  1565 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:15.015684  6417 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:15.016335  6417 raft_consensus.cc:697] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:15.012656  6589 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:15.017369  6417 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.892277  6454 consensus_queue.cc:1059] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:14.892954  6576 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:15.019415  6375 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:15.021006  6576 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:15.020591  6375 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:14.800448  6116 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:15.023561  6116 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:15.023936  6454 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:15.024417  6454 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:14.837253  6551 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:15.027460  6551 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:14.893792  6432 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:14.808146  6572 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:15.028486  6584 raft_consensus.cc:3055] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:15.028888  6584 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:15.030370  6572 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:15.031672  6584 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:15.032452  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:15.033344  6538 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:14.894706  6507 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:15.035455  6538 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:14.874642  6570 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:15.036360  6570 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:15.038100  6116 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:14.932407  6580 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.380s	user 0.002s	sys 0.000s
W20260709 13:03:14.940201  6586 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.198s	user 0.002s	sys 0.000s
I20260709 13:03:15.039752  6551 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.944535  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:15.041697  2299 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:15.044094  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:15.045084  6432 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:15.045605  6432 raft_consensus.cc:697] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:03:15.046049  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 256477us
I20260709 13:03:15.046643  6432 consensus_queue.cc:237] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:15.054457  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:15.056705  2297 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:15.057040  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:15.057814  1565 raft_consensus.cc:2368] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14.
I20260709 13:03:14.946477  6591 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.946858  2302 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:14.948550  6595 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.144s	user 0.002s	sys 0.001s
I20260709 13:03:14.952071  6582 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:14.956972  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:14.957585  6583 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.206s	user 0.000s	sys 0.004s
I20260709 13:03:14.958766  6568 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:15.001904  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:14.877458  6574 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:15.006528  6359 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:15.033736  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:15.063731  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:15.073742  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:15.077502  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:15.040472  6551 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:15.038759  6570 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:14.886618  1714 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:14.918735  5629 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:15.086701  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 74498.3us
I20260709 13:03:15.082222  2297 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:15.093485  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:15.112097  1564 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:15.112609  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:15.112610  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:15.123217  1566 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:15.118863  1682 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:15.126717  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:15.102166  2299 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:15.135185  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:15.134837  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:15.138697  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:15.080135  6393 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:15.149180  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:15.151012  1682 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:15.154349  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:15.136114  1564 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 10: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:03:15.158818  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 66189.5us
W20260709 13:03:15.168228  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 88381.7us
I20260709 13:03:15.195187  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:15.193471  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:15.130609  6584 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:15.197918  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:15.198570  1639 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:15.201404  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:15.203756  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:15.206243  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:15.207137  1639 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:15.209033  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:15.209636  1639 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:03:15.115979  6151 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:15.215355  1643 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 13: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:03:15.216477  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 56589.4us
I20260709 13:03:15.134778  6570 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:15.038945  6116 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:15.148272  6566 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.223219  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:15.153038  6574 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:15.080439  6576 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.110s
I20260709 13:03:15.188549  6530 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:15.250583  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 112019us
I20260709 13:03:15.080724  6591 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:15.266105  1563 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:15.268052  1463 catalog_manager.cc:5697] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 6 to 8, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 8 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:03:15.269217  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:15.275224  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:15.277387  1607 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:15.302119  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 67799.6us
I20260709 13:03:15.277276  1710 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:15.148092  6544 consensus_queue.cc:1059] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 13:03:15.309124  1710 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:15.309777  1567 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:15.311697  1565 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:15.323268  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:15.246749  6584 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:15.214555  6507 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.121s
I20260709 13:03:15.349347  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:15.343938  2300 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:03:15.353710  1710 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:15.271126  6517 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:15.351300  1639 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:15.279951  6340 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:15.345271  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:15.269527  6335 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.283229  6576 consensus_queue.cc:1243] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 13.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:15.356406  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:15.435491  1565 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:15.359360  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:15.436676  1568 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:15.427820  1564 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:15.225455  6151 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:15.584205  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:15.585878  6613 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:03:15.495012  6462 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:15.499840  6572 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
W20260709 13:03:15.544608  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 235921us
I20260709 13:03:15.652715  1711 raft_consensus.cc:2393] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 10: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:15.434023  6340 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:15.521128  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:15.482298  6591 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:15.347661  6568 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.570851  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:15.568636  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 264572us
I20260709 13:03:15.458385  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:15.272979  6566 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:15.486552  6517 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:15.585629  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:15.490844  6584 raft_consensus.cc:697] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:03:15.584568  6606 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:03:15.439980  6335 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:15.629735  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:15.648936  6329 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:15.657582  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 380283us
I20260709 13:03:15.670815  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:15.344436  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 310783us
I20260709 13:03:15.628957  6533 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:15.649142  5629 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:15.570427  6551 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:15.721781  1718 raft_consensus.cc:2410] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:03:15.792932  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:15.793738  1718 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:03:15.722821  1712 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:03:15.755965  1713 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:15.753486  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:15.762758  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 458190us
I20260709 13:03:15.737906  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:15.799225  1639 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 8.
I20260709 13:03:15.773904  1568 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
W20260709 13:03:15.746423  6601 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:03:15.780805  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:15.798571  1683 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:15.797528  1712 pending_rounds.cc:85] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 7
I20260709 13:03:15.810461  1712 pending_rounds.cc:107] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:03:15.810771  1712 raft_consensus.cc:2889] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:15.816226  2304 raft_consensus.cc:1217] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Deduplicated request from leader. Original: 10.11->[12.12-16.14]   Dedup: 12.12->[14.13-16.14]
I20260709 13:03:15.816926  2304 pending_rounds.cc:85] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 12
I20260709 13:03:15.817246  2304 pending_rounds.cc:107] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 13:03:15.817545  2304 raft_consensus.cc:2889] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:15.662053  6116 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:15.823668  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2394) took 1240 ms. Trace:
W20260709 13:03:15.824328  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 66301.8us
I20260709 13:03:15.830885  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:15.834491  6632 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:15.835127  6632 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:15.837347  6632 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:15.837862  6632 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:15.828806  1561 rpcz_store.cc:276] 0709 13:03:14.583071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.584877 (+  1806us) service_pool.cc:224] Handling call
0709 13:03:14.628710 (+ 43833us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.629174 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.629216 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:14.637969 (+  8753us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.191746 (+553777us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:15.191788 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.191861 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:15.823399 (+631538us) raft_consensus.cc:1685] finished
0709 13:03:15.823426 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:15.823593 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":142,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":5019,"thread_start_us":418542,"threads_started":1,"wal-append.queue_time_us":593287}
I20260709 13:03:15.847604  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:15.845278  5438 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.016s	sys 0.010s
I20260709 13:03:15.807278  5629 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.801143  6635 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.710204  6568 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:15.850994  5438 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.017s	sys 0.010s
I20260709 13:03:15.754731  6329 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:15.839589  6551 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:15.889200  2269 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:15.846239  6579 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:15.907961  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 224665us
W20260709 13:03:15.912374  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 159422us
I20260709 13:03:15.842165  6116 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:15.913543  1534 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:15.853443  6375 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:15.914510  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:15.923780  1712 raft_consensus.cc:2410] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 4.
I20260709 13:03:15.923637  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:15.889303  6462 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:15.884133  6628 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:15.936822  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:15.856139  6618 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:03:15.949921  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:15.946051  1532 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:15.912385  6579 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:15.986233  1637 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:15.884259  5629 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:15.926615  6617 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:15.891898  6632 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:16.005074  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 115965us
I20260709 13:03:15.887987  6635 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:15.928580  6116 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:16.001982  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 152323us
I20260709 13:03:15.989804  6579 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:15.941886  6623 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:15.950901  6504 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:15.918416  6551 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.049273  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:16.049417  6644 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.928911  6584 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:15.941538  6643 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.116s
W20260709 13:03:16.047398  6601 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:03:16.050899  6517 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:16.051755  6116 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.055918  6589 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.056782  6635 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:15.995522  6628 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:16.057314  6605 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.057651  6566 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.059815  6335 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.062376  6578 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.087903  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2395) took 1502 ms. Trace:
I20260709 13:03:15.939162  6375 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:16.088941  6504 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.092998  6609 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:16.094295  6650 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:16.097208  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 47998.7us
I20260709 13:03:15.915751  6633 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:16.094432  6551 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.073002  6533 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:16.094463  6623 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:16.115186  6618 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:16.155340  1569 rpcz_store.cc:276] 0709 13:03:14.585441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.585592 (+   151us) service_pool.cc:224] Handling call
0709 13:03:14.585905 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.634094 (+ 48189us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:14.750249 (+116155us) log.cc:844] Serialized 14 byte log entry
0709 13:03:15.323549 (+573300us) log.cc:844] Serialized 14 byte log entry
0709 13:03:15.345789 (+ 22240us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:15.639920 (+294131us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.640168 (+   248us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:15.641342 (+  1174us) log.cc:844] Serialized 14 byte log entry
0709 13:03:16.001850 (+360508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.003857 (+  2007us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:16.004744 (+   887us) raft_consensus.cc:1685] finished
0709 13:03:16.030717 (+ 25973us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.087816 (+ 57099us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":184027,"prepare.queue_time_us":171785,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":134,"raft.queue_time_us":1165441,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":29227,"spinlock_wait_cycles":2001792,"thread_start_us":790932,"threads_started":4,"wal-append.queue_time_us":655771}
W20260709 13:03:16.158074  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.010s	sys 0.048s
W20260709 13:03:16.181856  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.010s	sys 0.048s
I20260709 13:03:16.183919  2268 leader_election.cc:400] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14.
I20260709 13:03:16.184306  2268 leader_election.cc:403] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:16.185151  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 162512us
W20260709 13:03:16.158161  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 212212us
W20260709 13:03:16.202488  6611 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.492s	user 0.000s	sys 0.002s
I20260709 13:03:16.203836  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 1125) took 1453 ms. Trace:
I20260709 13:03:16.204643  6641 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:16.205370  6641 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.204566  2301 rpcz_store.cc:276] 0709 13:03:14.750639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.751359 (+   720us) service_pool.cc:224] Handling call
0709 13:03:14.790370 (+ 39011us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:14.827958 (+ 37588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:14.827995 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:14.990858 (+162863us) log.cc:844] Serialized 26 byte log entry
0709 13:03:15.157735 (+166877us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:15.157776 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:15.157849 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:16.203563 (+1045714us) raft_consensus.cc:1685] finished
0709 13:03:16.203587 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.203747 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18786,"raft.queue_time_us":33824,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":20076,"thread_start_us":166380,"threads_started":1,"wal-append.queue_time_us":438530}
I20260709 13:03:16.165441  6650 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.203847  6624 consensus_queue.cc:1059] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:03:16.105492  6644 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:16.206068  6641 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:16.170393  6633 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:16.170064  6620 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:16.238849  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:16.244261  1569 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:16.147536  6589 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:16.158898  6605 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.197018  6628 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:16.235749  6650 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:16.197328  6623 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.215682  6643 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:16.161826  6578 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.272532  1562 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 13:03:16.253734  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 64861.3us
I20260709 13:03:16.245293  6630 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:16.307948  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:16.272976  6626 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:16.243732  6633 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:16.255110  6329 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.268460  6551 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:16.317029  1569 raft_consensus.cc:2410] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:03:16.323549  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:16.320353  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:16.310868  1711 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:16.317490  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.329337  1564 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:16.189199  6533 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.185353  6609 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:03:16.309391  6582 consensus_meta.cc:287] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.229s	user 0.015s	sys 0.000s
I20260709 13:03:16.289801  6572 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:16.344347  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:16.348551  1560 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:16.349390  1711 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:03:16.320350  6626 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:16.354908  6627 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:03:16.253989  6620 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.327559  5629 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:16.359313  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 147498us
W20260709 13:03:16.361707  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 177953us
I20260709 13:03:16.362651  2268 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:16.272169  6628 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:16.361243  1532 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:16.351224  6635 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.357542  6568 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.376446  6646 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:03:16.377305  6634 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:03:16.367030  6432 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.011s	sys 0.020s
I20260709 13:03:16.347795  6551 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:16.315828  6643 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.377537  6626 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.326583  6630 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:16.357460  6572 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:16.384016  6432 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.012s	sys 0.020s
I20260709 13:03:16.373170  6647 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:16.408223  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:16.408617  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:16.411597  2302 raft_consensus.cc:2410] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:03:16.388123  6640 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14.
I20260709 13:03:16.371943  6635 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:16.421200  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:16.397866  6375 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.426812  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:16.427538  1711 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:16.431172  6663 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:16.431346  6454 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.432086  6654 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:03:16.432503  6454 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:16.434410  6662 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:16.434983  6662 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.435475  6620 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.437237  6662 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.438037  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:16.439042  6620 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:16.439759  6620 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.440279  6620 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:16.439073  6630 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.406144  6647 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.443209  6647 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:16.443513  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:16.444254  1711 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:16.443603  6393 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.445459  6393 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:16.447639  1532 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:16.447847  6641 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:16.448457  6572 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.449535  6609 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:16.450793  6568 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:16.456377  6643 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.456511  6568 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.457338  2301 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:03:16.457504  6605 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.451128  6551 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.451802  6623 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.460004  6568 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.460388  6628 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.426723  6644 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.461076  1562 raft_consensus.cc:1240] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:16.462285  6652 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:03:16.410434  6504 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.462777  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.463042  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.463789  1562 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:16.464670  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.464752  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.465570  6151 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:16.466176  1569 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:03:16.469409  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:16.470674  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:16.470981  6641 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.472055  6551 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.472302  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 1137) took 1517 ms. Trace:
I20260709 13:03:16.472745  2300 rpcz_store.cc:276] 0709 13:03:14.954909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.091742 (+136833us) service_pool.cc:224] Handling call
0709 13:03:16.472123 (+1380381us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400168354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:16.472219 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437498624}
I20260709 13:03:16.472611  6551 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.473824  6650 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.455330  6620 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:16.477680  6622 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.693s	user 0.002s	sys 0.000s
I20260709 13:03:16.478271  6620 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.479391  6431 raft_consensus.cc:3055] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:16.479830  6431 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:16.480862  6633 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.480729  6431 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.482604  6431 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:16.483230  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:16.483821  1716 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:16.451891  6589 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:16.488440  6645 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 13:03:16.489499  6578 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.489739  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 960) took 1193 ms. Trace:
I20260709 13:03:16.490644  6533 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.490131  1714 rpcz_store.cc:276] 0709 13:03:15.296626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.305225 (+  8599us) service_pool.cc:224] Handling call
0709 13:03:15.345334 (+ 40109us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:15.672223 (+326889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.672277 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:15.685714 (+ 13437us) log.cc:844] Serialized 26 byte log entry
0709 13:03:16.001267 (+315553us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:16.001917 (+   650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.003581 (+  1664us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:16.487701 (+484120us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:16.489434 (+  1733us) raft_consensus.cc:1685] finished
0709 13:03:16.489459 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.489662 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":329068,"raft.queue_time_us":347702,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":12605,"spinlock_wait_cycles":9173888,"thread_start_us":189667,"threads_started":1,"wal-append.queue_time_us":345327}
I20260709 13:03:16.495280  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.464011  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.477716  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:16.520607  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:16.521577  1709 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:03:16.523263  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 160727us
I20260709 13:03:16.524458  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:16.526054  2268 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:16.529217  6588 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:16.480649  6551 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.531698  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:16.532461  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:16.386538  6646 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.513595  6578 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.536755  5438 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:16.537295  5438 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:16.538062  5438 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:16.540988  1681 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:16.541529  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 181147us
I20260709 13:03:16.530023  6588 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:16.551873  1716 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:16.558387  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:16.559357  6604 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.559844  6313 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:16.560125  6604 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.562770  1568 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:16.563162  1569 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:16.563978  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.557168  1532 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:16.540450  6650 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.569805  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:16.570426  1712 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:16.555601  6633 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.571632  6533 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:16.575850  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:16.551482  6623 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:16.533715  6605 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:03:16.579272  6633 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:16.580158  6375 consensus_queue.cc:1059] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:16.580791  6579 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.587796  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.545878  6335 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:03:16.617060  6650 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:16.617898  6650 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.620900  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.619287  6670 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:03:16.625895  6650 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:16.627192  6551 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:16.628253  6579 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.627900  6551 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.629438  6635 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.630388  6551 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.598546  6393 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:16.546475  6578 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.589389  6454 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:16.594429  6668 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:16.523360  6647 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:16.595389  6674 log.cc:927] Time spent T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:03:16.578688  6533 raft_consensus.cc:697] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:16.645736  6647 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.645412  6533 consensus_queue.cc:237] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:16.577641  6504 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:16.647334  6504 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.648789  6169 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:16.649533  6191 raft_consensus.cc:3055] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:16.649883  6504 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.603334  6646 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.607858  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.650171  6191 raft_consensus.cc:740] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:16.650854  1563 raft_consensus.cc:2368] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:16.614415  6633 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.651856  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:16.653524  6643 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.578244  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:16.607256  6638 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:16.610255  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:16.655344  1718 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:16.655550  2300 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:03:16.655413  6638 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.656467  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:16.657112  1718 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:16.658093  6638 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:16.608093  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:16.611788  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.659096  1569 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:16.540077  6662 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:03:16.541730  6591 consensus_queue.cc:1059] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
W20260709 13:03:16.589135  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.624711  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.661867  1568 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:16.636873  6393 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:16.638167  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:16.682241  1714 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:16.640998  1712 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:16.641022  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:16.634449  6375 raft_consensus.cc:3055] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:16.649468  6169 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.651899  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.778479  1607 leader_election.cc:400] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:16.778837  1607 leader_election.cc:403] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:16.601975  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2278) took 1326 ms. Trace:
I20260709 13:03:16.565717  6530 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:16.660362  6664 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
W20260709 13:03:16.667901  6629 log.cc:927] Time spent T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.579s	user 0.000s	sys 0.002s
I20260709 13:03:16.651980  6633 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:16.654176  6643 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:16.662670  1681 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 2.067s (SENT) [suppressed 37 similar messages]
I20260709 13:03:16.677475  6431 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:16.578831  6604 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.779459  1713 rpcz_store.cc:276] 0709 13:03:15.275162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.275293 (+   131us) service_pool.cc:224] Handling call
0709 13:03:16.601775 (+1326482us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b54003dc7d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:16.601911 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948588416}
I20260709 13:03:16.796450  1709 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
W20260709 13:03:16.800515  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 53610.8us
I20260709 13:03:16.681231  1718 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
W20260709 13:03:16.807271  6677 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:03:16.806754  6191 consensus_queue.cc:260] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.677778  6454 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.807763  6669 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:16.765453  6375 raft_consensus.cc:740] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:16.790261  6646 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:16.791586  6530 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:16.809226  1569 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 13. Ops: []
I20260709 13:03:16.811575  6578 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.765151  6647 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:16.830327  6647 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.833122  6647 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.841331  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2803) took 1584 ms. Trace:
I20260709 13:03:16.841859  1565 rpcz_store.cc:276] 0709 13:03:15.256624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.583550 (+326926us) service_pool.cc:224] Handling call
0709 13:03:15.649532 (+ 65982us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.842848 (+193316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.842886 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.842964 (+    78us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:15.843006 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.841086 (+998080us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54000a6ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:16.841141 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.841267 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444025600}
I20260709 13:03:16.843775  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2274) took 1591 ms. Trace:
I20260709 13:03:16.844199  1717 rpcz_store.cc:276] 0709 13:03:15.262033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.279681 (+ 17648us) service_pool.cc:224] Handling call
0709 13:03:15.346660 (+ 66979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.750568 (+403908us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.750619 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.750701 (+    82us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:15.750752 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:16.843552 (+1092800us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b54003381d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:16.843597 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:16.843718 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":928253696}
I20260709 13:03:16.845345  6591 raft_consensus.cc:3055] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:16.845769  6591 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:16.846774  6591 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.848294  6591 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:16.852247  1714 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:16.865373  6454 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.867502  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.826244  6191 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:16.816834  6431 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:16.869374  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 206732us
I20260709 13:03:16.828213  6530 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:16.797142  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 189730us
I20260709 13:03:16.887918  2268 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:16.889688  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:16.891834  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:16.878455  6375 consensus_queue.cc:260] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:16.896037  6375 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:16.899117  6682 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:16.828373  6646 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.912292  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:16.878834  6328 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:16.922274  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.917301  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:16.923651  2268 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:16.909324  6271 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:16.908522  6664 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:16.926721  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:16.928151  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.916260  6682 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:16.930627  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:16.932149  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.932798  1565 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
W20260709 13:03:16.934594  6678 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:03:16.924554  6328 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.933348  6393 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:16.935607  6393 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:16.936040  6393 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:16.936913  6328 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.938992  6635 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:16.933167  6650 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:03:16.940136  6646 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.947607  6643 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:16.950450  2301 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:16.952157  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:16.952318  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:16.952702  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:16.953089  2303 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:16.926968  6668 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:16.955816  6668 raft_consensus.cc:697] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:16.956828  6668 consensus_queue.cc:237] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.959321  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:16.960525  6680 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:16.961171  6680 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.952342  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:16.965253  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:16.966367  6639 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:16.967211  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:16.968086  6669 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:16.968595  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:16.969864  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:16.948415  6393 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.977015  2300 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:16.976833  2268 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:16.908962  6519 consensus_queue.cc:1059] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:03:16.925980  6662 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:16.922853  6638 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:03:16.942830  6340 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:16.980430  6662 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:16.990434  1566 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:16.995576  6662 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:17.006804  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:16.990942  6587 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:17.022657  5438 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:17.025465  5438 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:16.998889  6669 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.020308  6587 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:16.991919  6191 consensus_queue.cc:1059] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:03:17.047577  6191 raft_consensus.cc:3055] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:17.048087  6191 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:17.016409  6269 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:17.042672  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.032833  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2272) took 1781 ms. Trace:
I20260709 13:03:16.991112  6579 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:03:17.058441  1715 rpcz_store.cc:276] 0709 13:03:15.251510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.251691 (+   181us) service_pool.cc:224] Handling call
0709 13:03:15.308246 (+ 56555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:15.682402 (+374156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:15.682449 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:15.682514 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:15.682560 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.032564 (+1350004us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b5400474354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:17.032611 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:17.032757 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":859917056}
W20260709 13:03:17.080364  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 69115.8us
I20260709 13:03:17.021216  6375 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:17.085239  5438 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:17.023321  6551 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:17.007670  6271 consensus_queue.cc:1059] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:03:17.049518  6587 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:17.113215  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:17.081391  6680 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:17.085822  6431 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:17.085114  6533 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:17.049906  6669 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:17.049010  6191 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:17.135509  6685 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.211s	user 0.000s	sys 0.002s
I20260709 13:03:17.137416  6632 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:17.145406  6169 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:17.146812  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.147640  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:17.211768  1568 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:03:17.151690  6683 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:03:17.156328  6682 raft_consensus.cc:3055] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:17.157400  6679 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:03:17.165333  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:17.123054  6519 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:17.214389  6624 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:17.215463  1568 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:17.218547  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.096017  6269 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:17.230538  6191 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:17.230360  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.231699  2299 raft_consensus.cc:2393] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:17.231427  6624 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.232944  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.233744  6624 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:17.214460  6693 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:17.223481  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.128068  6393 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:17.101079  6551 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
W20260709 13:03:17.188019  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.220232  6682 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:17.228369  6519 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.253264  6682 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.254952  6519 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:17.096450  6375 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:17.266233  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:17.267230  1567 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:17.271377  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:17.171195  6533 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.259471  6269 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:17.282084  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:17.285564  2298 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:17.299520  1568 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:17.304663  1609 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:17.306030  1609 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:17.296258  6454 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:17.311910  1568 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:17.283169  6519 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:17.297034  6169 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.259503  6693 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.319516  6669 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:17.330415  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.331938  6662 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.341208  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.342317  2301 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 15: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:17.343278  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.344372  1639 raft_consensus.cc:2410] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 4.
I20260709 13:03:17.347488  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.348197  1640 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:17.348894  6664 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:17.349264  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:17.350059  1714 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:17.351195  6669 raft_consensus.cc:1064] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:03:17.351598  1715 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:17.353758  6269 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:17.354056  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:17.353672  6669 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.355144  1718 raft_consensus.cc:2368] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10.
I20260709 13:03:17.357043  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.360199  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.354339  6269 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:17.357903  1640 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:17.350713  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.364132  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:17.374545  1712 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:17.378574  2305 raft_consensus.cc:1240] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:17.366364  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:17.390093  1717 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:17.315938  6454 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:17.391129  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:17.363986  6618 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:17.353696  6642 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.316376  6519 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:17.409404  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.376847  6269 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:17.322600  6530 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.423713  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:17.424471  1715 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:17.425856  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:17.426581  1715 raft_consensus.cc:2368] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
W20260709 13:03:17.426954  6612 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 1.935s	user 0.004s	sys 0.033s
I20260709 13:03:17.427243  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:17.428455  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:17.429674  6688 log.cc:927] Time spent T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.420s	user 0.001s	sys 0.000s
I20260709 13:03:17.433648  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:17.434676  1716 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:17.432972  6642 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.436189  6533 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:17.437080  6689 log.cc:927] Time spent T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 13:03:17.438118  6642 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:17.417024  6519 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:17.439899  1718 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 13. Ops: [11.7-11.7]
I20260709 13:03:17.441138  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:17.441706  1718 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:03:17.442589  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:17.452720  1718 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:17.452911  1710 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:17.413938  6618 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:17.453801  6618 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:17.455060  6530 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:17.455297  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:17.465798  6682 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:17.466279  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.443188  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:17.467482  2301 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:17.436887  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:17.445526  6587 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.446241  1716 raft_consensus.cc:1240] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:03:17.447361  1715 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:03:17.448259  6693 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:17.449805  6454 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:17.450451  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:17.471423  2298 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:17.487181  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.429683  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:17.488744  1711 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:17.363395  6662 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:17.363768  6169 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.474951  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.490191  2300 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:17.476470  6587 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:17.477129  2299 raft_consensus.cc:2393] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:03:17.482007  6633 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.666s	user 0.004s	sys 0.007s
W20260709 13:03:17.483147  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.488286  2298 raft_consensus.cc:2393] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 13: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:17.496410  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.501410  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.504899  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.507006  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.478214  6618 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.505759  1638 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:17.515108  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.529279  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.530033  1644 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:17.531734  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.535851  2301 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:17.540519  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.540737  1637 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:17.544009  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:17.544456  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.542210  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.525697  6642 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:17.546663  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.549809  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.550498  1638 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
W20260709 13:03:17.553531  1560 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2796) took 2978 ms (client timeout 2067 ms). Trace:
W20260709 13:03:17.554070  1560 rpcz_store.cc:269] 0709 13:03:14.572610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:14.573959 (+  1349us) service_pool.cc:224] Handling call
0709 13:03:17.550487 (+2976528us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b54002aab54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:17.550585 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4074556800}
I20260709 13:03:17.524659  6633 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:17.556711  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.558660  1644 raft_consensus.cc:2410] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:03:17.559079  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.559701  1636 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:17.566272  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.572613  6519 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.574239  2302 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:03:17.581784  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.593266  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.552261  6642 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:17.607666  1644 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:17.614521  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.615511  1644 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:17.582399  6269 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.643275  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:03:17.617892  6697 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:03:17.645646  1640 raft_consensus.cc:2368] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13.
I20260709 13:03:17.650208  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:17.652132  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:17.656059  1682 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:17.658850  1682 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:17.646677  6517 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:17.661659  6578 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:17.661743  6517 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.665027  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:17.666213  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:17.666996  6570 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:17.667472  6570 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:17.667543  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:17.667907  6570 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:17.668802  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.669560  2296 raft_consensus.cc:2393] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:17.670605  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2404) took 1398 ms. Trace:
I20260709 13:03:17.671108  1561 rpcz_store.cc:276] 0709 13:03:16.272034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.272142 (+   108us) service_pool.cc:224] Handling call
0709 13:03:16.308833 (+ 36691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:16.415444 (+106611us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:16.426584 (+ 11140us) log.cc:844] Serialized 14 byte log entry
0709 13:03:16.543934 (+117350us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:16.543982 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:16.544026 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.670369 (+1126343us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54002977d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:17.670415 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:17.670525 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34562,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":97,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":6660,"spinlock_wait_cycles":244522880,"thread_start_us":115558,"threads_started":1,"wal-append.queue_time_us":241877,"wal-append.run_cpu_time_us":1123,"wal-append.run_wall_time_us":1001279}
I20260709 13:03:17.674211  6623 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:03:17.674606  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.674764  6623 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:17.676397  1534 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:17.653242  6530 raft_consensus.cc:2764] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:03:17.686440  6393 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:17.689414  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 177652us
I20260709 13:03:17.688755  1682 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:17.659632  6340 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:03:17.694600  1682 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:17.699826  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.700729  1639 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:03:17.704300  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:17.705520  1641 raft_consensus.cc:2410] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:03:17.708279  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.708578  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.675390  6618 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:17.709213  1639 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:17.712118  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.683404  6617 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:17.712744  1639 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:17.714300  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.715389  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:17.718391  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.678087  6169 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:17.719058  1644 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:17.723495  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.721573  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:17.694991  6393 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:17.725217  1638 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:17.719491  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.690333  6538 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:17.728950  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.732134  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.733096  1644 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:17.708585  6579 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:17.755893  1711 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:17.758839  2298 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
W20260709 13:03:17.766378  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.779356  6517 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:17.792243  6701 log.cc:927] Time spent T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:03:17.795192  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.797350  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.819968  1641 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
W20260709 13:03:17.799182  6700 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:03:17.718528  6530 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:17.718753  6519 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:17.731557  6393 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:17.747833  6633 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:17.807904  6679 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:03:17.751965  6579 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:17.807441  6632 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:17.817408  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:17.831336  1636 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:17.820361  1642 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
W20260709 13:03:17.826754  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 150579us
W20260709 13:03:17.830143  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:17.828354  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 133892us
I20260709 13:03:17.919324  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.920744  2304 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:03:17.910990  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:17.914804  1533 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:17.915336  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:17.902750  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.896885  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 969) took 2266 ms. Trace:
I20260709 13:03:17.927556  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:17.929210  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.934116  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.936717  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.938087  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:17.922859  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:17.940593  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.909991  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:17.941298  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:17.941622  1638 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:17.917610  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:17.925959  1609 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:17.942950  1568 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:17.941530  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.879654  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:17.943667  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:17.945338  1564 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:17.922749  1683 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:17.881870  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:17.825083  6642 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:17.946087  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:17.928607  1710 rpcz_store.cc:276] 0709 13:03:15.629999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:15.642550 (+ 12551us) service_pool.cc:224] Handling call
0709 13:03:16.603495 (+960945us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:17.453420 (+849925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:17.820493 (+367073us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54003dc7d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:17.896569 (+ 76076us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b54003dc7d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:17.896761 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4162196736}
I20260709 13:03:17.947084  1566 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:17.918100  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.903666  1636 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:17.941322  2304 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:17.949543  1533 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:17.827692  6337 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:17.973369  2302 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:17.975561  2269 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:17.963644  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:17.961987  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.978029  1641 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:17.984705  2304 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:17.991354  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:17.974526  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:17.996433  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:18.001040  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.003569  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:17.834546  6504 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:18.007864  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:17.829646  6579 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:17.853559  6530 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:17.885667  6538 raft_consensus.cc:697] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:18.024875  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:17.961385  6533 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:17.961802  6654 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:18.038455  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:18.039808  1565 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:17.874318  6633 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:17.863680  1563 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:03:18.041679  1563 raft_consensus.cc:606] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:18.042156  1563 raft_consensus.cc:614] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:17.844303  6269 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:17.951692  6340 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:18.007864  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:18.026566  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 51103.7us
I20260709 13:03:17.899221  6617 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:18.044764  6617 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:18.045449  6633 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:17.881634  6589 consensus_queue.cc:1059] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.122s
I20260709 13:03:17.962996  6619 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:18.029357  1532 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:18.050796  6633 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:18.053112  6617 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:17.878968  6517 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.137s
I20260709 13:03:18.059415  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:17.985656  6417 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.061129  5438 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:17.902278  6587 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.079885  5438 raft_consensus.cc:697] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:03:18.069128  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 123121us
I20260709 13:03:18.083794  1682 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:18.015228  6504 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:18.094522  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.095953  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:18.096787  1568 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:18.029055  6271 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:18.099828  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:18.116317  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:18.104617  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 158075us
I20260709 13:03:18.114378  1566 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
W20260709 13:03:18.120424  6710 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:03:18.120715  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:18.121461  2298 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:18.121852  6551 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:18.075788  6712 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:18.034164  6530 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:18.060303  6393 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:18.122439  6709 log.cc:927] Time spent T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:03:18.076102  6654 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:18.124271  6329 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:18.116313  5438 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:18.095079  6711 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:18.092113  6576 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:18.134950  6712 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.122591  6642 raft_consensus.cc:3055] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:18.092124  6587 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:18.155396  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 195470us
W20260709 13:03:18.154713  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 71050.4us
I20260709 13:03:18.178985  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:18.129628  1638 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
W20260709 13:03:18.157344  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 234614us
I20260709 13:03:18.092160  6417 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:18.139439  6533 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:18.168171  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 218752us
I20260709 13:03:18.183573  2305 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:18.195052  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:18.189831  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:18.195705  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:18.196030  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 245284us
I20260709 13:03:18.197350  1533 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:03:18.200672  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 171421us
I20260709 13:03:18.201905  1609 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:18.203441  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 856) took 1277 ms. Trace:
I20260709 13:03:18.202054  1533 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:18.203791  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:18.203853  1569 rpcz_store.cc:276] 0709 13:03:16.926228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.926365 (+   137us) service_pool.cc:224] Handling call
0709 13:03:16.926689 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:16.927099 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:16.927126 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:16.927572 (+   446us) log.cc:844] Serialized 26 byte log entry
0709 13:03:17.801396 (+873824us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b18004def60. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:03:18.041255 (+239859us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:18.041322 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:18.041430 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:18.198274 (+156844us) raft_consensus.cc:1685] finished
0709 13:03:18.198293 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:18.203378 (+  5085us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89,"raft.queue_time_us":982,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":3057,"spinlock_wait_cycles":78387456,"thread_start_us":239583,"threads_started":1,"wal-append.queue_time_us":1180491}
I20260709 13:03:18.207979  2299 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:18.175343  6642 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:18.212636  1682 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:18.170287  6712 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:18.200242  6716 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:18.217396  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:18.205305  6619 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:18.217751  1569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:03:18.221262  2300 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:18.223028  1569 raft_consensus.cc:606] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:18.177451  6576 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:18.223400  1569 raft_consensus.cc:614] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:18.209147  6632 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:18.252971  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 50992.7us
I20260709 13:03:18.208818  6533 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:18.253878  1533 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:18.199738  6169 consensus_queue.cc:1059] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:03:18.165498  6504 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:18.257648  1711 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:18.156924  6538 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:18.258219  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:18.222473  6716 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:18.261240  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 48698us
I20260709 13:03:18.261969  1717 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:18.263275  6721 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:18.273401  6576 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:18.263399  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 43850.4us
W20260709 13:03:18.264358  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 95556.6us
W20260709 13:03:18.274315  6718 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:03:18.254186  6632 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.286299  6375 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:18.286983  6191 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:18.288272  6530 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:18.289558  6393 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:18.290535  6169 raft_consensus.cc:3055] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:18.293427  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 89713.5us
I20260709 13:03:18.325052  1532 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:18.295850  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:18.319628  6191 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.335701  6191 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:18.328691  6728 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:03:18.327219  1463 catalog_manager.cc:5697] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 3 to 8, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 8 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:18.336830  6169 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:18.334045  6579 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:18.337822  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.337766  6169 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:18.338555  1712 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:18.339071  6169 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:18.266979  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:18.270711  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:18.327628  6722 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:18.343685  1718 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:18.300154  1682 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:18.344859  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 46017.8us
I20260709 13:03:18.345605  1682 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:18.346638  1718 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:18.302995  6711 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:18.354867  6271 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:18.355600  6591 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:18.356143  6591 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:18.357422  6714 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.353s	user 0.000s	sys 0.001s
I20260709 13:03:18.358382  6591 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:18.303377  6716 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:18.303902  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:18.271327  6726 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:18.305482  6713 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:18.305599  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:18.265880  6578 consensus_meta.cc:287] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.604s	user 0.009s	sys 0.002s
W20260709 13:03:18.375355  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.354635  6711 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:18.300546  6570 consensus_meta.cc:287] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.004s
W20260709 13:03:18.318933  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 119255us
W20260709 13:03:18.319437  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 117605us
I20260709 13:03:18.327603  1683 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:18.343777  6722 raft_consensus.cc:697] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:18.268083  1709 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:03:18.347118  1682 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:18.348475  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 303686us
W20260709 13:03:18.350044  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.350987  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:18.389612  1714 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:18.228207  6619 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:18.363318  6337 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:18.268428  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:18.352413  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 180311us
I20260709 13:03:18.350102  5438 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:18.345531  6617 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:18.318238  6504 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:18.424607  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 77702.6us
I20260709 13:03:18.425453  1682 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:18.426483  5438 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:18.311774  6633 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.312201  6632 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:18.318442  6729 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.260042  6732 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:18.385614  2299 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:18.390302  2269 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:18.391376  2304 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:18.395637  1563 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:18.398418  6579 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:18.417022  2296 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:18.380982  6578 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:18.426667  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:18.423579  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:18.271967  6642 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:18.375998  6726 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:18.430862  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:18.439811  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 113318us
I20260709 13:03:18.440095  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:18.440865  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:18.441284  1532 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:18.441617  5438 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:18.442288  5438 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:18.444594  1718 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:03:18.446135  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:18.450399  6271 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:18.439033  6633 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:18.451531  1463 catalog_manager.cc:5697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 7 to 9, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:18.423404  6617 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:18.452895  6617 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:18.453234  6623 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.766s	user 0.004s	sys 0.008s
I20260709 13:03:18.453903  6633 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:18.453691  6623 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:18.428061  6618 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:03:18.455250  6375 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:18.455749  6375 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.456127  6375 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:18.420042  6619 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:18.382301  6591 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:03:18.417153  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 89651.5us
I20260709 13:03:18.465312  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:18.442737  1607 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:18.457422  1560 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:18.470804  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:18.471012  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.458537  1682 leader_election.cc:400] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13.
I20260709 13:03:18.434484  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:18.453255  1463 catalog_manager.cc:5697] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:18.422233  6337 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.379184  6720 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:03:18.487752  5438 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:18.419479  6654 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:18.473961  1682 leader_election.cc:403] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:03:18.473476  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 137013us
I20260709 13:03:18.475376  6271 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:18.433771  6589 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:18.490307  1533 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:18.491953  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:18.499786  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:18.505255  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:18.506522  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:18.507997  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:18.508742  1565 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:18.484234  6337 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:18.383899  6570 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:18.436820  6417 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:18.434046  6732 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.438905  6728 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:18.513999  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 55639.2us
I20260709 13:03:18.518205  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:18.521067  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:18.521864  1565 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:18.511036  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:18.424189  6519 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:03:18.424410  6662 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:03:18.447968  6726 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:18.532267  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:18.533911  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 87325.6us
I20260709 13:03:18.534159  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:18.465672  6605 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:18.466166  6587 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:18.464919  6591 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:18.537834  1564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:03:18.439118  6729 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:18.551545  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.485217  6713 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:18.545586  1564 raft_consensus.cc:606] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:18.483973  6169 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:18.536994  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:18.517174  6578 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13.
I20260709 13:03:18.416375  6533 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:18.552893  6740 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:18.539279  6605 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:18.548398  6417 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:03:18.539917  6537 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.516333  6732 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:18.537418  6728 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:18.576979  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.591347  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:18.554354  6713 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.617869  6740 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:18.618764  6713 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:18.594657  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:18.601253  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 126347us
I20260709 13:03:18.623456  2268 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:18.639400  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 147551us
I20260709 13:03:18.640597  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:18.643707  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 253477us
I20260709 13:03:18.597532  1463 catalog_manager.cc:5697] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 3 to 9, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:18.645533  1560 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:18.644855  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 110287us
I20260709 13:03:18.528445  6637 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:03:18.557701  6169 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:18.656175  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:18.647639  1463 catalog_manager.cc:5697] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 6 to 11, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:18.662822  6637 raft_consensus.cc:3055] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:18.663329  6637 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:18.664244  6637 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:18.664294  1463 catalog_manager.cc:5697] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:18.665720  6637 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:18.667308  6742 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:18.667358  6519 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:18.669127  6519 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:18.670075  6519 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:18.670415  6519 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:18.670744  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.671478  6570 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:18.657869  6537 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:18.580451  6717 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.183s	user 0.001s	sys 0.001s
I20260709 13:03:18.574836  6633 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:18.673350  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:18.615247  6605 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:18.577776  6735 log.cc:927] Time spent T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:03:18.678617  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:18.658489  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 157406us
W20260709 13:03:18.686029  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 167920us
I20260709 13:03:18.666741  1463 catalog_manager.cc:5697] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:18.593335  6649 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:18.615041  6271 raft_consensus.cc:697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:18.685036  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:18.660979  6169 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:18.707468  1718 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:03:18.658759  6654 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:18.593550  6551 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.119s
I20260709 13:03:18.674654  6744 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:18.714206  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 54818.5us
I20260709 13:03:18.718437  2298 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:18.719650  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:18.737493  1566 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:18.684273  6538 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:18.721220  1463 catalog_manager.cc:5697] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 8 to 9, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 9 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:18.534583  6711 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:18.534178  6722 consensus_queue.cc:237] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:18.694330  6743 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:18.651134  6728 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:18.704728  6740 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:18.766618  6737 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:03:18.768723  6649 consensus_queue.cc:1059] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:18.764304  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:18.738008  1681 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:18.824582  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:18.824894  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 89750.7us
W20260709 13:03:18.795382  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 154910us
I20260709 13:03:18.737695  6623 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:18.694540  6633 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:18.801254  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 117296us
I20260709 13:03:18.694947  6591 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:18.714905  6618 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:03:18.737106  6745 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:18.853956  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:18.831195  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 67022.6us
I20260709 13:03:18.855195  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:18.834965  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:18.718827  6191 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:18.838574  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:18.877539  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:18.878049  1712 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:18.839921  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:18.774785  6746 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:18.833906  1463 catalog_manager.cc:5697] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 10 to 9, leader changed from <none> to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 9 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:18.738549  1532 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:18.848838  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 323662us
W20260709 13:03:18.876255  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.898456  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:18.832322  6740 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:18.719108  6519 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:18.903221  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:18.903517  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:18.904253  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:18.904927  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:18.906240  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:18.830829  6617 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:18.924083  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:18.963158  6736 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
I20260709 13:03:18.834517  6576 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:18.965726  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 130864us
W20260709 13:03:18.966667  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 228187us
I20260709 13:03:18.979192  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:18.854332  6745 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:18.972766  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 117663us
I20260709 13:03:18.971864  6733 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:18.742617  6739 consensus_queue.cc:1059] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.099s
W20260709 13:03:18.976845  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:18.978427  1463 catalog_manager.cc:5697] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:18.965811  6591 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:19.040542  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:19.041035  1712 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
W20260709 13:03:19.045918  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.057622  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.046511  1638 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:19.006748  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:18.900408  6749 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:19.049718  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:19.039891  6337 consensus_meta.cc:287] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.000s
I20260709 13:03:19.063057  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:19.064322  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:18.977715  6649 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:18.976176  6748 raft_consensus.cc:3055] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:18.977571  6758 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:18.842389  6271 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.079576  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:19.081357  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:19.055086  6720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.023s	sys 0.036s
I20260709 13:03:19.083313  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:18.983691  6740 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:19.082783  1717 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
W20260709 13:03:19.083031  1465 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45142 (request call id 350) took 29494 ms (client timeout 15000 ms). Trace:
I20260709 13:03:19.053791  6733 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:19.085860  1463 catalog_manager.cc:5697] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:19.091367  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:19.052949  6745 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:18.980365  6618 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.102s
W20260709 13:03:19.092381  6720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.026s	sys 0.036s
I20260709 13:03:19.104346  1463 catalog_manager.cc:5697] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 6 to 10, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:19.060966  6749 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:19.065658  6729 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:19.117887  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:19.075517  6537 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:19.075736  6748 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:18.981091  6538 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:19.110869  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:19.111023  1463 catalog_manager.cc:5697] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 9 to 10, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:19.093578  6623 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.126371  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:19.126492  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:19.101068  6740 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:19.146066  1717 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:19.146660  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:19.120437  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 218950us
I20260709 13:03:19.136881  6746 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.083784  6337 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.151074  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:19.130944  6729 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:19.124281  6739 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:03:19.154978  6728 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:19.155408  6752 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:03:19.121433  6749 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:19.207762  1714 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:19.117507  6747 consensus_queue.cc:1059] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.108s
W20260709 13:03:19.222896  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 175909us
I20260709 13:03:19.152916  1609 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:19.225240  1711 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:03:19.213768  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 61765.2us
W20260709 13:03:19.226514  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 73683.7us
I20260709 13:03:19.218780  1718 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:03:19.224963  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.006s	sys 0.006s
W20260709 13:03:19.229259  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.006s	sys 0.006s
I20260709 13:03:19.111529  6713 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.230232  1532 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:19.211117  6769 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:03:19.239403  2270 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.013s	sys 0.014s
W20260709 13:03:19.239774  2270 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.013s	sys 0.014s
I20260709 13:03:19.208374  6750 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:19.238794  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.007s	sys 0.020s
I20260709 13:03:19.148262  6623 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:19.240253  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.008s	sys 0.020s
W20260709 13:03:19.241991  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.220369  6633 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:19.245594  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.244590  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 536628us
I20260709 13:03:19.207444  6537 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
W20260709 13:03:19.252887  2304 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.015s	sys 0.007s
W20260709 13:03:19.094219  1465 rpcz_store.cc:269] 0709 13:02:49.242276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.249341 (+  7065us) service_pool.cc:224] Handling call
0709 13:03:18.718091 (+29468750us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:06.967071 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:06.967929 (+   858us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:06.967991 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:03:06.968016 (+    25us) tablet.cc:662] Decoding operations
0709 13:03:06.974540 (+  6524us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:06.974688 (+   148us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:06.974725 (+    37us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:03:06.977610 (+  2885us) tablet.cc:701] Row locks acquired
0709 13:03:06.977665 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:03:06.977847 (+   182us) write_op.cc:270] Start()
0709 13:03:06.978013 (+   166us) write_op.cc:276] Timestamp: P: 1783602186977776 usec, L: 0
0709 13:03:06.978069 (+    56us) op_driver.cc:348] REPLICATION: starting
0709 13:03:06.984143 (+  6074us) log.cc:844] Serialized 10002 byte log entry
0709 13:03:07.475008 (+490865us) op_driver.cc:464] REPLICATION: finished
0709 13:03:07.670418 (+195410us) write_op.cc:301] APPLY: starting
0709 13:03:07.670546 (+   128us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:08.044927 (+374381us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:08.045045 (+   118us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:10.158602 (+2113557us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b180028d500. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a19e00ec 00007fd7a19e2af7 00007fd7a19e3fb0 00007fd7a56f81bb 00007fd7a573f5d4 00007fd7a55eb8da 00007fd7a55ee3cb 00007fd7a55edfd3 00007fd7a56ece34
0709 13:03:10.986640 (+828038us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:10.987111 (+   471us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 13:03:10.987177 (+    66us) write_op.cc:312] APPLY: finished
0709 13:03:10.987712 (+   535us) log.cc:844] Serialized 220 byte log entry
0709 13:03:11.244011 (+256299us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:11.244234 (+   223us) write_op.cc:454] Released schema lock
0709 13:03:11.244316 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:11.244720 (+   404us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:11.244756 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":957031,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":188254,"apply.run_cpu_time_us":105007,"apply.run_wall_time_us":3574302,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108677,"num_ops":25,"prepare.queue_time_us":111885,"prepare.run_cpu_time_us":38691,"prepare.run_wall_time_us":179003,"raft.queue_time_us":139810,"raft.run_cpu_time_us":20391,"raft.run_wall_time_us":58540,"replication_time_us":496718,"spinlock_wait_cycles":7779584,"thread_start_us":600790,"threads_started":5,"wal-append.queue_time_us":583274,"wal-append.run_cpu_time_us":38131,"wal-append.run_wall_time_us":2471095}]]}
W20260709 13:03:19.263963  2304 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.016s	sys 0.008s
I20260709 13:03:19.234472  6191 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:19.242499  6750 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.248221  6747 consensus_queue.cc:1243] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:03:19.257851  6712 consensus_meta.cc:287] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.032s	user 0.011s	sys 0.000s
I20260709 13:03:19.210286  6746 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.238595  6734 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:19.254436  6769 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:19.230738  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 513837us
I20260709 13:03:19.280119  2269 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:19.283373  6623 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:19.272675  2300 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:19.263795  6751 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:19.259107  6767 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:19.273535  6750 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:19.293540  6617 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:19.328850  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 101628us
W20260709 13:03:19.331143  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 251715us
I20260709 13:03:19.304867  6769 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:19.335297  1682 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:19.330768  6637 consensus_meta.cc:287] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.665s	user 0.014s	sys 0.000s
W20260709 13:03:19.332355  6732 consensus_meta.cc:287] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.660s	user 0.007s	sys 0.003s
I20260709 13:03:19.230969  6748 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:19.351724  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 71693.2us
W20260709 13:03:19.352629  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 124892us
I20260709 13:03:19.334039  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:19.296900  6712 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.355731  1609 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:03:19.363024  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:19.388599  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 55774.3us
I20260709 13:03:19.389518  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:19.352906  6732 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.391175  1716 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 13. Ops: []
I20260709 13:03:19.391983  6782 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:19.392131  6732 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:19.392861  6782 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:19.393275  6782 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.393637  6782 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:19.390746  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:19.394963  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 50911.1us
I20260709 13:03:19.356685  6637 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:19.398355  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:19.398144  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:19.394865  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:19.402056  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 46391.9us
I20260709 13:03:19.407371  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:19.409495  1682 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:19.418677  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:19.335913  6775 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:03:19.395644  6617 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:19.399824  6778 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:19.409945  6632 consensus_meta.cc:287] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.965s	user 0.010s	sys 0.001s
I20260709 13:03:19.408560  6721 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:19.431324  6722 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:19.405428  6537 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:19.415902  6504 consensus_meta.cc:287] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.981s	user 0.012s	sys 0.001s
I20260709 13:03:19.357091  6734 consensus_queue.cc:1059] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:03:19.409855  6618 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:19.403434  6787 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:19.411543  6711 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:19.422075  6337 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:19.432936  6579 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:19.462267  2303 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:19.434350  6271 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:19.455367  6537 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:19.455071  6722 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.441923  6788 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:19.460841  6375 consensus_meta.cc:287] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.983s	user 0.011s	sys 0.002s
I20260709 13:03:19.413362  6191 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:03:19.441860  6617 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.511133  6747 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:19.511472  6191 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:19.511756  6747 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:19.448163  6721 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:19.512931  6721 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:19.519601  6783 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:03:19.519703  6587 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:19.520445  6748 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:19.507664  6537 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:19.451372  6778 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.525800  6778 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:19.526516  6788 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.491600  6504 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:19.527824  6716 consensus_meta.cc:287] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.146s	user 0.012s	sys 0.000s
I20260709 13:03:19.528343  6716 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:19.529136  6788 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:19.494115  6726 consensus_meta.cc:287] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.940s	user 0.008s	sys 0.004s
I20260709 13:03:19.499308  6519 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:19.502442  6759 raft_consensus.cc:3055] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:19.504168  6633 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:19.542184  2303 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:19.451105  6632 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:19.488490  6784 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:19.541136  6649 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:19.520470  6587 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:19.530490  6713 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:19.533157  6720 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:19.541342  6759 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:19.486696  6711 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.527871  6504 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.557035  6716 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:19.531078  6375 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:19.556303  6788 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:19.571769  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 164583us
I20260709 13:03:19.567768  6746 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:19.553092  6726 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.587435  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:19.591657  1709 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:19.595518  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:19.596287  1711 raft_consensus.cc:2368] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:19.559895  6740 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:19.581741  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 170465us
I20260709 13:03:19.598495  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:19.599241  1717 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:03:19.598996  6587 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.599623  6375 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:19.600157  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:19.600317  6587 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:19.600735  1718 raft_consensus.cc:2376] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13. Re-sending same reply.
I20260709 13:03:19.601583  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:19.602196  1714 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:19.604373  6721 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:19.605242  1643 raft_consensus.cc:2368] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13.
I20260709 13:03:19.609126  6504 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:19.616369  6711 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:19.617398  6793 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:03:19.559876  6784 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:19.620220  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.620474  6726 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:19.620857  6759 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.622349  6759 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:19.623077  6775 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:19.623607  6775 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:19.624050  6619 consensus_meta.cc:287] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.138s	user 0.008s	sys 0.004s
I20260709 13:03:19.624660  6619 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.625978  6775 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:19.591214  6788 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:19.627779  6744 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.854s	user 0.006s	sys 0.005s
I20260709 13:03:19.628199  6744 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:19.628950  1562 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:19.629032  6788 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:19.620505  6784 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:19.614693  6269 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:19.630483  6749 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:19.631726  6269 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:19.632596  6269 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.633354  6749 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:19.634025  6269 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:19.616371  6169 consensus_meta.cc:287] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.893s	user 0.012s	sys 0.000s
I20260709 13:03:19.634670  6169 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:19.626029  6785 log.cc:927] Time spent T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:03:19.636826  6169 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:19.638352  6605 consensus_meta.cc:287] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.933s	user 0.005s	sys 0.005s
I20260709 13:03:19.604176  6733 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.650619  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:19.644220  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:19.651511  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.652697  1643 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:19.645411  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:19.661875  1642 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:19.645684  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:19.645720  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:19.664752  1638 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:19.647116  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:19.642582  1682 leader_election.cc:400] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13.
I20260709 13:03:19.644526  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:19.665959  1682 leader_election.cc:403] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:03:19.665982  1636 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:19.666267  1641 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:19.672187  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:19.672812  1712 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:19.682003  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 84497.3us
I20260709 13:03:19.686025  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:19.686713  2305 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:03:19.687727  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:19.688309  2297 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:19.685400  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:19.689277  2302 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:19.606772  6721 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:19.691007  1640 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:19.691941  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:19.692255  1644 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:03:19.692562  1718 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:19.638860  6538 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:19.692713  1644 pending_rounds.cc:85] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 6
I20260709 13:03:19.640357  6632 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:19.694746  2305 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:19.626823  6619 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:19.697992  1712 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:19.695356  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:19.698655  1644 pending_rounds.cc:107] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:03:19.705950  1644 raft_consensus.cc:2889] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:19.705744  1640 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:19.680449  6719 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:19.708901  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:19.579239  6720 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.710795  1609 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:19.712203  1641 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:19.716632  1532 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:19.717145  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:19.719660  2301 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:19.727806  6794 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:19.728402  6794 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:19.728612  6740 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:19.728766  6794 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:19.731926  6801 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:03:19.745288  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.751753  6533 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:19.638834  6605 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.703904  6551 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:19.769899  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:19.697835  6734 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
W20260709 13:03:19.766054  6779 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:03:19.745018  6537 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.810391  6537 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:19.765693  6533 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:19.835886  6605 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:19.835793  6533 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:03:19.836627  6755 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.870s	user 0.003s	sys 0.000s
W20260709 13:03:19.837935  6750 consensus_meta.cc:287] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.501s	user 0.000s	sys 0.009s
I20260709 13:03:19.838095  6642 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:19.841241  6745 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.722s	user 0.000s	sys 0.010s
I20260709 13:03:19.841687  6745 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.838342  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1416) took 1283 ms. Trace:
W20260709 13:03:19.844229  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 120122us
I20260709 13:03:19.849720  1718 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:19.850538  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:19.853485  1638 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:19.855129  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:19.856225  2296 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:03:19.862691  2297 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:19.865342  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:19.844967  2304 rpcz_store.cc:276] 0709 13:03:18.554964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.555444 (+   480us) service_pool.cc:224] Handling call
0709 13:03:18.555767 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:18.556172 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:18.556204 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:18.581305 (+ 25101us) log.cc:844] Serialized 26 byte log entry
0709 13:03:18.675373 (+ 94068us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:18.711223 (+ 35850us) log.cc:844] Serialized 14 byte log entry
0709 13:03:19.265387 (+554164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:19.265628 (+   241us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:19.836059 (+570431us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:19.838034 (+  1975us) raft_consensus.cc:1685] finished
0709 13:03:19.838152 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:19.838286 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6278,"prepare.queue_time_us":38442,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":1745,"raft.queue_time_us":425762,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":23855,"spinlock_wait_cycles":7125632,"thread_start_us":624431,"threads_started":2,"wal-append.queue_time_us":207291}
I20260709 13:03:19.838392  6750 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.774641  6551 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:19.880865  6782 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:19.922108  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.051s	sys 0.088s
I20260709 13:03:20.000224  2305 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 13:03:20.083913  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.057s	sys 0.089s
W20260709 13:03:20.084471  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 674901us
I20260709 13:03:20.000530  1713 raft_consensus.cc:2368] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12.
I20260709 13:03:20.003209  1609 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:20.000051  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:20.004407  6417 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:20.004441  6570 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.005292  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.008642  2301 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
W20260709 13:03:20.112248  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 247387us
I20260709 13:03:20.114710  1609 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:20.114888  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.115367  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:20.117313  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.119699  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:20.121206  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.123538  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.124377  1644 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 12: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:20.126190  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:20.126808  1709 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
W20260709 13:03:20.127547  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.127938  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:20.130326  2304 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.131716  1636 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:20.133111  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.005833  6723 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:20.008494  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57836 (request call id 24) took 1485 ms. Trace:
I20260709 13:03:20.006088  6759 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:20.106591  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:20.110981  1639 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 14: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:20.140455  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.142544  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:20.146598  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:20.150528  1709 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 12: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:20.012863  2300 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:20.147149  1564 rpcz_store.cc:276] 0709 13:03:18.520298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.520422 (+   124us) service_pool.cc:224] Handling call
0709 13:03:20.005420 (+1484998us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b5400277ca0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c1e520 00007fd7a1c9b652 00007fd7a6c6960a 00007fd7a01ef7d4 00007fd7a01ef762 00007fd7a01ef691 00007fd7a01ef60c 00007fd7a01ee8c2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:20.005685 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1339805952}
W20260709 13:03:20.155983  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.165197  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:20.165401  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:20.146588  6723 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:20.167295  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:20.171087  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.172731  1718 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:20.173869  6662 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:20.175099  6504 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:20.176223  2302 raft_consensus.cc:2368] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15.
I20260709 13:03:20.176348  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.176966  2301 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:20.183718  2305 raft_consensus.cc:2393] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:20.186306  2299 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
W20260709 13:03:20.192741  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 185118us
I20260709 13:03:20.194008  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:20.213634  2302 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:20.121636  6719 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13.
W20260709 13:03:20.211805  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 74686.5us
I20260709 13:03:20.222298  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:20.222729  2268 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:20.223932  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:20.225106  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:20.140846  6734 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:20.215648  1609 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:20.231544  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.235000  1641 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:20.174546  6662 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.237243  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:20.188150  6800 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:20.115754  6417 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.213790  6654 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.201834  6742 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:20.203104  6777 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:20.123852  6570 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.269817  1712 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:20.221494  6797 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.226114  6744 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:20.234999  6537 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:20.175838  6504 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.194360  6759 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.250797  6767 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:20.250488  6519 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:20.242830  6649 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:20.249369  6619 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:20.249925  6734 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 15 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:20.277855  6798 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:20.221324  6538 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:03:20.302320  6749 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:20.257033  6804 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:03:20.320746  6519 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.379046  6519 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:20.385989  6720 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.387373  6734 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.303097  6797 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.388885  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:20.389793  2302 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:20.354552  6798 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:20.392032  6798 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.392656  6619 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:20.398358  6570 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.287575  6742 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.404196  6750 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.404757  6504 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:20.322712  6745 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:20.287703  6805 consensus_queue.cc:1059] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:03:20.290621  6777 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.282467  6654 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.259002  6723 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:20.363360  6794 consensus_meta.cc:287] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.634s	user 0.011s	sys 0.004s
I20260709 13:03:20.402631  6742 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:20.432114  6797 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:20.414301  6756 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:03:20.437242  6806 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 13:03:20.453518  2304 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
W20260709 13:03:20.449553  6642 consensus_meta.cc:287] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.604s	user 0.015s	sys 0.001s
W20260709 13:03:20.450934  6587 consensus_meta.cc:287] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.850s	user 0.010s	sys 0.002s
W20260709 13:03:20.456614  6269 consensus_meta.cc:287] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.822s	user 0.012s	sys 0.003s
I20260709 13:03:20.434491  6794 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.477861  6794 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.460469  6538 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:20.479252  1636 raft_consensus.cc:2393] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 11: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:20.478798  6538 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.480975  6662 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:20.481802  6538 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.482123  6759 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.472570  6770 raft_consensus.cc:3055] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:20.482764  6770 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:20.483950  6654 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:20.486238  6777 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:20.488138  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:20.455847  6756 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.460847  6551 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.468611  6417 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:20.478052  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.504379  1710 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:20.500600  1643 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
W20260709 13:03:20.495698  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.310111  6744 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:20.496511  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.453626  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:20.498113  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 260976us
I20260709 13:03:20.483909  6770 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.519740  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:20.521394  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:20.526597  1607 leader_election.cc:400] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:20.501474  6642 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:20.523151  2296 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:20.516853  6719 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:20.517042  6775 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.467795  6519 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.536722  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:20.524219  6794 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:20.527251  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:20.531147  6809 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:20.533999  6770 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.557433  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:20.559170  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 334768us
I20260709 13:03:20.566642  1534 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:20.567813  6533 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:20.569334  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:20.570429  6533 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:20.574555  1534 leader_election.cc:400] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15.
I20260709 13:03:20.576505  1534 leader_election.cc:403] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:03:20.554301  6809 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.577217  6533 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.578787  6809 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.564815  6800 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:20.579885  6519 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:20.580556  6800 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.587508  6720 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:20.559788  6749 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:20.591264  6749 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:20.591715  6749 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.556026  6756 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.595942  6744 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:20.596458  1534 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:20.597741  6744 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:20.598376  6744 raft_consensus.cc:697] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:20.550519  6719 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.599615  6744 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.600656  6719 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.550817  6775 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.604770  6775 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:20.607348  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 50038.5us
I20260709 13:03:20.608126  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:20.549111  6642 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.613034  6794 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:20.613546  6642 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.614264  6619 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:20.616917  1635 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:20.620389  2302 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:20.623631  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:20.624531  1635 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.613914  6809 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:20.614735  6619 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.613598  6794 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:20.621788  6533 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:20.642587  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.634330  6720 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15.
I20260709 13:03:20.638652  6619 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:20.632125  6551 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
W20260709 13:03:20.649386  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:20.681300  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.682169  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.682597  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.682945  2304 raft_consensus.cc:2393] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:20.673293  6823 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.694973  1635 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:20.696480  6830 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:20.697471  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1424) took 1303 ms. Trace:
I20260709 13:03:20.673447  6720 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:20.698274  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 986) took 2435 ms. Trace:
I20260709 13:03:20.703091  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:20.702421  2303 rpcz_store.cc:276] 0709 13:03:19.394391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.394563 (+   172us) service_pool.cc:224] Handling call
0709 13:03:20.697282 (+1302719us) spinlock_profiling.cc:243] Waited 62.6 ms on lock 0x7b5400661554. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:20.697414 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143930624}
I20260709 13:03:20.704445  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:20.703148  1716 rpcz_store.cc:276] 0709 13:03:18.262909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.268541 (+  5632us) service_pool.cc:224] Handling call
0709 13:03:18.268794 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.391603 (+1122809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:20.698041 (+1306438us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001c1454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:20.698206 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2581004672}
I20260709 13:03:20.677430  6820 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:20.706048  1635 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:20.694378  6739 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:20.693954  6756 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:20.715991  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 186398us
I20260709 13:03:20.716830  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.718142  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:20.705951  6809 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:20.718689  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:20.721957  1683 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:20.722131  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:20.722957  1635 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.723021  1683 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:20.726034  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.701129  6772 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:03:20.727921  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.728277  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.729889  2304 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 14: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:20.679571  6794 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.705880  6720 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.738317  1635 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:20.727614  6719 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:20.741787  6720 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.742240  6719 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:20.725674  6530 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:20.743474  6530 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.743165  6719 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.743863  6530 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:20.747855  6789 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 1.288s	user 0.025s	sys 0.000s
I20260709 13:03:20.748688  6749 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.751302  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.753180  6617 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.753639   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:03:20.753876  6617 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:20.754815  6822 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:20.755892   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:20.756731  6712 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.727450  6775 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:20.703604  6823 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.755415  6822 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.756760  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:20.711798  6820 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:20.780419  1566 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:20.739748  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.761880  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:20.787319  6820 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:20.767105  6823 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.775504  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.792037  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.792191  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:20.771457  6538 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.794214  6538 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:20.796886  6538 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:20.796978  6742 raft_consensus.cc:3055] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:20.771287  1564 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:03:20.797402  6742 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:20.778780  1569 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.798346  6742 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.720454  6756 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.771267  6775 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:20.805078  6775 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.805704  6756 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.754374  6617 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:20.806870  1569 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:03:20.808130  6617 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:20.808312  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.809619  6742 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:20.810683  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:20.811273  6837 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:03:20.811995  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.812518  6749 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.813879  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2778) took 1170 ms. Trace:
I20260709 13:03:20.814266  1639 rpcz_store.cc:276] 0709 13:03:19.643843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.643956 (+   113us) service_pool.cc:224] Handling call
0709 13:03:20.813692 (+1169736us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b54000d16d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:20.813823 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041397632}
I20260709 13:03:20.815482  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.816542  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.819208  2269 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:20.819428  6533 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.820559  2269 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:20.821782  6533 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.821985  6788 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:20.822415  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.822763  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:20.822692  6788 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:20.718233  6739 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.823370  2298 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:20.825625  6739 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.826618  6712 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:20.821789  2269 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:20.801918  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.828737  2269 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:20.829319  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:20.830056  1568 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:20.830605  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:20.799654  6530 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.832012  6784 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:20.835639  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.836272  1636 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:20.837080  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.837724  1636 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 13: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:20.838778  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:20.839535  1641 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:03:20.840142  2298 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:20.841853  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:20.832103  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:20.828014  6337 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:20.829434  6835 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:20.827271  6712 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:20.843434  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:20.851660  1565 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:20.845681  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.853039  2269 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:20.854624  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.829909  6642 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:20.846378  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:20.833977  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.858639  1638 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:20.834688  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:20.848094  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:20.847517  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:20.859661  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:20.760617  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:20.849440  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:20.835228  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:20.861168  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:20.823235  1637 raft_consensus.cc:2393] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 15: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:20.853776  1636 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:20.855806  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:20.830643  6822 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:20.859333  1569 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:20.830935  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:20.861760  1717 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:20.862149  1642 raft_consensus.cc:2393] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 11: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:20.813148  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.867874  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:20.868714  1560 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.832185  6788 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:20.884989  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:20.887987  1637 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:03:20.889503  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:20.890506  1566 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:03:20.891636  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:20.863485  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:20.876392  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:20.894016  1718 raft_consensus.cc:2393] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:20.894862  1636 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:20.895215  1716 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:20.896750  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:20.899292  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:20.899798  1635 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:20.900082  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:20.895925  1637 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:20.865953  6533 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:20.910292  1644 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:20.904815  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:20.911356  1716 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:20.910974  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:20.916153  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:20.865545  6829 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:20.916774  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:20.919005  1564 raft_consensus.cc:2393] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:20.920118  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:20.920318  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:20.921109  1567 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:20.921087  1718 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.922582  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:20.923997  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:20.926220  1569 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:03:20.927628  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:20.928871  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:20.929507  1569 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:20.901103  6668 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:03:20.931408  6712 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:20.931550  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:20.932206  2303 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:20.851109  6337 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.886718  6788 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:20.945664  2303 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:20.904345  1560 raft_consensus.cc:2410] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:03:20.953758  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:20.955054  1709 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:20.956122  1717 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:20.957666  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:20.959538  1717 raft_consensus.cc:2393] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:20.865902  6642 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:20.971952  1569 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:20.915210  6530 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:21.063987  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:21.087916  1635 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:21.064404  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:20.915210  6753 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.091693  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:21.092368  1637 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:20.918643  6829 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.094404  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:21.095695  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:21.096441  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:21.095417  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:21.105151  1709 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:20.935343  6533 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:20.942713  6788 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:20.911717  6619 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:21.138447  6844 log.cc:927] Time spent T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.213s	user 0.000s	sys 0.000s
I20260709 13:03:21.095778  6712 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.116971  1714 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:20.918524  6827 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.177s
I20260709 13:03:21.093422  1715 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:21.135039  1717 raft_consensus.cc:2393] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:20.965564  6668 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.164s
I20260709 13:03:21.139830  1642 raft_consensus.cc:1240] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:03:20.857666  6835 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.086614  1718 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:21.148977  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:21.148881  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 240304us
I20260709 13:03:21.097388  6829 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:21.148203  1561 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
W20260709 13:03:21.157433  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.172044  1718 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:21.138739  6619 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:21.204496  6533 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.107236  6753 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.209159  6753 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:21.178983  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:21.152890  6712 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:21.193116  1710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2300) took 2921 ms (client timeout 1929 ms). Trace:
W20260709 13:03:21.255328  1710 rpcz_store.cc:269] 0709 13:03:18.271457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.305328 (+ 33871us) service_pool.cc:224] Handling call
0709 13:03:21.192773 (+2887445us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b54004051d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:21.192958 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4875829632}
I20260709 13:03:21.200978  1717 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:21.210449  6794 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:21.212234  6337 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:21.218856  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:21.262231  1569 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:21.223124  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:21.263357  1716 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:21.232470  6668 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:21.228318  1567 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:21.229338  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:21.177950  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:21.266968  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 312993us
I20260709 13:03:21.267242  1635 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:21.229873  1711 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:03:21.267762  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:21.201045  6642 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:21.239511  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:21.244576  6637 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:21.246289  1681 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:21.250994  6845 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
W20260709 13:03:21.291559  6852 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:03:21.230419  6827 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:21.254511  6848 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:03:21.225006  2304 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEARNER]: Advancing to term 17
I20260709 13:03:21.292513  6827 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.224869  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:21.296332  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2447) took 2602 ms. Trace:
I20260709 13:03:21.296852  1562 rpcz_store.cc:276] 0709 13:03:18.693835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.701838 (+  8003us) service_pool.cc:224] Handling call
0709 13:03:18.736310 (+ 34472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:19.629353 (+893043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.296028 (+1666675us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b54002aa654. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:21.296262 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2052614016}
I20260709 13:03:21.297389  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:21.298638  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:21.299383  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:21.299654  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:21.299748  6782 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:21.300221  2301 raft_consensus.cc:2410] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:03:21.300307  6712 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:21.300665  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:21.301116  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:21.301456  1709 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 13: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:21.301810  2301 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:21.301929  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:21.270238  1564 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:03:21.271623  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:21.274348  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:21.305260  1638 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:21.305938  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2364) took 1061 ms. Trace:
I20260709 13:03:21.269160  6745 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:21.259377  6754 consensus_queue.cc:1059] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:03:21.309026  2304 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEARNER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:21.306288  1638 rpcz_store.cc:276] 0709 13:03:20.244598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.859480 (+614882us) service_pool.cc:224] Handling call
0709 13:03:21.304729 (+445249us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c7afc0 00007fd7a6c65929 00007fd7a6c653d2 00007fd7a01dcbd4
0709 13:03:21.304926 (+   197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:21.305687 (+   761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.305878 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14845184}
I20260709 13:03:21.263015  6794 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.275810  2300 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:21.273679  6775 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:21.275381  6271 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:21.289131  1532 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:21.310278  1709 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:03:21.335335  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 47383.3us
I20260709 13:03:21.268931  6716 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:03:21.277341  6538 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:03:21.319769  6747 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:21.279366  6742 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:21.269551  6668 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.307904  6753 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:21.342402  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 71036.8us
I20260709 13:03:21.346865  6642 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:21.347932  6642 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.348308  6642 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:21.333967  6775 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.314800  6745 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.358692  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:21.347193  6637 raft_consensus.cc:3055] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:21.349768  6775 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:21.367467  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:21.364249  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 75430.1us
I20260709 13:03:21.345121  6742 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.356577  6827 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:21.375938  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:21.377421  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:21.377911  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:21.402490  6753 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:21.364987  6637 raft_consensus.cc:740] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:21.381958  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:21.403198  6753 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.403815  6637 consensus_queue.cc:260] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:21.405676  6637 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:21.406190  6753 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:21.385285  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:21.375698  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:21.389521  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1104) took 1275 ms. Trace:
I20260709 13:03:21.409945  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:21.394171  6855 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:21.395488  6722 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:21.367956  6538 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:21.400545  6728 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:21.379199  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:21.421947  1560 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:21.402537  1569 raft_consensus.cc:2410] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:03:21.410421  1641 rpcz_store.cc:276] 0709 13:03:20.114284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.236163 (+121879us) service_pool.cc:224] Handling call
0709 13:03:20.839313 (+603150us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:20.840020 (+   707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:21.389204 (+549184us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b54000d16d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:21.389428 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1370979584}
I20260709 13:03:21.411350  1532 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:21.424428  1532 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:21.370347  6271 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.374194  6747 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.391148  6754 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:21.427441  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:21.418889  2300 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:21.414039  6823 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:21.497484  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:21.472930  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:21.486510  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:21.465745  1563 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:21.469494  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:21.484678  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:21.503191  1711 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:21.474736  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:21.452530  2304 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:21.441846  1639 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:03:21.469099  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:21.472563  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:21.483896  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:21.505576  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:21.457664  2299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1417) took 2779 ms (client timeout 1737 ms). Trace:
W20260709 13:03:21.514675  2299 rpcz_store.cc:269] 0709 13:03:18.678246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:18.678368 (+   122us) service_pool.cc:224] Handling call
0709 13:03:21.457381 (+2779013us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54005dcd54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:21.457507 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3273280896}
I20260709 13:03:21.517268  2299 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:21.456537  1642 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:21.519873  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:21.471127  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:21.440729  1532 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:21.474392  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:21.522886  2301 raft_consensus.cc:2393] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:21.509065  2302 raft_consensus.cc:2393] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:21.532145  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:21.532963  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:21.536036  1642 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:21.539217  1533 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:21.540174  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 99572.5us
I20260709 13:03:21.542428  1532 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:21.435422  6728 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:21.508580  6632 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:21.467976  6668 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:21.467170  6742 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:21.439406  6747 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.441188  6822 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 13:03:21.508539  6741 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:21.453171  6723 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:21.469687  6748 raft_consensus.cc:3055] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:21.465538  6745 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:21.546819  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:21.549633  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:21.436184  6271 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.436275  6827 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.119s
I20260709 13:03:21.431365  6829 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.462671  6716 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 13:03:21.439769  6393 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:21.463380  6835 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:21.469199  6733 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:21.459342  6784 raft_consensus.cc:3055] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:21.440663  6722 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:21.475073  6754 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:21.572432  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:21.573684  6759 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:21.586130  6728 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:21.552459  6823 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:21.546085  6746 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.589335  6739 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.573268  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:21.465914  6794 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:21.595624  2296 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:03:21.598609  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:21.551956  6740 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:21.551651  6617 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:21.444725  6788 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.602492  1683 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:21.603818  1711 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:21.611544  1682 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:21.612880  1714 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:21.623765  1644 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:21.624588  1568 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:21.625695  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:21.646598  6784 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:21.629587  6533 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:21.647243  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:21.647898  2268 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:21.727648  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 79819us
I20260709 13:03:21.728896  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:21.730302  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:21.654538  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:21.612403  6748 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:21.625061  6618 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:21.669091  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 96731.2us
I20260709 13:03:21.702878  6835 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:21.617372  6742 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:03:21.606814  6375 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:03:21.655628  6743 raft_consensus.cc:3055] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:21.614337  6741 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:21.675289  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 102086us
I20260709 13:03:21.592068  6632 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:21.688817  1639 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:21.746832  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:21.672210  6821 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:21.695056  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 95649.8us
W20260709 13:03:21.695041  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 83626.1us
I20260709 13:03:21.700039  6271 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:21.700717  6754 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:21.698289  6728 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:21.672371  6722 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:21.684298  6759 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:21.684012  6823 raft_consensus.cc:697] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:21.680998  6617 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:21.772655  6728 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:21.808449  6728 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:21.809022  6393 raft_consensus.cc:3055] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:21.809655  6393 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:21.811293  6722 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:21.814266  6741 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:21.693409  6740 raft_consensus.cc:697] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:21.823022  6740 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.676445  6739 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.674288  6746 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.828890  6746 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:21.686950  6649 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:03:21.810582  6393 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:21.749909  6642 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:21.756292  6504 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:03:21.832293  6728 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.748628  1533 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:21.900190  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 151644us
I20260709 13:03:21.849080  6739 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:21.901314  6739 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:21.901718  1533 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:21.806094  6617 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:21.748849  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:21.762962  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:21.765336  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:21.903318  1641 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:21.903555  1639 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 11: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:21.903708  6712 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:21.903887  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 138649us
I20260709 13:03:21.904474  6538 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:21.904796  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:21.905593  6712 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:21.905650  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:21.906368  6712 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:21.765523  6830 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:21.907086  6830 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:21.738695  6827 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:21.747576  6743 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:21.736135  6860 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:21.742357  6668 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:21.906590  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:21.742632  6618 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:21.911566  6530 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:21.911988  6530 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.908617  6860 raft_consensus.cc:697] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:21.912413  6530 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:21.908056  6830 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:21.913167  6860 consensus_queue.cc:237] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.914124  6830 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:21.908900  6668 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:21.908715  6743 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:21.915838  6668 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:21.908587  1533 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:21.760058  6632 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:21.725997  6775 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:21.907092  6759 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:21.784214  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:21.832172  6784 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:21.941680  1681 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.030s	sys 0.035s
I20260709 13:03:21.884449  1640 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
W20260709 13:03:21.945304  1681 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.030s	sys 0.036s
I20260709 13:03:21.747299  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:21.950027  1683 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:21.950527  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 166641us
I20260709 13:03:21.670008  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:21.912115  6728 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:21.977129  1714 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:03:21.993942  1711 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: [13.7-13.7]
W20260709 13:03:21.958081  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 210987us
I20260709 13:03:21.999441  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:21.907510  6712 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:21.978029  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2359) took 1734 ms (client timeout 1727 ms). Trace:
W20260709 13:03:21.981371  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2357) took 2268 ms (client timeout 2078 ms). Trace:
I20260709 13:03:21.942183  6739 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:21.943334  6759 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:21.879216  6744 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.840044  6788 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:21.838362  6747 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:22.002128  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:21.951524  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 409180us
I20260709 13:03:22.022025  1532 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:22.019354  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:22.023438  1532 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:21.947420  6743 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:22.005841  1640 rpcz_store.cc:269] 0709 13:03:19.708531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:19.708663 (+   132us) service_pool.cc:224] Handling call
0709 13:03:21.959813 (+2251150us) spinlock_profiling.cc:243] Waited 2.17 s on lock 0x7b5400123c54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:21.964242 (+  4429us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5000427776}
I20260709 13:03:21.912124  6618 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:21.777143  6551 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:22.004902  1643 rpcz_store.cc:269] 0709 13:03:20.243140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.523617 (+280477us) service_pool.cc:224] Handling call
0709 13:03:21.977724 (+1454107us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54002d8a54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:21.977932 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3241153408}
I20260709 13:03:21.947405  6767 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:21.980486  1636 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:21.973251  6827 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:21.942169  6732 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:03:21.965181  6637 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:21.958315  6716 consensus_queue.cc:1059] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:03:22.012624  6728 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:21.964722  6538 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:03:21.839026  6829 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:21.948140  6719 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:03:22.032837  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:22.016752  6759 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:21.949476  6822 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
W20260709 13:03:22.044059  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.031481  6788 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:22.017128  6867 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:22.017293  6533 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:03:22.030292  6756 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:22.035456  6747 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:22.036914  6827 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:22.023599  6746 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:03:22.038626  6637 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:03:22.052371  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 296228us
I20260709 13:03:22.027559  6739 consensus_queue.cc:1059] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:03:22.061679  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:22.023707  6744 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.064380  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:22.065021  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:22.043043  6728 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:22.062739  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:22.065258  1560 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:22.070657  1712 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:22.066613  6733 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:21.887066  6823 consensus_queue.cc:237] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:22.068565   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:22.080878  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:22.069139  2298 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:22.064043  6637 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.060611  6827 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:22.083874  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 337114us
I20260709 13:03:22.087504  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:22.088629  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:22.072811  6728 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:22.066797  6551 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.065549  6829 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:22.090929  1566 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:22.099568  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:22.103716  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:22.104396  1560 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:22.108608  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:22.111328  1709 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:22.083972  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 147123us
I20260709 13:03:22.097082  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:22.121518  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:22.122035  1533 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:22.122831  1560 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:22.125824  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:22.126880  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.073899  6756 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:03:22.145315  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:22.124559  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:22.146220  1643 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:22.141621  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:22.146948  1681 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:22.147878  1636 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:22.142583  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:22.150167  1640 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:22.150343  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:22.124915  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:22.143923  1639 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:03:22.151105  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 125569us
I20260709 13:03:22.151500  6786 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:22.100997  6821 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:22.154799  1681 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:22.157233  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:22.080804  6733 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:22.154899  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:22.160063  2303 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:22.163235  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:22.164260  2304 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:22.165998  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:22.089430  6169 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:03:22.126024  6271 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:22.126631  6822 raft_consensus.cc:3055] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:22.163162  2269 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:22.173336  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:22.154145  1607 leader_election.cc:400] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12.
I20260709 13:03:22.120301  6868 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:22.171097  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 90335.5us
I20260709 13:03:22.161902  2305 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:22.144990  6775 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:22.150695  6533 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:22.110020  6551 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.153712  6642 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:22.168287  6829 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.037890  6767 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.209626  1681 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:22.162138  6746 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:22.170112  6867 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.169912  6649 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:22.174526  6271 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:22.217779  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.218907  1709 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:03:22.174482  6822 raft_consensus.cc:740] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:22.220325  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:22.228245  1607 leader_election.cc:403] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:03:22.222515  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 72278.7us
I20260709 13:03:22.264842  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:22.211633  6716 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:03:22.233628  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:22.172231  6875 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.061s
W20260709 13:03:22.234491  6869 log.cc:927] Time spent T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:03:22.208269  6879 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:03:22.219238  6880 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:22.230736  6742 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:22.261523  1642 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:03:22.280224  1642 pending_rounds.cc:85] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 11
I20260709 13:03:22.267493  1636 raft_consensus.cc:3055] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:22.281235  1636 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:22.268929  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:22.256682  6271 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.275243  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:22.283855  2305 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:03:22.269937  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 160195us
I20260709 13:03:22.290464  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:22.257062  6754 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:22.278044  6865 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:03:22.272372  6169 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12.
I20260709 13:03:22.324118  1718 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:22.243593  6829 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.273957  6649 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.276465  6504 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:22.272228  6716 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:22.315474  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:22.282006  1636 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.258059  6882 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:03:22.310353  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 155572us
I20260709 13:03:22.280550  1642 pending_rounds.cc:107] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 16.12
I20260709 13:03:22.282552  6880 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.268668  6533 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:03:22.255576  6746 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.297526  6744 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:22.343286  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:22.344184  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:22.345379  1640 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:22.344396  1642 raft_consensus.cc:2917] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Aborting config change with OpId 16.12: Aborted: Op aborted by new leader
I20260709 13:03:22.351079  1642 raft_consensus.cc:2889] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:22.323253  6872 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:22.239313  6821 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:22.354395  2269 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:22.356745  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:22.358763  1643 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:22.376201  6732 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:22.381278  6742 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:22.381842  6742 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:22.335183  6876 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:03:22.330343  6822 consensus_queue.cc:260] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:22.385144  6822 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:22.341743  6716 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:22.261624  6867 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.339037  6504 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:22.369886  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 404121us
W20260709 13:03:22.400502  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 46177.8us
I20260709 13:03:22.417833  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:22.401298  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 44618.9us
W20260709 13:03:22.370559  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 407268us
I20260709 13:03:22.348627  6741 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:22.356606  6642 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.339658  6649 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:22.387365  6733 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:22.355856  6169 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:22.408275  6886 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:22.436725  6393 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:22.442487  6746 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:22.443953  6829 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:22.412433  2269 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:22.408744  6871 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:22.442323  6742 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.347328  6880 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.429171  6504 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:22.411089  6876 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:03:22.445264  6877 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 13:03:22.428892  6875 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:22.478708  6890 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:03:22.426097  6530 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.494050  6728 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.496430  6728 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:22.481287  6886 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.498422  6649 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:22.494099  6742 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:22.479050  6871 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:22.499752  6871 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:22.497236  6830 consensus_meta.cc:287] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.583s	user 0.008s	sys 0.002s
I20260709 13:03:22.499788  6886 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:22.458557  6888 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:22.448756  6642 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:22.500720  6748 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.491560  6875 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.505934  6784 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.507717  6722 raft_consensus.cc:3055] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:22.508579  6867 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:22.487831  6668 consensus_meta.cc:287] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.002s
I20260709 13:03:22.477053  6893 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:22.475083  6745 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:22.513338  6759 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:22.517164  6884 log.cc:927] Time spent T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.241s	user 0.000s	sys 0.002s
I20260709 13:03:22.519246  6893 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:22.490763  6876 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
W20260709 13:03:22.557416  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:22.377553  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 97356us
W20260709 13:03:22.725924  6637 consensus_meta.cc:287] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.631s	user 0.009s	sys 0.006s
I20260709 13:03:22.559406  1568 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:22.448097  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:22.565769  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 222369us
W20260709 13:03:22.588393  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.623677  1716 raft_consensus.cc:2393] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:22.624155  6530 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:22.489780  6551 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:22.625306  6649 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.517127  6668 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.516527  6722 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:22.628764  6642 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:22.749501  6668 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:22.750790  2304 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:22.632478  6745 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.634214  6742 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:22.752234  6742 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:22.752704  6649 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:22.753896  6742 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.634773  6759 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:22.514860  6767 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:22.505589  6632 consensus_meta.cc:287] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.000s
I20260709 13:03:22.755705  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:22.756634  6632 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:22.757566  1713 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:22.515506  6794 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:22.760926  6794 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:22.646884  6712 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.765020  6632 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:22.765345  6712 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:22.766052  6668 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:22.766851  6668 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.768217  6712 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:22.515677  6875 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:22.524936  6893 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:22.769361  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:22.711102  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 260219us
W20260709 13:03:22.728010  6617 consensus_meta.cc:287] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.825s	user 0.017s	sys 0.005s
I20260709 13:03:22.726584  6637 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.726581  6740 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:22.728111  6271 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.726812  6830 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:22.762558  6794 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.782806  6830 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.784432  6794 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.786375  6830 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:22.731451  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:22.787972  6716 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:22.790771  6716 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:22.731724  6888 raft_consensus.cc:697] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:22.795775  6888 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.742019  1607 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:22.744316  6876 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:22.748240  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 300093us
I20260709 13:03:22.750555  6722 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:22.756631  2305 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:22.777623  6740 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:22.780663  6637 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:22.781591  6728 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:22.781838  1710 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:22.804533  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:22.804788  6722 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:22.805848  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 64098.3us
I20260709 13:03:22.805771  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:22.774832  6617 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:22.806864  6876 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.807040  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:22.816632  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:22.819413  1639 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:22.819024  6504 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:22.820971  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.821887  6504 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:22.816737  6668 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:22.823024  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:22.824523  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 399836us
I20260709 13:03:22.829048  2297 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:03:22.829228  6271 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:22.830436  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:22.832553  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:22.833583  1717 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:22.837644  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:22.833659  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2386) took 1362 ms. Trace:
I20260709 13:03:22.851164  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:22.851506  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 884) took 1935 ms (client timeout 1788 ms). Trace:
I20260709 13:03:22.856783  1637 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:22.809882  6649 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:22.857851  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:22.868369  6649 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:22.860000  1534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.010s	sys 0.031s
I20260709 13:03:22.869133  6649 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:22.869834  1534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.010s	sys 0.031s
W20260709 13:03:22.870275  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 642197us
I20260709 13:03:22.841804  1642 rpcz_store.cc:276] 0709 13:03:21.471367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:21.546339 (+ 74972us) service_pool.cc:224] Handling call
0709 13:03:21.960499 (+414160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:22.351690 (+391191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:22.697208 (+345518us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b5400123c54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:22.833174 (+135966us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b5400123c50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:22.833457 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1643347712}
I20260709 13:03:22.873028  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:22.874315  6823 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:22.876017  6868 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:22.877024  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:22.860327  1643 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:22.863322  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:22.864920  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:22.864351  6732 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:22.866678  6745 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:22.867692  1640 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:22.853085  1566 rpcz_store.cc:269] 0709 13:03:20.916358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:20.916524 (+   166us) service_pool.cc:224] Handling call
0709 13:03:22.845371 (+1928847us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54001ff254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:22.851218 (+  5847us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2698555776}
I20260709 13:03:22.885489  6649 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:22.885623  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:22.892681  6748 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:22.873056  1534 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:22.902405  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:22.906172  2268 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.788s (SENT) [suppressed 13 similar messages]
I20260709 13:03:22.907251  1636 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:22.912070  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:22.913254  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:22.916484  6787 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:22.919035  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:22.922742  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:22.923612  1640 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:22.930361  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:22.931444  1564 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:22.932717  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:22.933652  1566 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:22.932926  6740 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:22.946734  6617 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:22.951671  1637 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:22.952494  6821 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:22.952986  6821 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.954161  1643 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:22.956389  6871 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:22.961113  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:22.978600  1639 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:22.979017  2305 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:22.981559  1566 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
W20260709 13:03:22.982597  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 109684us
I20260709 13:03:22.985409  1642 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:22.990320  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:22.987272  1644 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:22.999344  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:23.000223  1639 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.001480  2300 raft_consensus.cc:2368] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15.
I20260709 13:03:23.004669  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:22.977731  6668 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:23.010737  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 49741.3us
I20260709 13:03:23.016541  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:23.013437  6827 consensus_meta.cc:287] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.759s	user 0.016s	sys 0.000s
I20260709 13:03:23.034271  6794 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:23.037683  6570 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:23.038482  6570 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:23.041220  6570 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:23.041337  6880 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:23.044862  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 49046.1us
I20260709 13:03:23.045826  1709 raft_consensus.cc:2368] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14.
I20260709 13:03:23.036602  6871 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:23.047223  6784 consensus_meta.cc:287] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.533s	user 0.013s	sys 0.004s
I20260709 13:03:23.048214  6827 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.053447  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:23.054049  6775 consensus_queue.cc:1059] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:23.055164  6775 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:23.056591  6864 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:23.057185  6864 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:23.058290  6864 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.059044  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.060518  6716 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:23.062763  6896 raft_consensus.cc:3055] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:23.063117  6530 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:23.063813  6530 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:23.063951  6830 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:03:23.064184  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.065012  6530 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.065878  6896 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:23.066840  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.067130  6896 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.068234  1566 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:03:23.068882  6896 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:23.063184  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:23.069729  1564 raft_consensus.cc:2368] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
I20260709 13:03:23.073086  6821 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:23.073764  6742 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:23.074632  6742 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.077131  6742 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:23.087399  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:23.088148  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:23.088994  2299 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
W20260709 13:03:23.090189  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 175408us
I20260709 13:03:23.091202  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:23.092921  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:23.095131  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:23.097010  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:23.099884  6530 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:23.088280  2303 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:23.113364  2300 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:03:23.114174  2304 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:23.115788  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:23.087142  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:23.121614  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 215496us
W20260709 13:03:23.115444  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 83699.6us
I20260709 13:03:23.033143  6617 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:23.135314  1644 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:23.143525  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:23.050899  6827 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:23.115103  6876 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:23.048204  6880 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:23.147475  1636 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:03:23.119236  6830 raft_consensus.cc:2764] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:03:23.139523  6823 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:23.160965  1636 pending_rounds.cc:85] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 11
I20260709 13:03:23.133414  6821 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:23.152303  6921 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:03:23.139643  6617 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.174763  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:23.177259  1635 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:23.170253  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:23.108630  6668 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.175273  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:23.173105  1636 pending_rounds.cc:107] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:03:23.185593  1636 raft_consensus.cc:2889] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:23.157951  6794 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:23.187433  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.136184  6740 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:03:23.175371  2296 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:03:23.136351  6716 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:03:23.154953  6748 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:23.181901  6617 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.221848  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:23.223840  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 63877.5us
I20260709 13:03:23.223752  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:23.230510  1643 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 13: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:23.232597  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:23.233060  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:23.233477  1640 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:23.194540  6722 consensus_queue.cc:1059] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:03:23.183215  6821 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.243753  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:23.248028  1639 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:23.250173  6801 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:23.250854  6801 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.253290  6801 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:23.253070  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:23.254139  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:23.255336  6827 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:23.255678  2268 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:23.256314  6827 consensus_queue.cc:1059] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:03:23.258270  6919 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.050s	user 0.002s	sys 0.000s
I20260709 13:03:23.259377  6271 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:23.259644  6519 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.107s
I20260709 13:03:23.260177  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:23.272703  1532 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:23.227155  6794 raft_consensus.cc:697] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:23.287284  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:23.290261  6821 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:23.237486  6880 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:23.291231  6860 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.296228  6668 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:23.300848  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 136311us
W20260709 13:03:23.306080  6893 consensus_meta.cc:287] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.003s
W20260709 13:03:23.309110  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.308981  6888 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:23.367229  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:23.367569  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.324857  1644 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:23.370182  1636 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:03:23.384171  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 124410us
I20260709 13:03:23.389442  1681 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:23.390406  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 59199.8us
I20260709 13:03:23.390503  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.333352  6860 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:23.401458  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.368010  6668 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:23.395437  1533 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:23.402412  1609 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:23.368643  6732 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:23.415068  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.419049  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:23.419700  1642 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 9.
I20260709 13:03:23.422325  6897 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:23.423213  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:23.425995  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.422765  6897 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:23.428162  6897 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:23.431598  1564 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:23.432953  1669 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:37201 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:37201 timed out after 15.000s (SENT)
I20260709 13:03:23.433300  1566 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:23.383572  6716 raft_consensus.cc:3055] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:23.442001  1683 leader_election.cc:400] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15.
I20260709 13:03:23.442376  1683 leader_election.cc:403] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:03:23.445915  6732 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:23.446447  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.446813  1566 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:23.448092  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 175496us
I20260709 13:03:23.448773  6732 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:23.449296  1532 leader_election.cc:400] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14.
I20260709 13:03:23.449654  1532 leader_election.cc:403] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:03:23.368155  6893 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.451237  6830 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:03:23.451458  6745 raft_consensus.cc:3055] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:23.452591  6830 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:23.452705  6893 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:23.442394  2269 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:23.455287  2269 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:23.455700  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.456211  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:23.456360  1569 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:23.452100  6745 raft_consensus.cc:740] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:23.446543  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.459815  1640 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
W20260709 13:03:23.461741  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 59456us
I20260709 13:03:23.459753  2300 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:23.457332  2269 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:23.467453  1609 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:23.407703  6880 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:23.471215  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 75895us
I20260709 13:03:23.444101  6716 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:23.476969  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 150716us
I20260709 13:03:23.476114  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.479825  2298 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:03:23.481086  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 113609us
I20260709 13:03:23.484023  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:23.437146  6827 consensus_queue.cc:1059] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:03:23.464607  6519 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:23.394997  6888 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.487609  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:23.489611  6742 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:23.467072  6871 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:23.490818  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:23.513150  1711 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:23.491091  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.491252  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:23.510277  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 73418.3us
I20260709 13:03:23.511305  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:23.506549  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 174035us
I20260709 13:03:23.515829  1682 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:23.506210  1716 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:03:23.510775  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:23.491452  6393 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.526163  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.519529  1709 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:23.528234  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.479370  6821 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:23.528978  1564 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:23.531563  6716 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.536002  1636 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:03:23.537662  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.538239  6642 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:23.538601  6933 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.491091  6801 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:23.539433  6801 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.491653  6883 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:23.539961  6801 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:23.540388  6642 raft_consensus.cc:3055] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:23.511511  6823 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:23.540884  6642 raft_consensus.cc:740] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:23.541170  6823 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.541899  6642 consensus_queue.cc:260] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.543164  6932 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:23.543040  6794 consensus_queue.cc:237] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:23.492726  6732 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.548235  1681 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:23.549376  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:23.549778  6858 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:23.539141  6933 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.528375  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:23.525054  1565 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:23.532938  1717 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:03:23.526849  1563 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:23.557430  6871 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:23.563988  6931 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:03:23.564105  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:23.557323  6393 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:23.505100  6913 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:23.567646  6913 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:23.568575  6913 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:23.569830  6913 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:23.528990  6821 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:23.571506  6860 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:23.512702  6742 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:23.557703  6519 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:23.572317  1562 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:23.492429  6784 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:23.573285  6784 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.573114  6519 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.575173  6617 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:23.576429  6930 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:03:23.571321  6821 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:23.579613  1640 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:23.579826  6827 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:23.550223  6858 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.580886  6858 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:23.565963  6823 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:23.573879  6784 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:23.583864  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.583611  2268 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:23.561888  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:23.590984  2305 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:23.588624  1711 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:23.593397  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:23.572273  6742 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:23.594246  1640 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:23.595824  1681 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:23.593659  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:23.595948  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:23.597133  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:23.598340  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:23.602886  1568 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:03:23.623414  1718 raft_consensus.cc:2410] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:03:23.594974  6642 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:03:23.643663  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 127954us
I20260709 13:03:23.644582  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:23.645215  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 44597.8us
I20260709 13:03:23.651198  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:23.654791  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 71263.3us
I20260709 13:03:23.599045  6271 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:23.678591  6888 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:23.679867  6617 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:23.675284  1712 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:23.669797  6821 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.682312  6830 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14.
I20260709 13:03:23.685111  6897 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:23.725227  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 69679us
I20260709 13:03:23.686902  6393 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:23.686826  6570 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:23.687887  1682 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:23.683528  6801 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.564028  6871 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.603492  6637 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:23.703473  6784 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.660946  6934 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:23.653522  2269 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:23.655103  6722 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:03:23.711043  6748 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:03:23.585480  6896 consensus_queue.cc:1175] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3f64b76180a54492bb015e405caeacec" responder_term: 9 status { last_received { term: 9 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 9 index: 9 } }
I20260709 13:03:23.671084  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:23.675205  6868 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:23.663229  6800 consensus_queue.cc:1059] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:23.739595  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:23.656256  1565 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:23.721839  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:23.727764  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 64936.2us
I20260709 13:03:23.729437  1532 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:23.734951  1682 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:23.739871  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 96548.8us
I20260709 13:03:23.748255  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:23.752966  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:23.732236  6530 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.736682  6642 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:23.746784  6888 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:23.765575  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:23.739558  6934 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:23.740617  6868 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:23.749783  6271 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:23.781038  1709 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
W20260709 13:03:23.786659  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:23.787277  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.764492  6642 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.795746  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:23.796711  1534 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:23.768517  6722 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:23.804741  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:23.774422  6868 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.783798  6271 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.764348  6827 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:03:23.813509  1563 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 15: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:23.774605  6934 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:23.798585  6538 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.028s	sys 0.046s
W20260709 13:03:23.813835  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 82759.8us
I20260709 13:03:23.825637  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:23.817904  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 44859.5us
I20260709 13:03:23.781543  6860 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:23.834786  6732 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15.
I20260709 13:03:23.835479  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:23.835772  1565 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:03:23.836740  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:23.837379  6943 log.cc:927] Time spent T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:03:23.837870  6733 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:23.838374  6733 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:23.838749  6733 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.838973  6876 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:23.839716  6876 consensus_queue.cc:1243] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 10.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:03:23.842084  6937 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:03:23.843513  1714 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
W20260709 13:03:23.846156  6924 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.596s	user 0.005s	sys 0.027s
I20260709 13:03:23.849833  6858 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:23.850870  6801 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:23.851578  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:23.805768  6642 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.836239  1710 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:23.867023  1717 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:23.771517  6530 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.808983  6637 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:23.868768  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:23.827713  6538 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.029s	sys 0.047s
I20260709 13:03:23.820163  6271 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:23.879357  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 85319.4us
W20260709 13:03:23.883364  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 86744.6us
I20260709 13:03:23.874557  1561 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:23.859481  6893 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.859198  6632 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:23.868280  6827 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.897053  1534 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:23.911147  6858 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:23.913249  6393 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:23.915366  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:23.915920  1561 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:23.905323  6668 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:23.920380  6668 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.920672  6742 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.922526  6668 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:23.922888  6742 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:23.919869  6941 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:03:23.933173  1567 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:23.874502  6860 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:23.976295  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 79344.4us
I20260709 13:03:23.940968  1710 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
W20260709 13:03:23.981740  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 82853.8us
I20260709 13:03:23.914754  6393 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:23.994045  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:23.993769  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:24.023881  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 192964us
I20260709 13:03:24.109443  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:24.110386  1644 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:24.035568  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 267375us
I20260709 13:03:23.941329  6632 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:23.974696  6897 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:24.059932  6800 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.121040  1640 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:24.119472  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.122852  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:24.015479  6947 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:23.996194  6722 raft_consensus.cc:697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:24.134366  1638 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:24.133662  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:24.122483  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:23.954262  6821 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:24.016839  6530 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:24.010360  6880 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:24.121567  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:24.140023  2297 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:03:24.070688  6649 raft_consensus.cc:3055] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:24.015200  6949 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:24.110971  6637 raft_consensus.cc:697] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:24.032862  6716 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:23.929800  6893 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:23.954519  6827 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.120375  6775 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.136617  6393 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.081988  6745 consensus_queue.cc:260] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:24.150749  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.120683  6632 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:24.141918  6794 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.173285  1644 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:24.151784  1637 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
W20260709 13:03:24.156237  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 291387us
I20260709 13:03:24.169216  6893 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:24.144440  6649 raft_consensus.cc:740] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:03:24.176088  6957 log.cc:927] Time spent T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.107s	user 0.004s	sys 0.002s
I20260709 13:03:24.158100  6637 consensus_queue.cc:237] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.159960  6860 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:24.184607  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:24.188328  1637 raft_consensus.cc:2410] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 15 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:03:24.184777  6915 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:24.188668  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:24.189540  6939 log.cc:927] Time spent T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.236s	user 0.003s	sys 0.002s
I20260709 13:03:24.160249  6642 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:24.197984  1712 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:24.197561  6722 consensus_queue.cc:237] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:24.200173  6642 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:24.192415  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:24.200620  1637 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:24.201170  1642 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:24.203449  6649 consensus_queue.cc:260] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:24.205104  6649 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:24.207042  6794 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.209692  6794 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:24.210594  1640 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:03:24.193369  6393 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:24.194403  6632 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:24.190148  6897 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:24.217545  1566 pending_rounds.cc:85] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 3
I20260709 13:03:24.218170  1566 pending_rounds.cc:107] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 10.4
I20260709 13:03:24.218465  1566 raft_consensus.cc:2889] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:24.218393  6632 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:24.219199  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:24.219947  1564 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:24.221292  6784 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:24.223078  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:24.213244  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 219595us
I20260709 13:03:24.213228  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:24.213804  6860 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:24.212436  2297 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
W20260709 13:03:24.231221  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 51385.5us
I20260709 13:03:24.235441  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:24.144281  6733 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.237682  1568 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 8. Current term is 9. Ops: []
W20260709 13:03:24.237780  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.242899  6271 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.177729  6668 consensus_queue.cc:1059] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:03:24.245923  2304 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:24.230221  6860 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.248795  6860 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:24.216147  6775 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.251786  6775 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:24.248534  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:24.253377  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:24.256395  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 44260.2us
I20260709 13:03:24.234021  1682 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:24.261018  1564 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:24.263756  1567 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
W20260709 13:03:24.255122  6958 log.cc:927] Time spent T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.092s	user 0.001s	sys 0.001s
I20260709 13:03:24.161424  6880 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.268733  6933 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:24.149847  6947 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.167105  6800 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.270293  6864 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:24.271051  6959 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:03:24.255903  6538 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.272919  6530 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:24.273679  6868 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:24.274530  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:24.178010  6934 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:24.265071  6733 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:24.291872  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 57927.2us
I20260709 13:03:24.302254  2268 leader_election.cc:400] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
I20260709 13:03:24.303328  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:24.342254  2268 leader_election.cc:403] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:03:24.306416  2302 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:24.277047  6271 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:24.322510  6530 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.323910  1562 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:24.324899  6947 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:24.327359  6868 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:24.330077  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:24.319473  6938 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.370s	user 0.004s	sys 0.000s
I20260709 13:03:24.332197  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1325) took 1120 ms. Trace:
I20260709 13:03:24.332547  2300 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:03:24.334012  6745 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:24.339020  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.307279  6642 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:03:24.341179  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 154651us
I20260709 13:03:24.366377  1560 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 12. (term mismatch)
I20260709 13:03:24.366845  1560 pending_rounds.cc:85] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 11
I20260709 13:03:24.367126  1560 pending_rounds.cc:107] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 11.12
I20260709 13:03:24.367353  1560 raft_consensus.cc:2889] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:24.367631  6868 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:24.368575  6748 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.295023  6519 consensus_queue.cc:1059] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:03:24.369307  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:24.356400  2304 rpcz_store.cc:276] 0709 13:03:23.211817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.211973 (+   156us) service_pool.cc:224] Handling call
0709 13:03:23.919060 (+707087us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.246384 (+327324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.331182 (+ 84798us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54005d2ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:24.331969 (+   787us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b54005d2ad0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:24.332113 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2343555712}
I20260709 13:03:24.369974  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:24.377895  2305 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:24.366377  1533 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:24.371062  6880 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:24.372146  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:24.317569  6570 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:03:24.351416  6827 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:24.391338  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:24.392011  2296 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:24.392485  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:24.393666  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:24.395088  6722 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:24.395627  6722 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.396000  6722 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:24.397195  6519 raft_consensus.cc:3055] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:24.397648  6519 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:24.372568  1609 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:24.395155  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 99398.6us
I20260709 13:03:24.400864  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:24.368113  6668 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
I20260709 13:03:24.342535  1642 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:24.407524  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:24.370679  6530 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:24.394675  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:24.411221  1644 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:24.404489  1638 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:24.405279  2305 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:24.399605  1637 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 13:03:24.413764  1637 pending_rounds.cc:85] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 12
I20260709 13:03:24.413926  1636 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:24.414184  1637 pending_rounds.cc:107] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 13:03:24.414592  1637 raft_consensus.cc:2889] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:24.416302  1609 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:24.416929  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:24.319653  6538 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:24.418671  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:24.424319  1568 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:24.375885  6642 raft_consensus.cc:697] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:24.423638  1562 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:24.426637  6871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.006s	sys 0.013s
W20260709 13:03:24.427127  6871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.006s	sys 0.013s
I20260709 13:03:24.428429  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:24.415717  1683 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:24.431174  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:24.434051  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:24.437973  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:24.438705  1714 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:24.411252  6733 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:24.443689  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:24.444528  6642 consensus_queue.cc:237] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:24.447070  6966 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:03:24.447234  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.447878  6827 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:24.432340  6393 consensus_queue.cc:1059] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:03:24.448482  6827 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:24.448915  6827 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:24.449110  6949 raft_consensus.cc:3055] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:24.449529  6949 raft_consensus.cc:740] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:24.409209  6775 consensus_queue.cc:1059] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:03:24.450434  6949 consensus_queue.cc:260] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:24.451012  6728 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:24.377414  6748 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:24.451817  6949 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:24.451680  6728 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.453434  6748 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:24.454459  6946 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:24.441857  1716 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:03:24.454881  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1329) took 1203 ms. Trace:
I20260709 13:03:24.455945  2301 rpcz_store.cc:276] 0709 13:03:23.251795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.256132 (+  4337us) service_pool.cc:224] Handling call
0709 13:03:23.256615 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:23.258446 (+  1831us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:23.258481 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:23.327667 (+ 69186us) log.cc:844] Serialized 26 byte log entry
0709 13:03:23.733277 (+405610us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:23.733319 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:23.733385 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:24.206498 (+473113us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:24.454499 (+248001us) raft_consensus.cc:1685] finished
0709 13:03:24.454521 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:24.454726 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2815,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":6383,"spinlock_wait_cycles":41104768,"thread_start_us":339502,"threads_started":1,"wal-append.queue_time_us":429755}
I20260709 13:03:24.457774  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.448930  6393 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:24.458626  6393 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.459165  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.459599  2268 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:24.461783  2268 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:24.462510  1637 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 11: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:24.462841  6940 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:24.463721  6530 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:24.464181  6940 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:24.464349  6530 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.465744  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.462545  6951 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:24.455868  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.447942  1713 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:24.467083  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.467399  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.467803  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.468261  1715 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:24.468168  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.468691  1564 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:03:24.468884  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.469098  1564 pending_rounds.cc:85] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 7
I20260709 13:03:24.469491  1564 pending_rounds.cc:107] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:03:24.469583  1710 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:24.469834  1564 raft_consensus.cc:2889] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:24.470182  1713 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:24.469244  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.470446  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.433185  6712 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.471194  1711 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:24.465688  6393 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:24.471513  6712 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.472105  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.442865  6271 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:24.474327  1718 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:24.474771  2269 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:24.472129  6393 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:24.475651  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 60026.3us
I20260709 13:03:24.475837  6393 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.478456  6893 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:24.477113  2269 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:24.479053  6893 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:24.480314  6934 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:24.480149  6893 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.480791  6915 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:24.481389  6915 raft_consensus.cc:697] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:24.398469  6519 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.477713  6733 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:24.477762  6858 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:24.483351  6858 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:24.483624  6519 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:24.483834  6733 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.485975  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.462081  6860 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.465860  6538 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:24.489868  1716 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:24.491818  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:24.499527  1568 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:03:24.500789  6393 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:24.482357  6915 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:24.486716  1718 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:24.493209  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:24.489882  2298 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:24.504105  1561 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:24.512548  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:24.483850  6858 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:24.524564  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:24.531276  1564 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:24.525421  1561 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:03:24.526976  1609 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:24.529937  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:24.567401  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:24.569631  1681 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:24.494122  6946 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 15 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:24.573232  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:24.578418  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 116724us
W20260709 13:03:24.589598  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.619419  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 92613.4us
I20260709 13:03:24.618667  1681 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:24.633015  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:24.633767  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 103951us
I20260709 13:03:24.536379  6940 raft_consensus.cc:2764] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:03:24.646235  1718 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:24.635324  1681 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:24.660115  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:24.686741  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:24.739259  2305 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 15: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:24.733587  2302 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:24.733984  2301 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:24.686323  1564 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:24.747897  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:24.619047  6775 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:03:24.669232  6966 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:03:24.765004  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:24.771878  2298 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:24.574747  6880 consensus_queue.cc:1059] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 13:03:24.776819  1714 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:24.777846  1710 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:24.668998  6794 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:24.779333  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 144849us
I20260709 13:03:24.670192  6637 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:24.760344  6941 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:03:24.788233  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:24.782676  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:24.757046  6876 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.796341  6821 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:03:24.803227  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:24.805445  6963 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:24.805909  6969 log.cc:927] Time spent T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.379s	user 0.001s	sys 0.000s
I20260709 13:03:24.681092  6800 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:24.812034  6961 log.cc:927] Time spent T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:03:24.733052  6649 consensus_queue.cc:1059] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:03:24.785965  6794 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.816666  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:24.818312  6860 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:24.781694  6775 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:24.884184  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:24.884893  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:24.827286  6950 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:24.827729  6169 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:24.831722  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:24.832545  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:24.807734  1714 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:03:24.854312  1566 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:24.812286  6716 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:24.816316  6963 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:24.876214  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:25.034258  1713 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:24.813776  6876 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:24.878651  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:24.883695  6821 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:03:24.904630  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:24.995151  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 110347us
W20260709 13:03:24.868466  6966 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.055s	user 0.000s	sys 0.004s
W20260709 13:03:25.020216  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 133881us
I20260709 13:03:25.037542  1682 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:25.039026  1682 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:25.033808  2301 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 10: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:24.867332  6794 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:25.036338  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:25.045526  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:25.045471  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:25.047919  1643 raft_consensus.cc:2410] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:03:25.050333  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:25.028970  6950 raft_consensus.cc:697] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:03:25.052243  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.055145  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:25.058218  1566 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:25.027061  6951 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:25.020110  6271 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:25.059123  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:03:25.107096  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.108116  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:25.108168  6985 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.109064  6985 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.056577  6716 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.115072  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:25.115945  1641 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:25.118996  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:25.119912  1643 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:25.120162  6880 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:25.123132  6649 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.123986  6649 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.028296  6169 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:25.127691  6965 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.236s	user 0.000s	sys 0.000s
I20260709 13:03:25.037559  6821 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:25.129343  6169 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:24.815395  6946 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.129891  6821 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:25.035615  6933 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:25.131338  6821 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:25.131418  6933 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.038381  6963 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:25.135336  6964 log.cc:927] Time spent T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.596s	user 0.002s	sys 0.001s
I20260709 13:03:25.136869  6271 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.041730  6637 raft_consensus.cc:2764] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:03:25.097241  6860 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:25.141389  6860 raft_consensus.cc:697] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:25.051739  6897 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:25.148497  6876 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:25.306159  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 247142us
I20260709 13:03:25.308178  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:25.105965  6775 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:25.321085  6722 consensus_meta.cc:287] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.925s	user 0.002s	sys 0.008s
I20260709 13:03:25.321725  6722 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:25.324283  6985 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:25.142652  6860 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:25.329185  6858 consensus_meta.cc:287] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.719s	user 0.010s	sys 0.001s
I20260709 13:03:25.329805  6858 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.114635  6716 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:25.376227  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 68152.4us
I20260709 13:03:25.134627  6933 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:25.359261  1561 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 13:03:25.444830  1562 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.036s	sys 0.120s
W20260709 13:03:25.446543  1562 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.036s	sys 0.121s
W20260709 13:03:25.333573  6827 consensus_meta.cc:287] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.884s	user 0.011s	sys 0.000s
I20260709 13:03:25.478366  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2415) took 1301 ms. Trace:
I20260709 13:03:25.126796  6649 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:25.351202  6642 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.027s	sys 0.108s
W20260709 13:03:25.440310  6966 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.300s	user 0.000s	sys 0.000s
I20260709 13:03:25.309445  6876 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:25.488950  6822 consensus_queue.cc:1059] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:03:25.484633  1635 rpcz_store.cc:276] 0709 13:03:24.176963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.198533 (+ 21570us) service_pool.cc:224] Handling call
0709 13:03:24.198894 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.199368 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.199399 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:24.199933 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 13:03:24.307124 (+107191us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:24.307186 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.307284 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.478107 (+1170823us) raft_consensus.cc:1685] finished
0709 13:03:25.478133 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.478289 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5504,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":106759,"threads_started":1,"wal-append.queue_time_us":140132}
W20260709 13:03:25.501505  1567 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.003s	user 0.012s	sys 0.033s
W20260709 13:03:25.483085  6968 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.669s	user 0.001s	sys 0.000s
I20260709 13:03:25.484009  6915 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:03:25.504663  1712 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.009s	sys 0.031s
W20260709 13:03:25.487450  6642 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.028s	sys 0.108s
W20260709 13:03:25.521129  1712 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.009s	sys 0.031s
W20260709 13:03:25.521862  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:25.549711  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 440086us
I20260709 13:03:25.486797  6951 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:25.557914  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2369) took 1116 ms. Trace:
I20260709 13:03:25.555264  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:25.511214  6748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.036s	sys 0.055s
W20260709 13:03:25.567798  6748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.037s	sys 0.055s
W20260709 13:03:25.506729  1567 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.009s	user 0.012s	sys 0.033s
I20260709 13:03:25.560601  1712 rpcz_store.cc:276] 0709 13:03:24.434222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.448661 (+ 14439us) service_pool.cc:224] Handling call
0709 13:03:24.449011 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.449276 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:24.777707 (+328431us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.550500 (+772793us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:25.550552 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:25.550591 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.550653 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.550832 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":343097,"prepare.run_cpu_time_us":182,"prepare.run_wall_time_us":4795,"raft.queue_time_us":31688,"thread_start_us":1059229,"threads_started":2}
I20260709 13:03:25.570338  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2497) took 1076 ms. Trace:
W20260709 13:03:25.570600  1681 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.910s	user 0.012s	sys 0.044s
W20260709 13:03:25.571408  1681 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.911s	user 0.012s	sys 0.044s
I20260709 13:03:25.574322  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2814) took 1224 ms. Trace:
I20260709 13:03:25.575966  1459 catalog_manager.cc:5697] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 7 to 10, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 10 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:25.574802  1636 rpcz_store.cc:276] 0709 13:03:24.349854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.349996 (+   142us) service_pool.cc:224] Handling call
0709 13:03:24.413685 (+ 63689us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.414458 (+   773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.574122 (+1159664us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b5400324bd0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:25.574254 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119205248}
I20260709 13:03:25.517169  6933 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:25.593007  1534 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:25.492694  6827 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.570937  1565 rpcz_store.cc:276] 0709 13:03:24.494217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.569226 (+1075009us) service_pool.cc:224] Handling call
0709 13:03:25.569856 (+   630us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.570030 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:25.570051 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:25.570089 (+    38us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:25.570125 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.570180 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.570285 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:25.560570  6915 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:25.585747  6949 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.585809  6732 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.553777  6955 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:25.631695  1681 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:25.592365  6642 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:25.597033  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.010s	sys 0.001s
I20260709 13:03:25.592092  6932 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:25.521560  6876 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:03:25.641871  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.011s	sys 0.001s
W20260709 13:03:25.643733  6973 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:03:25.645401  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2925) took 1127 ms. Trace:
I20260709 13:03:25.608584  6951 raft_consensus.cc:697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:25.670008  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 77070.3us
I20260709 13:03:25.678723  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1488) took 1353 ms. Trace:
W20260709 13:03:25.653471  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 602541us
I20260709 13:03:25.680395  1682 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:25.625598  6897 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:25.662940  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2487) took 1383 ms. Trace:
I20260709 13:03:25.663563  6642 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:25.655614  6732 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:25.632860  6271 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:25.653002  6794 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.592s	user 0.012s	sys 0.000s
I20260709 13:03:25.655712  1568 rpcz_store.cc:276] 0709 13:03:24.517990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.528218 (+ 10228us) service_pool.cc:224] Handling call
0709 13:03:24.528582 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.565937 (+ 37355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.565988 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:24.609477 (+ 43489us) log.cc:844] Serialized 26 byte log entry
0709 13:03:25.030122 (+420645us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:25.030161 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.030224 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.645088 (+614864us) raft_consensus.cc:1685] finished
0709 13:03:25.645117 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.645301 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127,"spinlock_wait_cycles":76748544,"thread_start_us":273531,"threads_started":1,"wal-append.queue_time_us":851884}
I20260709 13:03:25.679526  2299 rpcz_store.cc:276] 0709 13:03:24.324984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.325147 (+   163us) service_pool.cc:224] Handling call
0709 13:03:24.325537 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.325901 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.325933 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:24.326439 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 13:03:24.543831 (+217392us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:24.543945 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:24.544053 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.678384 (+1134331us) raft_consensus.cc:1685] finished
0709 13:03:25.678416 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.678635 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1120,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":198589,"threads_started":1,"wal-append.queue_time_us":431974}
I20260709 13:03:25.654546  6637 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:25.698954  1641 consensus_meta.cc:287] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.582s	user 0.015s	sys 0.003s
I20260709 13:03:25.700480  1641 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:25.689098  1567 rpcz_store.cc:276] 0709 13:03:24.279072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.295860 (+ 16788us) service_pool.cc:224] Handling call
0709 13:03:24.296687 (+   827us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.297007 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.297047 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:24.360734 (+ 63687us) log.cc:844] Serialized 26 byte log entry
0709 13:03:25.568983 (+1208249us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:25.625099 (+ 56116us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.625325 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.625406 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.662575 (+ 37169us) raft_consensus.cc:1685] finished
0709 13:03:25.662609 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.662823 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23956,"prepare.queue_time_us":55935,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"thread_start_us":1070929,"threads_started":1,"wal-append.queue_time_us":1114090}
I20260709 13:03:25.627966  6933 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.642048  6949 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:25.587952  6950 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.696983  6945 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:25.700686  6993 raft_consensus.cc:3055] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:25.703011  6722 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:25.701082  6984 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:25.578680  1459 catalog_manager.cc:5697] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 11 to 13, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:03:25.726526  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:25.724858  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:25.695075  6897 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:25.723160  6963 consensus_meta.cc:287] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.588s	user 0.008s	sys 0.005s
I20260709 13:03:25.713578  6637 raft_consensus.cc:697] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:25.718487  6985 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:25.715273  6858 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:25.741997  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2494) took 1251 ms. Trace:
I20260709 13:03:25.702437  6794 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.694754  6932 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.722380  6993 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:25.748693  6519 consensus_meta.cc:287] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.264s	user 0.019s	sys 0.002s
I20260709 13:03:25.745739  1562 rpcz_store.cc:276] 0709 13:03:24.490823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.491014 (+   191us) service_pool.cc:224] Handling call
0709 13:03:24.500358 (+  9344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.500704 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.500734 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:24.501234 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:03:24.735695 (+234461us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:25.447400 (+711705us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.510029 (+ 62629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.510134 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:25.741776 (+231642us) raft_consensus.cc:1685] finished
0709 13:03:25.741799 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.741938 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76634,"prepare.queue_time_us":92714,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":149249,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":19330,"thread_start_us":907111,"threads_started":2,"wal-append.queue_time_us":528027}
I20260709 13:03:25.753037  6827 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:25.754824  6989 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:03:25.757573  6896 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:25.758775  6949 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:25.738899  6745 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:25.750804  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1158) took 1256 ms. Trace:
W20260709 13:03:25.793296  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 101945us
I20260709 13:03:25.802176  1534 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:25.785681  6722 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.748382  6963 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:25.804152  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1159) took 1008 ms. Trace:
I20260709 13:03:25.801563  6271 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:25.732941  6915 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.800839  1643 rpcz_store.cc:276] 0709 13:03:24.494565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.506032 (+1011467us) service_pool.cc:224] Handling call
0709 13:03:25.521456 (+ 15424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.550582 (+ 29126us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:25.705791 (+155209us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.750437 (+ 44646us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:25.750490 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:25.750532 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.750607 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.750737 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":247003,"raft.queue_time_us":2593,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":187086,"threads_started":2}
I20260709 13:03:25.743585  6822 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 13:03:25.803155  6995 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:25.816207  6995 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:25.819901  6897 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:25.820670  6876 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:25.821516  6732 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:25.801398  6992 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:25.822403  6992 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.823149  6933 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:25.760757  6951 consensus_queue.cc:237] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.824937  6992 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:25.826802  6963 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:25.799271  6896 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:25.828088  6642 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:25.807753  6722 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:25.829504  6896 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:25.831146  6977 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:25.831710  6977 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.832152  6977 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:25.836393  6932 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:25.838208  6821 consensus_meta.cc:287] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.706s	user 0.013s	sys 0.000s
I20260709 13:03:25.758790  6637 consensus_queue.cc:237] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:25.814962  1534 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:25.942858  6822 raft_consensus.cc:3055] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:25.943351  6822 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:25.944517  6949 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.944756  6997 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.945180  6949 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:25.945334  6997 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:25.947674  6949 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:25.947890  6997 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:25.951232  7004 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:25.952071  7004 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.942052  6993 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.955507  6993 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:25.887681  6794 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:25.981564  7004 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:25.982513  6642 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:25.983261  6642 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:25.985543  6642 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:25.803814  6745 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:25.989863  6745 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:25.960842  6896 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:25.991631  6896 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:25.994102  6896 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:25.996685  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 181894us
I20260709 13:03:25.997714  1534 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:25.998672  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 258907us
I20260709 13:03:25.806612  1635 rpcz_store.cc:276] 0709 13:03:24.796064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:25.549226 (+753162us) service_pool.cc:224] Handling call
0709 13:03:25.587974 (+ 38748us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:25.588195 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:25.726421 (+138226us) log.cc:844] Serialized 14 byte log entry
0709 13:03:25.803719 (+ 77298us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:25.803800 (+    81us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:25.803921 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.803984 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:25.804100 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":216404,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":79,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":1271,"thread_start_us":180741,"threads_started":2,"wal-append.queue_time_us":118015}
I20260709 13:03:26.000379  6953 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:25.996052  7004 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:26.013361  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 288562us
I20260709 13:03:26.014457  2268 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:26.015776  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:26.019060  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:26.020164  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:26.021019  1562 raft_consensus.cc:2410] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:03:26.021569  1565 raft_consensus.cc:2410] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:03:26.021936  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:26.025856  1567 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:26.026808  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:26.023204  6745 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:26.029846  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:26.030081  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:26.000528  6977 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.031359  6953 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:26.033464  7004 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.035969  1564 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:26.036217  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:26.035292  6745 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.039453  6977 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.054859  6860 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:26.041973  6530 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.044317  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:26.059739  6860 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.060135  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.060736  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:26.061774  6860 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:26.061978  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.052155  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2486) took 1835 ms. Trace:
I20260709 13:03:26.064776  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.052837  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.053524  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.065670  2298 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:26.067776  1562 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:26.056740  1567 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.059082  6745 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:26.049146  7008 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.061931  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:26.062366  7004 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.063791  2299 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:03:26.064997  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.082032  2305 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:03:26.074846  6947 raft_consensus.cc:3055] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:03:26.050776  1607 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.955s	user 0.048s	sys 0.049s
I20260709 13:03:26.082743  6947 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:26.073719  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1150) took 1946 ms. Trace:
I20260709 13:03:26.074623  1641 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:03:26.073582  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:26.076524  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.062402  6530 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:26.071691  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.085747  1644 raft_consensus.cc:2410] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:03:26.075438  7008 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.093917  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:26.081941  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:26.083033  1607 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.988s	user 0.051s	sys 0.049s
I20260709 13:03:26.096212  1562 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:26.065141  1563 rpcz_store.cc:276] 0709 13:03:24.217092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.051320 (+1834228us) service_pool.cc:224] Handling call
0709 13:03:26.051681 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.051844 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:26.051867 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:26.051905 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:26.051946 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.051997 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.052099 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:26.075089  6801 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:26.085263  1639 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 13: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:26.036832  6915 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:26.100911  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.071579  6950 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.103343  6950 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:26.104069  7008 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:26.105202  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:26.105705  6950 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:26.106294  2303 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:26.106503  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.107729  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:26.108246  7004 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.108891  7004 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:26.109222  7004 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:26.109284  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2818) took 1733 ms. Trace:
I20260709 13:03:26.109674  7004 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:26.109931  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:26.084079  6947 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:26.111622  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:26.112116  6947 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:03:26.109247  1534 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:26.099056  1567 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:26.085831  2304 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:26.090749  6712 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.101637  2305 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:26.114347  6953 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:26.091480  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.115129  6953 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:26.115562  1636 raft_consensus.cc:2410] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:03:26.115681  6953 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:26.116829  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.117730  1636 raft_consensus.cc:2368] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15.
I20260709 13:03:26.101398  6864 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:26.112195  1563 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:26.114373  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.120486  2304 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:26.099941  6860 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:26.118685  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.121410  6860 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.121841  1636 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:26.123020  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.123678  6860 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:26.124320  6801 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.119470  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:26.125744  6538 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:26.126281  6538 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:26.126125  6953 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.126856  6538 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:26.084591  6955 consensus_queue.cc:1059] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:03:26.135159  1641 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:26.136265  6953 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.140735  1635 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:26.141541  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2930) took 1619 ms. Trace:
I20260709 13:03:26.146034  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.084602  1640 rpcz_store.cc:276] 0709 13:03:24.127875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.128092 (+   217us) service_pool.cc:224] Handling call
0709 13:03:24.146198 (+ 18106us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.211068 (+ 64870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.073497 (+1862429us) spinlock_profiling.cc:243] Waited 64 ms on lock 0x7b54003188d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:26.073660 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147220352}
W20260709 13:03:26.147724  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 46840.9us
I20260709 13:03:26.149055  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:26.150391  6728 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.151091  6801 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.152742  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.153631  1640 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:26.115139  6393 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:03:26.142078  1561 rpcz_store.cc:276] 0709 13:03:24.521987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.783483 (+261496us) service_pool.cc:224] Handling call
0709 13:03:24.783958 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:25.443436 (+659478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.141052 (+697616us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54000a78d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:26.141351 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1261929728}
I20260709 13:03:26.157699  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.154970  6722 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:26.159300  1534 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:26.160151  6722 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:26.160576  6722 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:26.161003  6722 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:26.163447  1534 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:26.164655  6393 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:26.165045  6393 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.165380  6393 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:26.167353  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 73508.1us
I20260709 13:03:26.168308  6271 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:26.168834  6271 raft_consensus.cc:697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:26.169076  6822 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.169747  6271 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.170405  6822 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.109876  1642 rpcz_store.cc:276] 0709 13:03:24.376237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.418585 (+ 42348us) service_pool.cc:224] Handling call
0709 13:03:24.418859 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.447106 (+ 28247us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:24.447420 (+   314us) log.cc:844] Serialized 14 byte log entry
0709 13:03:26.038398 (+1590978us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b18004d8e40. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a 00007fd7a1caba2a
0709 13:03:26.058044 (+ 19646us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:26.058551 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 13:03:26.058691 (+   140us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:26.069897 (+ 11206us) log.cc:844] Serialized 14 byte log entry
0709 13:03:26.078879 (+  8982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.078976 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:26.109035 (+ 30059us) raft_consensus.cc:1685] finished
0709 13:03:26.109064 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.109214 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37173,"prepare.queue_time_us":13748,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":88,"raft.queue_time_us":21466,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":1166,"spinlock_wait_cycles":59943296,"thread_start_us":19410,"threads_started":1,"wal-append.queue_time_us":1610610}
I20260709 13:03:26.174609  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.176668  1682 leader_election.cc:400] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15.
I20260709 13:03:26.177084  1682 leader_election.cc:403] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:26.177932  1682 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:26.179201  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:26.181654  2305 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:26.186789  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:26.187408  2300 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:26.191179  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:26.191993  6827 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:26.192358  6827 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:26.178949  6632 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.724s	user 0.056s	sys 0.060s
I20260709 13:03:26.196439  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.193554  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:26.195724  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:26.186271  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:26.181391  6991 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:26.196669  6632 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.745s	user 0.056s	sys 0.060s
I20260709 13:03:26.183955  6570 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:03:26.208534  1641 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:26.227790  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:26.192752  6827 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:26.185652  6864 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:26.264287  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:26.268290  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.195001  6876 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:26.270661  1681 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:26.179519  6915 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.182710  6728 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 13:03:26.219161  6712 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:26.283787  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.284621  1635 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:26.285248  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.225261  7008 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:03:26.301534  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:26.286947  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:26.268710  6637 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:26.310137  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 117813us
I20260709 13:03:26.310267  2297 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:26.268882  6864 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:26.314096  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.314715  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:26.316606  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.319947  1640 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
W20260709 13:03:26.322048  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 122430us
I20260709 13:03:26.277681  6896 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:26.274192  6976 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:03:26.278224  6712 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:26.309733  6617 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.314193  6732 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:26.323764  6864 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:26.363916  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.277978  6876 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:26.370185  6915 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:26.371997  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 185806us
I20260709 13:03:26.383628  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:26.383873  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.375183  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.384562  1639 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:26.303735  6642 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:03:26.335091  6977 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:26.354662  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:26.388737  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 74092us
I20260709 13:03:26.319011  6897 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:03:26.403339  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:26.404002  1639 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:26.405287  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.393172  1641 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 8.
I20260709 13:03:26.391587  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:26.358444  6712 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:26.410392  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:26.414477  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:26.465863  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 51453.3us
I20260709 13:03:26.467259  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:26.390256  6860 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:26.469254  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 51529.4us
I20260709 13:03:26.388270  6915 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:26.473701  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:26.362798  6617 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:26.475982  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:26.375226  6722 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.486783  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:26.415186  6976 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:26.387775  6393 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.496528  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:26.498124  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.472819  6860 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.476437  6732 raft_consensus.cc:3055] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:26.503973  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:26.476709  6794 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:26.502085  6745 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.505767  6732 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:26.512991  6732 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:26.491487  6632 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.472944  7024 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:03:26.469080  6823 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:03:26.506258  6860 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.514607  6732 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:26.488751  6519 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:26.499600  6950 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:26.518327  2299 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 15: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:26.486044  6977 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:26.530465  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.499413  6271 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.516271  6617 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:26.517732  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:26.486382  6915 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.516934  6976 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:26.556730  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 904) took 2734 ms. Trace:
I20260709 13:03:26.504868  6991 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:26.559970  1560 rpcz_store.cc:276] 0709 13:03:23.807188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:23.807463 (+   275us) service_pool.cc:224] Handling call
0709 13:03:23.824715 (+ 17252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:24.367761 (+543046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:26.537303 (+2169542us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b5400217354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:26.541854 (+  4551us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244870016}
I20260709 13:03:26.537997  6871 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:26.564874  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 47241.6us
I20260709 13:03:26.506963  6821 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:03:26.536209  6632 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:26.569736  7033 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.570744  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:26.575124  6977 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.575395  6950 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:26.575525  7031 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:26.560724  7024 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:26.560732  6519 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:26.579301  6991 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:26.607784  1640 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:26.564330  6871 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.579396  7033 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.604676  6393 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.606230  7037 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:26.559489  6271 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.586778  2300 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:26.585964  1639 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
W20260709 13:03:26.587541  7019 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:03:26.637868  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:26.627326  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:26.578986  6950 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:26.631268  2305 raft_consensus.cc:1240] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:26.632579  2300 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:26.575395  7004 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.652982  1561 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 15: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:26.660691  7030 log.cc:927] Time spent T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:03:26.661592  7033 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:26.581557  7031 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.623618  6876 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:26.669986  1560 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:26.666674  2300 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:03:26.648299  6784 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.166s	user 0.039s	sys 0.025s
I20260709 13:03:26.669574  1563 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:26.673024  6784 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.191s	user 0.039s	sys 0.025s
I20260709 13:03:26.611912  6538 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.681295  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:26.682549  1561 raft_consensus.cc:2410] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:03:26.634289  6880 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:03:26.662317  7035 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:26.694229  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:26.607384  6519 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.695294  1712 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:26.696530  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:26.697485  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:26.648623  6801 consensus_queue.cc:1059] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:03:26.671864  7032 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:26.677989  6893 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.193s	user 0.030s	sys 0.055s
I20260709 13:03:26.699528  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:26.677552  6722 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.649084  6976 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:03:26.636225  7037 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.704360  6821 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:26.723232  6821 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.725847  6821 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.726981  1459 catalog_manager.cc:5697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 7 to 13, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 13 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:26.728098  1643 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:26.700802  7035 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.733012  7035 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:26.704993  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:26.705207  6538 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.705921  6271 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:26.706130  1717 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:03:26.706651  7004 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.707227  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:26.708039  7025 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:03:26.700039  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:26.747120  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:26.711854  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:26.736552  6733 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.237s	user 0.039s	sys 0.010s
I20260709 13:03:26.719283  7031 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:26.689579  6950 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:26.734658  1714 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
W20260709 13:03:26.709717  6822 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.539s	user 0.009s	sys 0.005s
I20260709 13:03:26.744549  1709 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:26.745146  2305 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:03:26.746138  6821 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.726110  6801 raft_consensus.cc:3055] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:03:26.750571  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 212473us
I20260709 13:03:26.754213  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:26.748339  1711 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 9.
I20260709 13:03:26.733858  6986 consensus_queue.cc:1059] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:03:26.751776  6897 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.750352  6794 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:26.752081  6728 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.756440  7009 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.747782  6632 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:26.746611  6993 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:26.758731  6733 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.259s	user 0.040s	sys 0.011s
W20260709 13:03:26.796305  1637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2821) took 2363 ms (client timeout 1661 ms). Trace:
I20260709 13:03:26.775655  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:26.760106  1713 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:26.749652  6570 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15.
I20260709 13:03:26.775333  6393 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:26.785933  1714 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:03:26.734176  7032 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:26.710745  6893 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.226s	user 0.030s	sys 0.055s
I20260709 13:03:26.793792  1636 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:26.803917  2298 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:26.759078  7037 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:26.796763  1637 rpcz_store.cc:269] 0709 13:03:24.428188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.430769 (+  2581us) service_pool.cc:224] Handling call
0709 13:03:26.790758 (+2359989us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b5400195ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:26.790983 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69999488}
I20260709 13:03:26.774896  6801 raft_consensus.cc:740] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:03:26.711989  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 44678.3us
I20260709 13:03:26.807180  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:26.809890  2305 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:26.814651  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.815442  2300 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:26.816339  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:26.821668  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:26.820663  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:26.805899  6393 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:26.824658  6871 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:26.826229  6393 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:26.827421  1459 catalog_manager.cc:5697] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 7 to 15, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 15 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:26.829099  7024 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:26.830024  7024 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:26.829555  1459 catalog_manager.cc:5697] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 9 to 11, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:26.823487  6952 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:26.837677  6952 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.838413  6993 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:26.839918  6993 consensus_queue.cc:1059] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:26.783463  6821 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:26.843283  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:26.844190  1714 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:26.794524  6728 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.795037  6794 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.866449  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.870584  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:26.869617  1534 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:26.822851  6868 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.867661  2305 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:26.878846  2301 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:26.801743  6897 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.881234  2300 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 16: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:26.881546  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:26.877471  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:26.797211  7009 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.885690  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.891312  1715 raft_consensus.cc:2393] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 13: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:26.893684  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:26.819872  6668 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
W20260709 13:03:26.896709  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 423149us
I20260709 13:03:26.896860  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:26.856534  6952 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.872382  6880 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.895970  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.885901  6393 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:26.885496  6955 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.902344  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:26.903578  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.904712  1714 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:26.905280  2300 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:26.897642  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:26.889485  6637 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.893389  6934 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:26.908703  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 141573us
I20260709 13:03:26.913151  1609 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:26.911713  1533 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:26.915342  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.886650  6868 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:26.905589  6827 consensus_meta.cc:287] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.001s
I20260709 13:03:26.891857  6712 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:26.917050  1534 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:26.909142  6864 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.001s
I20260709 13:03:26.921407  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:26.931438  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:26.908568  6393 raft_consensus.cc:697] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:26.910370  7054 raft_consensus.cc:3055] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:26.938467  1533 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:26.918776  6952 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:26.931064  6827 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.940793  7009 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.903362  6880 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.919627  6993 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:26.943459  6733 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:26.988754  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:26.989058  6733 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.989723  6733 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.990178  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.943697  6712 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:26.908872  6955 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.993031  6955 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.944206  1712 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:26.944706  7042 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:03:26.945227  7055 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:26.994266  7055 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.994758  7055 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:26.904306  6801 consensus_queue.cc:260] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:26.996150  6979 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 1.437s	user 0.005s	sys 0.010s
I20260709 13:03:26.997709  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2422) took 2595 ms. Trace:
I20260709 13:03:26.945648  7054 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:26.999799  6897 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:26.951085  7043 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 13:03:26.951606  1463 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45150 (request call id 326) took 37042 ms (client timeout 15000 ms). Trace:
I20260709 13:03:26.951726  7053 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:03:26.952842  7046 log.cc:927] Time spent T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:03:26.998095  1638 rpcz_store.cc:276] 0709 13:03:24.401888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.414958 (+ 13070us) service_pool.cc:224] Handling call
0709 13:03:24.490371 (+ 75413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:24.620587 (+130216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:24.620635 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:24.647071 (+ 26436us) log.cc:844] Serialized 26 byte log entry
0709 13:03:25.089782 (+442711us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:25.089839 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:25.089951 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:26.995422 (+1905471us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:26.997335 (+  1913us) raft_consensus.cc:1685] finished
0709 13:03:26.997360 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:26.997618 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":130652,"raft.queue_time_us":142159,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":13398,"spinlock_wait_cycles":64265984,"thread_start_us":316166,"threads_started":1,"wal-append.queue_time_us":717026}
W20260709 13:03:26.961678  7047 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:03:26.963344  6794 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.964818  6821 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:26.971426  6868 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:26.944319  6860 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.974025  1459 catalog_manager.cc:5697] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 8 to 11, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 11 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.977600  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.982785  6952 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.979712  6728 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:26.979950  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:27.031510  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.981196  1637 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:03:26.981922  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:26.984489  7051 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:26.987085  6893 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:26.999668  7054 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:26.986131  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.042249  1714 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:27.008741  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:27.042793  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 116103us
W20260709 13:03:27.008739  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 131388us
I20260709 13:03:27.044591  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:26.939337  6822 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:26.919601  6934 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:27.049373  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:26.972805  6935 raft_consensus.cc:3055] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:26.917200  6637 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.024775  2296 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:27.018486  2269 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:27.026393  1639 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:26.977196  6393 consensus_queue.cc:237] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.027829  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:27.016842  7053 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.988795  6993 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:26.936558  6864 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:27.052759  2300 raft_consensus.cc:2368] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:27.036545  6952 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.058025  2269 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:27.060271  1638 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:27.067443  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:27.068488  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 151765us
I20260709 13:03:27.070003  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:27.070828  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.080709  2269 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:27.029395  6794 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:03:27.047775  6822 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:27.052700  6935 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:27.079779  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.081615  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:27.082960  2304 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 16: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:27.062206  6784 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:27.098301  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:27.066044  6993 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:27.099454  1718 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:27.099637  1641 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 12: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:03:27.093765  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 49243.9us
I20260709 13:03:27.063673  7053 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:27.104797  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:27.101342  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.097863  2269 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:27.111515  1710 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:27.116413  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:27.088258  6668 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:27.119274  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:27.122641  1638 raft_consensus.cc:2393] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 15: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:27.120330  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.120559  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:27.095309  6822 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:27.096155  6519 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:27.126495  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.127027  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:27.128927  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:27.129114  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.110410  7051 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:27.130507  1609 leader_election.cc:400] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:27.134339  1609 leader_election.cc:403] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:27.135269  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:27.136665  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.088106  6712 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:03:27.089205  7037 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:03:27.144474  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.119167  6271 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:27.139410  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:27.146360  6827 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:27.099166  6897 consensus_queue.cc:1059] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:03:27.146237  7059 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:03:27.147207  7056 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:27.000833  1463 rpcz_store.cc:269] 0709 13:02:49.829409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.857548 (+ 28139us) service_pool.cc:224] Handling call
0709 13:03:26.868313 (+37010765us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:19.724702 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:19.725375 (+   673us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:19.725425 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:03:19.725448 (+    23us) tablet.cc:662] Decoding operations
0709 13:03:19.726873 (+  1425us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:19.771190 (+ 44317us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:19.773173 (+  1983us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:03:20.140065 (+366892us) tablet.cc:701] Row locks acquired
0709 13:03:20.167005 (+ 26940us) write_op.cc:260] PREPARE: finished
0709 13:03:20.167180 (+   175us) write_op.cc:270] Start()
0709 13:03:20.167352 (+   172us) write_op.cc:276] Timestamp: P: 1783602200167117 usec, L: 0
0709 13:03:20.167391 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:03:20.168221 (+   830us) log.cc:844] Serialized 6366 byte log entry
0709 13:03:20.724883 (+556662us) op_driver.cc:464] REPLICATION: finished
0709 13:03:20.789912 (+ 65029us) write_op.cc:301] APPLY: starting
0709 13:03:20.790016 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:21.311092 (+521076us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:21.330010 (+ 18918us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:22.178069 (+848059us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:22.259097 (+ 81028us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 13:03:22.259189 (+    92us) write_op.cc:312] APPLY: finished
0709 13:03:22.259668 (+   479us) log.cc:844] Serialized 148 byte log entry
0709 13:03:22.363931 (+104263us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:22.364326 (+   395us) write_op.cc:454] Released schema lock
0709 13:03:22.364456 (+   130us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:22.365575 (+  1119us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:22.365728 (+   153us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4877599,"reported_tablets":46,"spinlock_wait_cycles":5632,"child_traces":[["op",{"apply.queue_time_us":57557,"apply.run_cpu_time_us":60997,"apply.run_wall_time_us":1575534,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71039,"num_ops":16,"prepare.queue_time_us":276945,"prepare.run_cpu_time_us":39407,"prepare.run_wall_time_us":648990,"raft.queue_time_us":68906,"raft.run_cpu_time_us":37177,"raft.run_wall_time_us":65647,"replication_time_us":557277,"spinlock_wait_cycles":869760,"thread_start_us":531873,"threads_started":5,"wal-append.queue_time_us":578971,"wal-append.run_cpu_time_us":23091,"wal-append.run_wall_time_us":2237215}]]}
I20260709 13:03:27.141132  6823 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.163750  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.170325  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.174814  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.176764  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:27.140859  6632 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:27.176697  6823 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.185745  6823 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:27.186291  7061 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:03:27.188100  7004 raft_consensus.cc:3055] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:27.188517  7004 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:27.189334  7004 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.190383  7004 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:27.191318  6896 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:27.191196  6632 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.193506  6915 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:27.181123  6897 consensus_queue.cc:1243] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:03:27.197099  6732 consensus_meta.cc:287] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.655s	user 0.006s	sys 0.007s
I20260709 13:03:27.198518  7067 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:27.201274  1713 raft_consensus.cc:2410] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:03:27.183472  7060 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:03:27.151906  6271 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:27.215801  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.211755  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.212024  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:27.212242  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:27.199209  7067 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.163304  6716 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:27.219550  2304 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:27.194520  6632 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:27.220875  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:27.198763  6745 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:27.227439  7055 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:27.231114  6617 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:27.231654  6617 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.119089  1459 catalog_manager.cc:5697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:27.232065  6617 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:27.232882  1459 catalog_manager.cc:5697] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:27.174207  7056 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.235412  6880 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:27.236655  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:27.236908  7056 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:27.236989  6934 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:27.237636  7055 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:27.238653  1637 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:27.244656  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:27.242261  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:27.218456  6271 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:27.220255  6821 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.176218  6947 consensus_queue.cc:1059] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:03:27.235430  6733 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.247663  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.252233  6946 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.268841  1643 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:27.269750  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.271901  1716 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:27.275317  7051 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:27.276677  7067 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:27.289291  1635 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
W20260709 13:03:27.289494  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.247180  6821 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:27.305348  6995 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:27.306751  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.307457  1718 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:27.308298  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.308761  1718 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.310931  1717 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:27.393112  1635 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:03:27.376750  1637 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:27.426617  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.436780  1716 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.272902  6946 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.296088  6952 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:27.474676  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.501679  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1350) took 1388 ms. Trace:
I20260709 13:03:27.502254  2299 rpcz_store.cc:276] 0709 13:03:26.104813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.104939 (+   126us) service_pool.cc:224] Handling call
0709 13:03:27.493038 (+1388099us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b5400368b54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:27.493173 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":945048320}
I20260709 13:03:27.509439  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:27.511528  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 381184us
W20260709 13:03:27.512837  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 408173us
I20260709 13:03:27.513427  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:27.516842  6976 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:03:27.517374  6976 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:27.517809  6976 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:27.518095  6617 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.520725  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:27.520927  1711 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:27.524016  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.524739  1533 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:27.525594  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.527654  1533 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:27.608489  6984 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:03:27.608654  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 81081.6us
I20260709 13:03:27.273533  1459 catalog_manager.cc:5697] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 7 to 9, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:27.610198  1711 raft_consensus.cc:2376] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15. Re-sending same reply.
W20260709 13:03:27.613130  2297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1341) took 1560 ms (client timeout 1795 ms). Trace:
I20260709 13:03:27.482296  7074 consensus_queue.cc:1059] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.173s
I20260709 13:03:27.613837  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.615141  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.616110  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.616731  7032 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:27.617107  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.617699  1711 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:27.618497  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.523917  6632 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:27.613629  2297 rpcz_store.cc:269] 0709 13:03:26.052382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.052540 (+   158us) service_pool.cc:224] Handling call
0709 13:03:27.612024 (+1559484us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400643f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:27.613068 (+  1044us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562479488}
I20260709 13:03:27.621479  6821 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:27.622352  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1354) took 1148 ms. Trace:
I20260709 13:03:27.623176  2302 rpcz_store.cc:276] 0709 13:03:26.474296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.474424 (+   128us) service_pool.cc:224] Handling call
0709 13:03:26.474700 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.571479 (+ 96779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:26.571514 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:26.571558 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:26.571591 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.622142 (+1050551us) spinlock_profiling.cc:243] Waited 96.6 ms on lock 0x7b54004998d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:27.622184 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.622297 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222095744}
I20260709 13:03:27.523677  6716 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:27.526798  6823 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:27.609918  1459 catalog_manager.cc:5697] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 4 to 11. New cstate: current_term: 11 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:27.553344  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.631381  1717 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:27.630852  1459 catalog_manager.cc:5697] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:27.632205  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.633472  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.632953  1459 catalog_manager.cc:5697] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
W20260709 13:03:27.607532  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 463357us
I20260709 13:03:27.611936  6975 raft_consensus.cc:1064] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:03:27.612505  6868 raft_consensus.cc:1064] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:03:27.612372  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.605356  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:27.619138  6632 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:27.626005  7053 consensus_meta.cc:287] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.507s	user 0.017s	sys 0.000s
I20260709 13:03:27.519768  7055 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:27.627863  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:27.606277  7070 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.180s	user 0.002s	sys 0.006s
I20260709 13:03:27.641789  6716 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:03:27.652580  7057 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:27.653280  7057 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:27.611588  6984 raft_consensus.cc:1064] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:03:27.657750  1716 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:27.643260  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:27.644580  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:27.645325  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.640686  1683 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:27.661208  1532 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:27.646548  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:27.636730  6993 consensus_meta.cc:287] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.520s	user 0.003s	sys 0.008s
I20260709 13:03:27.634572  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.658320  1711 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:27.659145  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:27.662439  1636 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:27.663265  1710 raft_consensus.cc:2393] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 13: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:03:27.664170  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.665213  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.666016  1716 raft_consensus.cc:2376] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15. Re-sending same reply.
I20260709 13:03:27.670058  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.670910  1710 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:03:27.671501  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:27.673012  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.674038  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:27.674691  1636 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:03:27.676040  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.679661  1683 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:27.676751  1716 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 9.
I20260709 13:03:27.681308  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.682029  1683 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:27.688274  1683 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:27.687100  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.677464  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:27.701215  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.702046  1716 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:27.704623  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.663404  7069 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:27.671368  6722 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:27.705353  1715 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:27.708751  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:27.678871  6822 consensus_meta.cc:287] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.000s
I20260709 13:03:27.713064  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:27.718083  1717 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:27.677003  6716 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:27.655331  7053 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:27.685973  7064 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:27.704165  7074 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:27.724948  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:27.698473  7054 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:27.723687  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:27.762455  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:27.697836  7004 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:27.735338  1716 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:27.657575  7055 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.724404  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:27.757004  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 95879.7us
I20260709 13:03:27.759059  1636 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:27.727586  6822 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.703043  6775 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:27.671016  6993 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:27.767290  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:27.788331  2301 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:27.770385  1716 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:27.790096  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:27.717455  6722 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.771106  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:27.714810  7071 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:03:27.805871  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 136224us
I20260709 13:03:27.807348  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:27.807930  1643 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.810002  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:27.810850  1636 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:03:27.776538  1683 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:27.783430  2300 raft_consensus.cc:1240] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:27.739899  6728 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:27.634907  1459 catalog_manager.cc:5697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:27.790828  1716 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.792496  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 1220) took 1640 ms. Trace:
I20260709 13:03:27.795848  1715 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:27.797250  6822 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:27.803076  6617 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:27.803802  6946 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:27.768600  6952 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:27.815810  1718 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:27.823688  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:27.804077  6733 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:27.719794  7069 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:27.741015  6947 consensus_queue.cc:1059] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:03:27.873078  1716 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:27.876199  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:27.880646  6716 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:27.829821  2303 rpcz_store.cc:276] 0709 13:03:26.151941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.155765 (+  3824us) service_pool.cc:224] Handling call
0709 13:03:27.612149 (+1456384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.612344 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.612368 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.612407 (+    39us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:27.612437 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.612492 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:27.792343 (+179851us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400643f50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:27.792448 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3348857600}
I20260709 13:03:27.883052  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:27.805172  6722 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:27.805464  7071 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.887354  7071 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:27.887598  7080 log.cc:927] Time spent T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:03:27.891330  6733 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:27.891969  6733 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.894210  6733 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:27.897128  6722 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:27.881232  6822 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:27.898368  6822 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:27.900725  6822 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:27.901279  6822 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.901734  6822 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.878756  1715 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:03:27.871006  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 47448.2us
I20260709 13:03:27.906903  6822 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:27.765082  7064 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:27.825835  6775 raft_consensus.cc:697] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:27.840333  1636 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:27.832420  6530 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:27.874859  1716 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:27.918865  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:27.952850  1718 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:03:27.956151  1459 catalog_manager.cc:5697] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:03:27.961043  6271 consensus_meta.cc:287] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.714s	user 0.009s	sys 0.000s
I20260709 13:03:27.961436  1459 catalog_manager.cc:5697] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 6 to 14. New cstate: current_term: 14 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:27.963680  1459 catalog_manager.cc:5697] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 10 to 12, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 12 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:27.965994  1459 catalog_manager.cc:5697] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 8 to 12, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 12 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:27.968214  1718 pending_rounds.cc:85] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 10
I20260709 13:03:27.968691  1718 pending_rounds.cc:107] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:03:27.969013  1718 raft_consensus.cc:2889] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:27.942687  1716 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:03:27.977420  1532 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:27.959995  7055 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.011591  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1043) took 1102 ms. Trace:
I20260709 13:03:28.013784  1562 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:03:28.015659  1564 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:28.016083  1561 raft_consensus.cc:1240] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:28.019711  1561 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:28.012125  1709 rpcz_store.cc:276] 0709 13:03:26.909530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.799882 (+890352us) service_pool.cc:224] Handling call
0709 13:03:27.800250 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.800565 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:27.800884 (+   319us) log.cc:844] Serialized 14 byte log entry
0709 13:03:28.011037 (+210153us) log.cc:844] Serialized 14 byte log entry
0709 13:03:28.011241 (+   204us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:28.011289 (+    48us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:28.011328 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.011398 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.011533 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28602,"prepare.queue_time_us":4937,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":60,"raft.queue_time_us":36419,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":82,"thread_start_us":173914,"threads_started":1}
W20260709 13:03:28.026120  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 48776.1us
I20260709 13:03:28.028810  1533 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:28.031503  1563 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:28.027482  1532 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:28.033823  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:27.967813  1459 catalog_manager.cc:5697] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 8 to 11, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 11 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:28.038632  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.040305  1561 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:27.983312  6632 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.041978  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:27.981132  7083 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:27.975656  6271 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.042762  1562 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:28.023587  7009 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:28.048794  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.046446  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.047206  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2384) took 1275 ms. Trace:
I20260709 13:03:28.050166  1712 rpcz_store.cc:276] 0709 13:03:26.771582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.800528 (+ 28946us) service_pool.cc:224] Handling call
0709 13:03:26.811573 (+ 11045us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.993715 (+182142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.046956 (+1053241us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54001c0f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:28.047145 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304598912}
I20260709 13:03:28.051491  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.054196  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.038195  7055 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.037899  6952 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:28.047684  6976 consensus_meta.cc:287] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.000s
I20260709 13:03:28.042516  7086 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.055372  6632 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:28.068708  6993 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:28.048996  6823 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:28.049228  7087 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:28.018862  6775 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.060541  7053 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:28.086686  7081 log.cc:927] Time spent T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:03:28.092041  7064 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.100580  1643 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:28.061350  6976 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:28.071295  7086 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.106606  6722 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:28.104444  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 77075.2us
I20260709 13:03:28.122632  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:28.123121  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:28.123590  7086 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.124750  2268 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:28.125602  7008 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:28.131491  2268 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:28.132243  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.133028  2268 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:28.133770  6827 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.134266  6827 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.136581  6827 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:28.080612  7087 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.137130  6827 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:28.137408  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1036) took 1248 ms. Trace:
I20260709 13:03:28.137801  1710 rpcz_store.cc:276] 0709 13:03:26.888674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.769393 (+880719us) service_pool.cc:224] Handling call
0709 13:03:27.769682 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.842744 (+ 73062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.842794 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:27.842852 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:27.842897 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.137164 (+294267us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b5400136ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:28.137224 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.137353 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167400192}
I20260709 13:03:28.139209  7087 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.143922  2304 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:28.145334  6935 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.151039  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:28.152114  6712 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:28.152555  6712 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:28.153337  6712 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.158520  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.158988  1568 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:28.159965  6801 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:28.146721  6935 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:28.163481  6637 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.153363  1564 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:03:28.114594  7069 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:28.154820  6712 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:28.168109  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:28.169492  7037 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:28.169736  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1495) took 1543 ms. Trace:
I20260709 13:03:28.170167  2298 rpcz_store.cc:276] 0709 13:03:26.626943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.627065 (+   122us) service_pool.cc:224] Handling call
0709 13:03:26.627364 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.804382 (+177018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.169516 (+1365134us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54002a4254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:28.169679 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405178752}
I20260709 13:03:28.171260  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:28.172443  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:28.173091  2303 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:28.174026  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:28.174626  2301 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:28.176267  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:28.176776  6934 raft_consensus.cc:3055] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:28.177220  6934 raft_consensus.cc:740] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:28.178068  6934 consensus_queue.cc:260] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.179199  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:28.179889  1641 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:28.180891  6801 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:28.181012  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2849) took 1108 ms. Trace:
I20260709 13:03:28.181479  6801 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:28.183749  6801 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:28.186208  6728 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.187633  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2431) took 1913 ms. Trace:
I20260709 13:03:28.105881  1459 catalog_manager.cc:5697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 1 to 9. New cstate: current_term: 9 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:28.188045  1642 rpcz_store.cc:276] 0709 13:03:26.274491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.274894 (+   403us) service_pool.cc:224] Handling call
0709 13:03:26.275312 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:26.407944 (+132632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:26.407994 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:26.408048 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:26.408088 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.187381 (+1779293us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54002d7b54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:28.187433 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.187568 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304462720}
W20260709 13:03:28.189375  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.189003  1459 catalog_manager.cc:5697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 5 to 10. New cstate: current_term: 10 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:03:28.189905  1713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2386) took 1387 ms (client timeout 1594 ms). Trace:
I20260709 13:03:28.190222  1683 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:28.191253  6896 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:28.115744  6860 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:28.191639  6896 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.063051  7055 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.059314  7083 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.192059  6896 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:28.117446  7032 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.194420  7032 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.196085  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:28.196797  7032 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:28.198900  7083 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.200230  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1048) took 1273 ms. Trace:
I20260709 13:03:28.200610  1718 rpcz_store.cc:276] 0709 13:03:26.926399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.844432 (+918033us) service_pool.cc:224] Handling call
0709 13:03:27.844731 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:27.969719 (+124988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.199998 (+230279us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54001c1454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:28.200171 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242218368}
I20260709 13:03:28.113114  6271 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:28.160154  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:28.201771  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:28.201962  1710 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:28.202059  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:28.202670  2304 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:28.202865  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:28.204393  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:28.204875  1710 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:28.205181  7060 consensus_meta.cc:287] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.981s	user 0.005s	sys 0.007s
I20260709 13:03:28.206123  6959 consensus_queue.cc:1059] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:28.121085  6976 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:28.190328  1713 rpcz_store.cc:269] 0709 13:03:26.802122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:26.808173 (+  6051us) service_pool.cc:224] Handling call
0709 13:03:28.189650 (+1381477us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54001365d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:28.189827 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":867081984}
I20260709 13:03:28.181489  1641 rpcz_store.cc:276] 0709 13:03:27.072656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.215380 (+142724us) service_pool.cc:224] Handling call
0709 13:03:28.180846 (+965466us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b54001e9954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:28.180952 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2148513664}
W20260709 13:03:28.219964  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.220117  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:28.220805  1566 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.221556  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:28.223592  2301 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:28.226900  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:28.227458  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 110192us
I20260709 13:03:28.228076  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:28.231966  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:28.236268  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:28.236739  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:28.224403  1459 catalog_manager.cc:5697] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:28.259940  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:28.349278  1718 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:28.212884  7060 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:28.227414  6271 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:28.355374  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51929.1us
I20260709 13:03:28.232391  6637 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:28.320758  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:28.320734  1569 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:28.322541  1642 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:03:28.233965  7057 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:28.338464  1638 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
W20260709 13:03:28.340891  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 218324us
I20260709 13:03:28.387537  1532 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:28.345924  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:28.344564  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:28.389230  2301 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:28.349303  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:28.352857  1639 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:28.363380  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:28.365303  1641 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:28.389259  2304 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:28.357154  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:28.400945  1637 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 15: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:28.401391  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 44302us
W20260709 13:03:28.386281  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 154365us
I20260709 13:03:28.404349  1532 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:28.406025  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:28.409546  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:28.400763  1713 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 15: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:28.402577  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:28.411676  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:28.400822  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 45956.1us
I20260709 13:03:28.412935  1566 raft_consensus.cc:2410] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 16 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:03:28.416175  2303 raft_consensus.cc:2393] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 15: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:28.417867  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.350708  7008 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:28.419764  2269 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:28.418227  1710 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:28.186813  6728 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:28.355573  6821 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:28.348266  6871 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:28.354907  7086 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:03:28.430603  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 393116us
W20260709 13:03:28.401427  7097 log.cc:927] Time spent T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.154s	user 0.003s	sys 0.002s
W20260709 13:03:28.467777  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.467700  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:28.457746  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:28.475297  1718 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:03:28.475777  1533 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:28.480007  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 74010.4us
I20260709 13:03:28.416056  6733 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:28.480847  1564 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:28.483770  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:28.484186  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:28.478310  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:28.424686  6617 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:28.469111  6821 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.488190  7073 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:28.456859  7057 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:28.464792  7055 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.464241  7060 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:28.490216  6952 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:28.460616  7095 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:28.411595  6271 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.493121  7086 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:28.494952  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.497413  7049 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:28.482079  6871 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.498821  6716 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:28.499877  2268 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.620s (SENT) [suppressed 14 similar messages]
I20260709 13:03:28.501652  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.524752  6617 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:28.507445  7099 log.cc:927] Time spent T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
W20260709 13:03:28.462090  7081 log.cc:927] Time spent T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:03:28.483146  1532 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:28.481904  7083 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:28.462272  6822 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:28.456175  7053 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:28.522985  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.462138  6947 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:03:28.525465  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:28.592149  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.466054  6827 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:28.523505  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2459) took 1523 ms. Trace:
I20260709 13:03:28.572938  6617 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:28.526866  1635 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:28.383523  7037 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.130s
I20260709 13:03:28.485117  7064 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.530646  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:28.534192  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:28.463862  7036 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 13:03:28.490118  6896 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.521464  6990 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.412045  7009 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.119s
I20260709 13:03:28.525839  6632 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:28.531100  6821 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.544591  1561 raft_consensus.cc:2393] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 15: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:28.504900  6864 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:28.624596  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:28.624775  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:28.625672  1642 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:03:28.547839  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.531886  7087 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:28.532661  6950 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:28.561820  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:28.654969  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.562680  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:28.658010  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 108086us
W20260709 13:03:28.541476  7074 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.775s	user 0.009s	sys 0.004s
I20260709 13:03:28.547240  7049 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.654771  6632 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:28.580904  1709 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:28.543695  6952 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:28.596241  1714 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 10.
I20260709 13:03:28.526955  1566 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:28.609022  1718 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:28.598083  1715 rpcz_store.cc:276] 0709 13:03:27.000233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.903724 (+903491us) service_pool.cc:224] Handling call
0709 13:03:27.904116 (+   392us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:27.918738 (+ 14622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.918794 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:27.924200 (+  5406us) log.cc:844] Serialized 52 byte log entry
0709 13:03:28.156981 (+232781us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b18004cd8c0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:03:28.414137 (+257156us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:28.462639 (+ 48502us) log.cc:844] Serialized 14 byte log entry
0709 13:03:28.465833 (+  3194us) log.cc:844] Serialized 14 byte log entry
0709 13:03:28.486714 (+ 20881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.486808 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.523201 (+ 36393us) raft_consensus.cc:1685] finished
0709 13:03:28.523234 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.523410 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18513,"prepare.queue_time_us":132354,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":67,"raft.queue_time_us":62351,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":27168,"spinlock_wait_cycles":48047872,"thread_start_us":256908,"threads_started":1,"wal-append.queue_time_us":538772}
I20260709 13:03:28.608349  1713 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:28.674873  1711 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:28.504925  6975 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.581517  6822 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:28.655175  7082 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.606948  6794 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:28.603904  6947 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:28.619315  6990 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:28.620613  7009 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:28.614280  7064 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.622167  6951 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:28.627126  7037 raft_consensus.cc:3055] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:28.604995  7083 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:28.604709  6733 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:28.610404  7086 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:28.631735  2303 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:28.634418  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:28.634843  7054 consensus_meta.cc:287] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.854s	user 0.011s	sys 0.003s
I20260709 13:03:28.684238  6822 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:28.637732  7071 consensus_meta.cc:287] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.750s	user 0.008s	sys 0.005s
I20260709 13:03:28.639176  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2404) took 1007 ms. Trace:
W20260709 13:03:28.711289  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 291645us
I20260709 13:03:28.639899  6934 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:28.540745  7060 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:28.599575  7055 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:28.644121  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 161070us
I20260709 13:03:28.647301  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:28.607324  6995 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:28.722621  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:28.660249  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.630578  6896 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.536427  7095 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.613914  6716 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.651691  7087 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.654001  7053 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:28.681003  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.733804  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.673825  6530 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:28.690593  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 203101us
I20260709 13:03:28.668852  6976 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:28.695338  1715 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 13. Ops: []
I20260709 13:03:28.734725  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:28.670274  7074 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:28.672639  6728 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:28.635555  7051 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:28.763132  7083 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:28.764197  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.709548  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.696316  6951 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:28.694614  7009 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:28.727206  6952 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.732645  7087 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.767680  6951 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.767747  7009 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.769042  6951 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.769642  7009 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:28.766453  6821 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:28.714782  1643 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:28.696709  7037 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:28.772744  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:28.711833  1716 rpcz_store.cc:276] 0709 13:03:27.631622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.970544 (+338922us) service_pool.cc:224] Handling call
0709 13:03:27.970907 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.971442 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.971474 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.972033 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:03:28.221216 (+249183us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:28.221257 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.221329 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.638902 (+417573us) raft_consensus.cc:1685] finished
0709 13:03:28.638942 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.639106 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4888,"raft.queue_time_us":4422,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":9071,"thread_start_us":244122,"threads_started":1,"wal-append.queue_time_us":237236}
I20260709 13:03:28.767771  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:28.692528  6794 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.731096  1639 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:28.785257  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.786054  1567 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
W20260709 13:03:28.786445  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 127406us
I20260709 13:03:28.730516  6716 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.692852  7082 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.751399  6530 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.753139  6990 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:28.754019  7100 log.cc:927] Time spent T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:03:28.754765  7104 log.cc:927] Time spent T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:03:28.710880  6733 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.821173  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:28.821559  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.756371  7064 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.757033  6947 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:28.729239  7095 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.759069  7060 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:28.759816  6271 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:28.760260  6896 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:28.718724  7071 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.727247  7055 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.749359  7054 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:03:28.724074  6827 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:03:28.749563  6801 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.763751  7051 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:28.763967  7049 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.726496  1532 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:28.718257  2269 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:03:28.846063  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 119643us
W20260709 13:03:28.846274  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 128124us
I20260709 13:03:28.847497  1717 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 12: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:28.847765  2269 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:28.776604  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:28.747148  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 247300us
I20260709 13:03:28.788789  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:28.790478  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.802685  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:28.851728  1560 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.764067  6871 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:28.859412  6871 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:28.861014  6871 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:28.861606  6871 raft_consensus.cc:697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:28.815351  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:28.863927  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:28.865195  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:28.865787  1641 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:28.818835  1642 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:28.867869  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:28.867544  6801 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.869282  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:28.869863  1642 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:28.826226  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2405) took 1194 ms. Trace:
I20260709 13:03:28.870438  1709 rpcz_store.cc:276] 0709 13:03:27.632171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.025182 (+393011us) service_pool.cc:224] Handling call
0709 13:03:28.038672 (+ 13490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.669962 (+631290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:28.825935 (+155973us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54004051d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:28.826159 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1246664448}
I20260709 13:03:28.782266  6876 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.834030  6896 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.717952  6632 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:28.835439   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:28.832577  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 176085us
I20260709 13:03:28.853329  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:28.901572  2301 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:28.853911  7032 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:28.857154  6733 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.869894  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:28.872260  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 83564.5us
I20260709 13:03:28.876015  1718 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:03:28.877898  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:28.820179  1639 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:03:28.881438  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.884217  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:28.883638  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:28.887882  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.878664  1637 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:03:28.898428  1642 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
W20260709 13:03:28.898986  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 51179us
I20260709 13:03:28.882946  1533 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:28.841842  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:28.903956  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.909902  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.912168  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.912349  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:28.915256  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:28.916580  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:28.966562  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:28.967897  1567 raft_consensus.cc:2410] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:03:28.970364  1635 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:28.963750  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:28.971802  2298 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
W20260709 13:03:28.971561  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 56400.8us
I20260709 13:03:28.973186  1564 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:28.915172  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:28.969281  1643 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:28.873457  7082 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.972872  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:28.898900  7083 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:28.982448  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:28.981237  1607 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:28.880951  6794 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.873606  6716 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:28.874459  7101 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:03:28.989312  1564 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:28.997464  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:28.998252  1638 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 16: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:28.990896  1643 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:28.999717  1641 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:28.991572  1567 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:28.911654  6896 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:28.992648  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:28.968566  7075 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:28.993528  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 110704us
I20260709 13:03:28.994644  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:28.994801  1569 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:28.978013  1682 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:28.871690  6876 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.984041  6271 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:03:28.886713  6822 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.003018  7049 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:29.003443  7082 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:28.977241  7064 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:03:29.042577  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 64642.5us
I20260709 13:03:29.004487  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:29.004944  1564 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:28.993976  7069 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:28.977671  6955 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:03:28.992511  6947 consensus_queue.cc:1059] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:03:29.008105  7056 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:29.008159  7060 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:29.036414  6976 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:28.999750  1681 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:28.997138  7054 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:03:28.870887  6801 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:29.011917  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:28.875896  7055 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:28.903815  7032 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:29.016242  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.061446  1716 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:28.886339  6821 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:29.017853  1560 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:29.019615  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.063709  1715 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:29.052798  7069 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.064659  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:29.065299  1715 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:29.065804  7069 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:29.050406  7082 raft_consensus.cc:697] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:29.070842  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.071609  1713 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:29.071743  7082 consensus_queue.cc:237] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.072603  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.073802  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:29.074105  7071 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:29.068706  1640 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:29.075881  6946 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:29.076774  7121 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:03:29.059839  7060 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.081171  7060 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:29.084077  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:29.085812  1715 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 10.
I20260709 13:03:28.862643  6871 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.087445  6617 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.089072  1715 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []
I20260709 13:03:29.002034  6745 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:28.986341  7083 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:29.090301  7083 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:29.094004  1718 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
W20260709 13:03:29.036268  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 69850.4us
I20260709 13:03:28.915337  6952 consensus_queue.cc:1059] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.105s
I20260709 13:03:29.095757  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:29.096148  6952 raft_consensus.cc:3055] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:29.096556  6952 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:29.096948  6951 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:29.097355  6952 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:29.098534  6952 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:29.040315  1533 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:29.004019  6995 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.099733  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1067) took 1416 ms. Trace:
I20260709 13:03:29.100149  6995 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.100198  1711 rpcz_store.cc:276] 0709 13:03:27.683576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.640421 (+956845us) service_pool.cc:224] Handling call
0709 13:03:28.674486 (+ 34065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:28.675406 (+   920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.099517 (+424111us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b5400135bd0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:29.099677 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77492992}
I20260709 13:03:29.102129  7109 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:29.102530  7109 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:29.102838  7055 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:29.086716  7095 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.103277  7055 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.103368  1643 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:03:29.103683  7055 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:29.104902  7095 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:29.043360  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:29.107378  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 64097.2us
I20260709 13:03:28.984683  7106 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:03:29.134956  1713 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:29.135970  1561 raft_consensus.cc:2368] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:29.054033  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.011816  7057 consensus_queue.cc:1059] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:03:28.887210  7051 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:29.056648  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 56998.2us
I20260709 13:03:29.148684  1681 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:28.996786  7008 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:03:29.062390  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.051883  6986 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:29.151291  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:29.152211  1715 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:29.152756  7051 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:29.153657  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 45289.1us
I20260709 13:03:29.153761  7109 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.155604  7109 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:29.157407  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.158308  1717 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:29.159462  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.160393  6896 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.161036  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.161638  1717 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:29.162454  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.076418  6946 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.165325  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:28.841542  7037 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:28.885838  7087 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.108717  2301 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:29.037294  1567 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:03:29.109401  2268 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:29.099488  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 59250.8us
I20260709 13:03:29.139900  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.139355  6637 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:29.062951  7032 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:29.146380  7124 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:03:29.146727  6822 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:29.061731  6959 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:29.147418  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:29.138198  6993 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.162614  6896 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:29.026621  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:29.195267  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 168710us
I20260709 13:03:29.179770  6637 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.164813  1644 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 12. Current term is 13. Ops: []
W20260709 13:03:29.166033  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 262853us
I20260709 13:03:29.175386  1711 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:29.196197  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:29.202724  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:29.203944  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:29.175976  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:29.205670  1716 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 16: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:29.176416  1715 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
W20260709 13:03:29.179598  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 70264.1us
I20260709 13:03:29.143246  6860 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:29.185304  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:29.207230  1641 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 16: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:29.189649  1568 raft_consensus.cc:2393] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 16: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:29.187268  6986 raft_consensus.cc:697] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:29.192310  6728 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:29.187538  6959 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.193045  6993 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.163174  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:29.197366  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.353580  1717 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:29.182682  6951 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:29.171096  6934 consensus_queue.cc:1059] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:03:29.213500  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:29.382373  1644 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:03:29.336261  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 126461us
W20260709 13:03:29.378118  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 163387us
I20260709 13:03:29.182058  7083 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.201936  6637 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:29.201778  7049 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:29.386451  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 182602us
I20260709 13:03:29.387867  1532 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:29.191087  6632 consensus_queue.cc:1059] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.159s
I20260709 13:03:29.388908  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:29.210108  6821 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:29.213285  7051 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:03:29.328670  6728 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.393647  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:29.351835  6959 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:29.396727  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:29.416759  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:29.418609  2301 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:03:29.425691  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:29.404811  1644 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:29.411039  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:29.377259  7009 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:29.410229  7114 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:29.386525  7008 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:03:29.412760  6728 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:29.411271  7049 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:29.442435  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:29.463608  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:29.418411  6801 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:29.422147  7051 raft_consensus.cc:697] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:29.421684  7069 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:29.424340  6947 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:03:29.422662  7054 consensus_queue.cc:1059] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
W20260709 13:03:29.507236  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 43763.7us
W20260709 13:03:29.509809  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2448) took 2412 ms (client timeout 2007 ms). Trace:
W20260709 13:03:29.510262  1640 rpcz_store.cc:269] 0709 13:03:27.097376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.116115 (+ 18739us) service_pool.cc:224] Handling call
0709 13:03:29.509619 (+2393504us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b540032bed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:29.509729 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4488511360}
I20260709 13:03:29.511555  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:29.462476  6632 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:29.467778  6871 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:29.521649  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:29.515081  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 118429us
I20260709 13:03:29.511539  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:29.469295  7131 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:03:29.503209  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 86522us
I20260709 13:03:29.469944  6801 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:29.526023  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:29.526296  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:29.527568  1532 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:29.528918  1532 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:29.501541  7125 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:29.531118  1607 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:29.534133  7069 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:29.535080  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:29.535225  6271 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:29.534598  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:29.535709  2301 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:29.536026  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:29.536579  2305 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:29.537966  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:29.538311  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:29.531172  1532 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:29.538949  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.539552  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:29.537988  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.541121  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:29.534873  7126 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:03:29.540369  7085 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:29.522408  7071 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.525156  7060 raft_consensus.cc:3055] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:29.469014  7009 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.529383  6801 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:29.528731  7095 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:29.532867  7114 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:29.554129  1712 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:03:29.543465  7087 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:29.565848  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:29.548866  7082 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:29.543650  7054 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:29.533500  7106 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:03:29.556975  1714 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 1034) took 2673 ms (client timeout 2062 ms). Trace:
I20260709 13:03:29.528967  6934 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:29.548352  6732 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:29.548998  6632 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:03:29.575955  7127 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:03:29.573640  6871 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:29.583087  6947 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:29.585544  7083 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:29.554144  7131 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
W20260709 13:03:29.646250  7130 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.254s	user 0.004s	sys 0.000s
I20260709 13:03:29.560904  7114 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.505491  6986 consensus_queue.cc:237] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.590338  7091 raft_consensus.cc:3055] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:03:29.555190  7049 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.593195  6794 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:29.659267  6947 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.679620  6794 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:03:29.680785  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.680650  6794 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.681344  6947 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:29.682026  6794 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:29.669486  7091 raft_consensus.cc:740] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:29.682818  7106 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:29.594609  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1066) took 1911 ms. Trace:
I20260709 13:03:29.683854  7106 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.684916  1712 rpcz_store.cc:276] 0709 13:03:27.683240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.052768 (+369528us) service_pool.cc:224] Handling call
0709 13:03:29.551996 (+1499228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.558512 (+  6516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.594382 (+ 35870us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54003dc2d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:29.594536 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3447363968}
I20260709 13:03:29.596324  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:29.554935  7071 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.597265  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2521) took 1071 ms. Trace:
I20260709 13:03:29.687079  1561 rpcz_store.cc:276] 0709 13:03:28.525556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.907057 (+381501us) service_pool.cc:224] Handling call
0709 13:03:29.597074 (+690017us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400298bd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:29.597207 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330990336}
I20260709 13:03:29.689478  7071 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:29.678543  6728 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.617241  1563 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:29.692899  6995 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:29.572399  7051 consensus_queue.cc:237] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.636819  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:29.693933  6728 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:29.693993  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 57305.8us
I20260709 13:03:29.694509  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2523) took 1148 ms. Trace:
I20260709 13:03:29.694948  1563 rpcz_store.cc:276] 0709 13:03:28.546290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.012039 (+465749us) service_pool.cc:224] Handling call
0709 13:03:29.617006 (+604967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.692619 (+ 75613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.694271 (+  1652us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b54003e7e50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:29.694441 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1337597312}
I20260709 13:03:29.695835  7083 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:29.696422  7083 raft_consensus.cc:697] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:29.697222  7073 raft_consensus.cc:3055] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:29.697664  7073 raft_consensus.cc:740] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:29.700722  6876 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:29.654235  6632 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.701464  6632 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:29.701563  6935 raft_consensus.cc:3055] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:29.702039  6935 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:29.703003  6935 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.704121  1533 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:29.704348  6935 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:29.705308  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:29.706061  7140 consensus_queue.cc:1059] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:29.627249  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:29.555676  7060 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:29.707113  7140 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:29.707787  7060 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.709118  7113 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:29.709810  7113 raft_consensus.cc:697] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:29.650660  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:29.710134  6637 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.710667  7113 consensus_queue.cc:237] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.614821  7009 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:29.698760  7073 consensus_queue.cc:260] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.620914  7082 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:29.667550  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 136452us
I20260709 13:03:29.669844  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:29.646574  7087 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:29.737644  7087 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.739825  7087 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:29.647060  7054 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:29.740463  7054 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:29.666282  7114 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:29.675244  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:29.675658  7134 log.cc:927] Time spent T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.090s	user 0.000s	sys 0.003s
I20260709 13:03:29.751088  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:29.676980  7141 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.066s	user 0.000s	sys 0.003s
I20260709 13:03:29.691483  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:29.764595  1637 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:29.764629  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:29.646232  1714 rpcz_store.cc:269] 0709 13:03:26.888070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.174468 (+286398us) service_pool.cc:224] Handling call
0709 13:03:29.551874 (+2377406us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54003dc2d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:29.551992 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3117225856}
I20260709 13:03:29.711442  6822 raft_consensus.cc:3055] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:29.646059  7132 log.cc:927] Time spent T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.145s	user 0.003s	sys 0.000s
I20260709 13:03:29.599050  6946 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:29.713279  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:29.662072  1638 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.000s	sys 0.010s
I20260709 13:03:29.730109  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2864) took 1059 ms. Trace:
I20260709 13:03:29.730365  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:29.779671  1638 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.000s	sys 0.010s
I20260709 13:03:29.730671  1642 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:29.781144  1682 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:29.736491  7136 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:03:29.781749  6951 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:29.784591  6952 consensus_meta.cc:287] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.686s	user 0.015s	sys 0.001s
W20260709 13:03:29.737172  7138 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.205s	user 0.004s	sys 0.003s
I20260709 13:03:29.751015  1560 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:29.753125  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:29.790413  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:29.791781  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:29.793327  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:29.755020  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:29.755231  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:29.756317  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:29.795244  1569 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:03:29.653844  6934 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.709113  7060 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:29.769158  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:29.775691  1562 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:29.798053  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:29.798673  1568 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:29.735229  7082 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:29.782634  1563 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:03:29.787853  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:29.802727  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:29.779714  1639 rpcz_store.cc:276] 0709 13:03:28.670793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.722330 (+ 51537us) service_pool.cc:224] Handling call
0709 13:03:29.729897 (+1007567us) spinlock_profiling.cc:243] Waited 7.78 ms on lock 0x7b5400123c54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:29.730053 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17896192}
W20260709 13:03:29.799585  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 258249us
I20260709 13:03:29.768616  7073 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:29.803996  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:29.805189  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:29.808609  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:29.809240  1564 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:29.768219  6822 raft_consensus.cc:740] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:29.810706  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:29.835753  1562 raft_consensus.cc:2376] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12. Re-sending same reply.
I20260709 13:03:29.836364  1681 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:29.818780  1569 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:29.838109  1681 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:29.838559  7137 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:29.838733  7071 consensus_queue.cc:1059] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:29.839139  6728 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:29.832600  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.840399  1716 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:29.781244  7131 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:29.788784  7024 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:29.839061  6864 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:29.806246  1532 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:29.809242  7055 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.698s	user 0.012s	sys 0.000s
I20260709 13:03:29.843724  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:29.843843  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:29.845803  1717 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:03:29.847070  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:29.896401  1563 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 16: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:29.775626  6871 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:03:29.894040  1715 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:03:29.841773  7009 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:03:29.895164  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 51410.2us
I20260709 13:03:29.683935  7091 consensus_queue.cc:260] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.932686  7009 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.933804  7091 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:03:29.933503  6822 consensus_queue.cc:260] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.934388  7146 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:29.834323  6712 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.935524  6712 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.939786  1563 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:03:29.940977  6959 consensus_meta.cc:287] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.000s
I20260709 13:03:29.898593  1567 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
W20260709 13:03:29.841606  7127 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:03:29.778776  6632 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:29.748158  7083 consensus_queue.cc:237] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:29.942308  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1190) took 1050 ms. Trace:
I20260709 13:03:29.971923  1638 rpcz_store.cc:276] 0709 13:03:28.892110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.105509 (+213399us) service_pool.cc:224] Handling call
0709 13:03:29.105851 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:29.106177 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:29.106203 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:29.106621 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 13:03:29.780596 (+673975us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:29.780635 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:29.780710 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:29.942098 (+161388us) raft_consensus.cc:1685] finished
0709 13:03:29.942118 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:29.942248 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1564,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":9776,"spinlock_wait_cycles":13184,"thread_start_us":673627,"threads_started":1,"wal-append.queue_time_us":680687}
W20260709 13:03:29.929384  7032 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.712s	user 0.010s	sys 0.001s
I20260709 13:03:29.963790  1607 leader_election.cc:400] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:29.977744  1607 leader_election.cc:403] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:29.934870  7146 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.845127  6995 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:29.996026  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 150303us
I20260709 13:03:30.000842  1532 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:30.001940  1564 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:03:29.847371  6876 consensus_queue.cc:1059] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.121s
I20260709 13:03:29.897120  7055 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.008260  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:29.845485  7049 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.137s
I20260709 13:03:29.976519  7036 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.975778  7087 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:30.024816  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 190928us
I20260709 13:03:29.996686  7146 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:29.842051  7106 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:29.941455  6959 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:30.031536  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:29.993106  7085 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:30.040251  1717 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:30.042076  1718 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:30.047756  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:30.048626  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 49456.5us
I20260709 13:03:30.055153  1715 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:30.055667  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:30.057996  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:29.993569  7032 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.079681  1714 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 13:03:30.129149  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.146829  2303 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:30.078787  7109 consensus_meta.cc:287] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.918s	user 0.012s	sys 0.003s
I20260709 13:03:30.075125  6637 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:30.014129  6995 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.038676  7036 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.057174  6952 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 13:03:30.144647  7085 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:30.191071  7133 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:30.192221  7133 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.193801  6733 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:30.194343  6733 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.152463  7056 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:29.941936  7095 consensus_queue.cc:1175] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" responder_term: 10 status { last_received { term: 9 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:30.204707  1715 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:30.179317  6934 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:30.184473  7009 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:30.182546  7176 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:30.209722  6975 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.383s	user 0.041s	sys 0.045s
I20260709 13:03:30.197623  7056 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:30.227725  6975 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.421s	user 0.041s	sys 0.045s
I20260709 13:03:30.219482  7106 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:30.248317  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 75655.3us
I20260709 13:03:30.243794  6632 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:30.237565  7158 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
W20260709 13:03:30.237447  7155 log.cc:927] Time spent T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
I20260709 13:03:30.235919  7056 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:30.261086  1565 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
W20260709 13:03:30.251554  7153 log.cc:927] Time spent T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 13:03:30.257745  7164 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:03:30.230949  7139 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.251s
W20260709 13:03:30.268074  7157 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 13:03:30.257289  6801 consensus_meta.cc:287] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.686s	user 0.012s	sys 0.001s
I20260709 13:03:30.277444  1568 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:03:30.249161  7133 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:30.263634  7145 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:30.265970  7055 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:30.307898  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:30.312359  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:30.313237  1639 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:30.287575  7032 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:30.289516  6993 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:30.291683  7075 raft_consensus.cc:3055] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:30.291867  6995 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:30.231463  7176 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:30.258992  7106 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.309492  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:30.330574  7075 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:30.338186  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:30.339622  1712 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:30.340899  1637 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:30.334897  7095 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.352387  7106 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:30.353776  1643 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:03:30.355969  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:30.324728  7125 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.369865  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:30.300199  6801 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:30.353055  6393 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:30.376482  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:30.379922  1712 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:30.327104  6959 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:30.385890  1639 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:30.386650  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:30.391893  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:30.392462  1717 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:30.399994  2300 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:03:30.386354  7131 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.000s
W20260709 13:03:30.395083  1535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.017s	sys 0.071s
I20260709 13:03:30.395936  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:30.408288  1535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.017s	sys 0.071s
W20260709 13:03:30.408660  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.409121  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:30.403836  1709 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:30.392609  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:30.364871  7095 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:30.413779  1644 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:30.412465  2300 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:30.416433  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:30.416714  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:30.417272  1716 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:30.382992  7156 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:30.418339  2300 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:03:30.418422  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:30.419243  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:30.394493  6733 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:30.419320  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:30.423074  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:30.399065  6393 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:30.397838  7036 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:30.415208  7131 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:30.426102  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:30.429391  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:30.380975  7055 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:03:30.440069  2301 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:30.425164  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:30.441488  7182 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.441552  7181 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.441776  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:30.400779  7125 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.481191  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:30.443142  7173 log.cc:927] Time spent T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:03:30.399358  7049 consensus_queue.cc:1059] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:03:30.445379  7075 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.449060  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:30.465392  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:30.473157  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:30.510326  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2464) took 1855 ms. Trace:
I20260709 13:03:30.505909  1683 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:30.510738  1642 rpcz_store.cc:276] 0709 13:03:28.654887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:28.982281 (+327394us) service_pool.cc:224] Handling call
0709 13:03:29.730497 (+748216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:29.780776 (+ 50279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.510132 (+729356us) spinlock_profiling.cc:243] Waited 748 ms on lock 0x7b5400123c50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:30.510265 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1720297856}
I20260709 13:03:30.511348  1639 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:30.512116  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:30.481940  6975 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:30.448669  7060 consensus_meta.cc:287] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.640s	user 0.011s	sys 0.000s
I20260709 13:03:30.483841  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:30.456208  7166 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:30.462141  7140 consensus_meta.cc:287] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.754s	user 0.003s	sys 0.010s
I20260709 13:03:30.490443  1635 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:30.492009  2297 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:03:30.462066  7082 consensus_meta.cc:287] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.628s	user 0.007s	sys 0.004s
I20260709 13:03:30.450973  7176 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:30.505803  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:30.505873  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:30.506613  2268 leader_election.cc:336] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.620s (SENT)
I20260709 13:03:30.460824  7032 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:30.531921  1717 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:30.532110  2304 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:30.532128  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:30.519651  1644 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:30.519759  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:30.504879  6952 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.428050  7156 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.529187  2269 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:30.442699  6617 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:30.516742  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:30.519384  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:30.535851  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.539431  6271 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:30.539769  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:30.540482  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:30.518370  6955 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:30.488866  7182 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.559551  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:30.563716  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:30.526822  7049 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:30.531565  7140 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:30.566779  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:30.570143  1563 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:30.568399  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:30.566057  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:30.568936  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:30.565464  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 115013us
I20260709 13:03:30.490535  7181 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:30.571234  1567 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 11: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:30.573390  1533 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:30.573879  1635 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:30.574621  1638 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:30.576992  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:30.523209  7087 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:03:30.577711  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:03:30.572721  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:30.581905  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:30.579785  1639 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:30.558974  7126 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:30.590359  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:30.525259  6975 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.527840  7166 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.541023  7109 consensus_queue.cc:1059] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:03:30.569710  7060 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:30.566272  7176 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:30.568017  7075 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:30.566569  7049 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:30.570041  6801 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:30.555214  6952 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.555938  7032 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.620720  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:30.592763  7126 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:30.536089  7082 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:30.595992  7177 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.140s	user 0.001s	sys 0.001s
I20260709 13:03:30.580267  7178 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:30.531438  7146 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:30.514986  6393 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.589617  6733 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:30.612829  7185 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:30.623062  7049 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:30.565318  7160 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:03:30.634459  7174 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:30.636821  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:30.596761  6876 consensus_queue.cc:1059] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 13:03:30.644734  7172 log.cc:927] Time spent T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
W20260709 13:03:30.645298  6794 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.963s	user 0.009s	sys 0.005s
I20260709 13:03:30.646023  7182 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:30.646881  7114 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.904s	user 0.011s	sys 0.002s
I20260709 13:03:30.648218  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:30.626956  7171 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:30.664815  7171 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:30.665900  7171 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:30.667106  7192 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:30.668061  7192 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:30.677433  7055 consensus_queue.cc:1059] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:30.661362  7087 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:30.679281  7087 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.679803  7087 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:30.680099  6946 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:30.680570  6946 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:30.681421  6946 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.651715  7174 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:30.686075  7174 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:30.688285  6952 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:30.688896  6952 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:30.689254  6952 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:30.638514  7095 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:30.652849  1713 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:30.631398  7126 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:30.635289  7054 consensus_meta.cc:287] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.894s	user 0.009s	sys 0.003s
I20260709 13:03:30.663717  7194 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:30.663465  7198 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:03:30.663854  7186 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:03:30.664074  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:30.690537  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 72731.7us
I20260709 13:03:30.647818  7125 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:30.707816  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:30.649430  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:30.718499  1639 raft_consensus.cc:2410] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 18 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:03:30.689687  7166 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:30.719442  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:30.715060  1563 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:30.720177  1566 raft_consensus.cc:2368] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
W20260709 13:03:30.721488  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 143830us
I20260709 13:03:30.719789  1532 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:30.724635  1534 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:30.730793  7193 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:03:30.726455  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:30.727433  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:30.735859  1639 raft_consensus.cc:2410] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 13:03:30.728556  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 146796us
W20260709 13:03:30.730873  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 140653us
I20260709 13:03:30.739759  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:30.737519  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:30.669021  7192 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.742480  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:30.743625  1563 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:30.747996  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:30.749876  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:30.755968  7201 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:30.757361  7168 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:30.757823  7168 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.758164  7168 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:30.691409  7114 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.755458  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:30.751569  7193 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:03:30.776474  6530 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:30.783767  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:30.784348  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 941) took 1023 ms. Trace:
I20260709 13:03:30.784006  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:30.715000  7054 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.786759  1563 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:30.719569  7194 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.786651  1569 rpcz_store.cc:276] 0709 13:03:29.760909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.837481 (+ 76572us) service_pool.cc:224] Handling call
0709 13:03:29.837819 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:30.061696 (+223877us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:30.061738 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:30.073654 (+ 11916us) log.cc:844] Serialized 26 byte log entry
0709 13:03:30.381854 (+308200us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:30.381888 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.381957 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:30.784061 (+402104us) raft_consensus.cc:1685] finished
0709 13:03:30.784089 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:30.784270 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122729,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":16044,"thread_start_us":529619,"threads_started":2,"wal-append.queue_time_us":388153}
I20260709 13:03:30.789037  1569 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:03:30.729264  7182 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:03:30.797943  1560 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:30.800349  6975 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:30.801008  6975 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:30.801105  7114 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:30.801779  6975 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:30.729524  6952 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:30.803447  6952 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:30.803789  7032 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:30.752467  6801 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:30.744199  7060 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:30.802299  6975 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.806990  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:30.783294  7197 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:03:30.808377  7191 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:03:30.808634  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:30.835752  7194 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:30.842522  7056 consensus_meta.cc:287] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
I20260709 13:03:30.811095  6860 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.811223  7049 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.783281  7160 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:30.820950  1640 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:30.825834  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:30.804746  7086 consensus_queue.cc:1175] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" responder_term: 16 status { last_received { term: 14 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:30.861907  1718 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:30.786088  7178 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:30.824496  2268 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:30.801023  6827 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.804404  7032 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:30.797891  7125 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:03:30.806633  1643 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:30.832522  6975 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:30.809767  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:30.788530  7095 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:03:30.785943  7166 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:03:30.897432  1639 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:30.906456  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:30.857957  7181 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:30.896649  7037 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:30.897794  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 73392.2us
I20260709 13:03:30.915014  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:30.915777  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:30.925944  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:30.862982  6860 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.940596  7133 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:30.910439  6871 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:30.943452  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 938) took 1183 ms. Trace:
I20260709 13:03:30.943631  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:30.943936  1560 rpcz_store.cc:276] 0709 13:03:29.759934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.793024 (+ 33090us) service_pool.cc:224] Handling call
0709 13:03:30.943226 (+1150202us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54002a9c54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:30.943388 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2308574464}
I20260709 13:03:30.944738  6986 raft_consensus.cc:3055] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:30.945240  6986 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:30.945295  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:30.946640  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:30.951777  7087 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.952455  1560 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:30.896730  7082 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:30.961048  6984 raft_consensus.cc:1068] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Unable to promote non-voter 6208c2fa49f643b5a50ab8f6d292fab2: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
  Committed config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }.
  Pending config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.964186  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1377) took 1429 ms. Trace:
I20260709 13:03:30.966701  1560 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:30.856829  7056 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:30.946298  6986 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:30.983114  7054 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:30.984565  7073 consensus_meta.cc:287] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.153s	user 0.014s	sys 0.000s
I20260709 13:03:30.909039  7146 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:30.908316  6827 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:30.985955  7049 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:30.942792  6860 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:30.989275  2303 rpcz_store.cc:276] 0709 13:03:29.535033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.535152 (+   119us) service_pool.cc:224] Handling call
0709 13:03:29.535494 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.150063 (+614569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:30.963948 (+813885us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b54004998d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:30.964126 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1405492480}
I20260709 13:03:30.990173  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:30.982892  7179 raft_consensus.cc:3055] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:30.933648  6801 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:31.017529  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:30.954864  7087 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:31.020077  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 295531us
I20260709 13:03:30.896286  7178 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:31.022197  1560 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:31.025923  1566 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:30.927181  7200 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 13:03:31.028169  7179 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:31.032181  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:31.033025  1560 raft_consensus.cc:2393] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:31.033147  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:31.036041  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:31.039710  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 311541us
I20260709 13:03:31.051044  1532 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:31.053488  7087 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:31.047618  7024 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:31.054051  7024 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:31.045550  7056 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:31.059499  6827 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:31.044062  7085 raft_consensus.cc:3055] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:31.060143  7085 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:31.060796  7145 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:31.060984  2305 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:31.062073  1561 raft_consensus.cc:2410] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:03:31.063648  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1380) took 1526 ms. Trace:
I20260709 13:03:31.064745  6951 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.064039  2305 rpcz_store.cc:276] 0709 13:03:29.537575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.537728 (+   153us) service_pool.cc:224] Handling call
0709 13:03:31.063481 (+1525753us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b5400583ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:31.063595 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3501629312}
I20260709 13:03:31.070217  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:31.071225  1637 raft_consensus.cc:2410] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:03:31.072188  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:31.073451  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:31.074091  1637 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:31.074302  7160 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:31.038583  7082 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:31.038444  6632 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:31.077050  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:31.061318  7145 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:31.042907  7168 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:31.080668  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:31.081475  6993 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:31.082526  7168 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:31.081594  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.066457  6951 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.076741  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:31.030346  7179 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:31.078567  7156 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:31.093566  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:31.090169  1566 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:31.048794  7126 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:31.105368  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.108129  1534 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:31.083359  7087 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:31.091055  6716 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:03:31.087364  7201 raft_consensus.cc:697] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:31.127446  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1074) took 1330 ms. Trace:
I20260709 13:03:31.127946  1714 rpcz_store.cc:276] 0709 13:03:29.797544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.800115 (+  2571us) service_pool.cc:224] Handling call
0709 13:03:29.800495 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:30.080242 (+279747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.127120 (+1046878us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b540033a4d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:31.127373 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641552896}
I20260709 13:03:31.131668  7178 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:31.061069  7085 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.136373  1641 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:03:31.098028  7179 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:31.126039  1642 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:31.131582  7036 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:31.142881  1639 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:31.148932  1534 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:31.149734  6722 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:31.165197  7126 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:31.168052  7210 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:31.139693  7087 raft_consensus.cc:697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:31.172480  6722 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:31.165283  6986 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:31.146629  7036 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.175595  7208 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:31.177081  7208 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:31.178624  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:31.153518  7024 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.181057  2305 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:31.182093  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:31.182997  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:31.183856  1562 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:31.178161  7036 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:31.190193  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:31.193055  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:31.193429  2303 raft_consensus.cc:2410] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:03:31.150198  7192 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:03:31.191079  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:31.193694  2305 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:31.177953  7208 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:31.196403  6712 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:31.168202  7201 consensus_queue.cc:237] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:31.200949  1533 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:31.203300  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:31.205875  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:31.210884  2300 raft_consensus.cc:2393] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 15: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:31.207621  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:31.208516  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:31.219550  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:31.223987  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:31.223840  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:31.225574  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:31.226461  2303 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:31.226980  2297 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:03:31.227756  2297 pending_rounds.cc:85] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 12
I20260709 13:03:31.228096  2297 pending_rounds.cc:107] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:03:31.228420  2297 raft_consensus.cc:2889] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:31.234853  7181 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:31.235457  7181 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.235844  7181 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:31.187517  7156 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:03:31.208269  7178 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.237421  7160 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:31.245004  7220 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:03:31.180917  6984 consensus_queue.cc:1059] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:03:31.216300  7182 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:03:31.190877  7087 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.247475  7185 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.241494  7056 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:03:31.272751  2305 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:31.258232  7036 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:31.248064  7178 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:31.282014  7218 log.cc:927] Time spent T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:03:31.283882  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 75154.7us
I20260709 13:03:31.284883  1534 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:31.258284  7160 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.300324  7024 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:31.289280  7036 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.322283  7181 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.351045  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:31.328836  7223 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:03:31.332069  7156 raft_consensus.cc:3055] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:31.352304  7156 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:31.352852  7181 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:31.333051  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.272974  7185 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.334594  7178 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:31.335988  7109 raft_consensus.cc:3055] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:03:31.336113  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2529) took 1578 ms (client timeout 1798 ms). Trace:
W20260709 13:03:31.379189  1568 rpcz_store.cc:269] 0709 13:03:29.757523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.808370 (+ 50847us) service_pool.cc:224] Handling call
0709 13:03:31.335916 (+1527546us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b54000f5d54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:31.336053 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1076669824}
I20260709 13:03:31.336997  1563 raft_consensus.cc:1275] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:03:31.337067  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:31.337831  7137 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:03:31.341543  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.381042  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.342218  7149 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:31.343340  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:31.388140  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 44932.6us
I20260709 13:03:31.336622  7145 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.345435  7009 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:31.347146  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:31.348354  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:31.348742  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1196) took 1999 ms. Trace:
I20260709 13:03:31.355721  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:31.363359  7036 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:31.364589  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 2976) took 1615 ms. Trace:
I20260709 13:03:31.367026  7207 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:31.368896  7161 raft_consensus.cc:3055] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:31.369240  7185 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:31.371109  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:31.412495  7145 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:31.412585  1635 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:31.373441  2297 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:31.375223  2304 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:31.344722  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:31.390759  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 53818.2us
I20260709 13:03:31.353228  7156 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.415812  7156 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:31.394057  1640 rpcz_store.cc:276] 0709 13:03:29.349716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.619549 (+269833us) service_pool.cc:224] Handling call
0709 13:03:29.619871 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:30.861115 (+1241244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.348558 (+487443us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54001955d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:31.348694 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2762007424}
I20260709 13:03:31.419513  1716 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:31.392302  6637 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:31.420881  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:31.421098  6637 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.421576  1716 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:31.423470  6637 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:31.408106  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:31.394546  6934 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:31.423586  1715 raft_consensus.cc:1275] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:03:31.405695  1565 rpcz_store.cc:276] 0709 13:03:29.749689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.751171 (+  1482us) service_pool.cc:224] Handling call
0709 13:03:29.751555 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:30.263097 (+511542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:31.364331 (+1101234us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b54002c5954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:31.364528 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1171117824}
I20260709 13:03:31.424219  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:31.425019  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:31.379105  7109 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:03:31.425904  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 81267.2us
I20260709 13:03:31.426406  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:31.428139  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:31.428516  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:31.437489  1718 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:31.394250  6827 consensus_queue.cc:1059] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:03:31.441730  6827 raft_consensus.cc:3055] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:31.442099  6827 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:31.410604  7161 raft_consensus.cc:740] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:31.440179  7109 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.394164  7149 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.437875  1713 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:31.409850  7073 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.447088  1681 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:31.446401  1711 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:31.422667  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:31.392072  7137 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:31.454246  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:31.431825  6955 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:31.405725  7009 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.442934  6827 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.487376  7071 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:31.491647  6827 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:31.492164  7071 raft_consensus.cc:697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:31.493510  7071 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:31.443583  6637 consensus_queue.cc:1059] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:03:31.462229  6934 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:03:31.496628  6393 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:31.443037  7146 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:03:31.500088  7146 raft_consensus.cc:3055] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:31.500522  7146 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:31.501493  7146 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.506166  1636 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:31.427567  7207 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.522531  2269 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:31.542477  1563 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:03:31.457811  6986 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:03:31.542974  1718 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
W20260709 13:03:31.542054  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 109976us
I20260709 13:03:31.473783  7073 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:31.560417  2304 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:31.541913  6955 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:31.551541  6868 raft_consensus.cc:1068] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Unable to promote non-voter 6208c2fa49f643b5a50ab8f6d292fab2: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
  Committed config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }.
  Pending config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.566882  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:31.556182  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 124825us
I20260709 13:03:31.544252  1640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:44380
I20260709 13:03:31.584560  1640 raft_consensus.cc:606] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:31.585041  1640 raft_consensus.cc:614] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:31.569720  7036 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:31.542047  7161 consensus_queue.cc:260] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.585078  7137 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:31.584097  6959 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:31.590054  7036 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:03:31.625524  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 84705.6us
I20260709 13:03:31.613446  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:31.617189  7009 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:31.572817  6955 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:31.641805  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:31.625242  7228 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:03:31.622144  7036 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:31.624879  7161 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:31.620252  6959 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:31.661063  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 138646us
I20260709 13:03:31.663573  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:31.672931  7149 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:31.677959  7160 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:31.683209  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:31.721079  1561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2971) took 1973 ms (client timeout 2255 ms). Trace:
W20260709 13:03:31.721589  1561 rpcz_store.cc:269] 0709 13:03:29.748075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.754979 (+  6904us) service_pool.cc:224] Handling call
0709 13:03:31.720854 (+1965875us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54001798d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:31.721006 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2914539776}
I20260709 13:03:31.683387  6822 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:31.659899  7232 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:31.728427  6951 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:31.729007  7232 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:31.730072  7232 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:31.744128  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:31.766997  1561 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:31.744942  1567 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:31.753482  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:31.768736  1718 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:31.753690  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:31.754029  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:31.770530  1714 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:31.771096  1711 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:31.695142  6955 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:31.696080  1565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6711207c14044518a57e7c8914ac0091"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:03:31.701632  2269 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:31.703830  7207 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:31.709245  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:31.718787  7229 log.cc:927] Time spent T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 13:03:31.683490  7227 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
W20260709 13:03:31.757694  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.760000  1568 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:03:31.761936  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:31.762317  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:31.755364  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:31.762720  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:31.777834  1607 leader_election.cc:336] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.798s (SENT)
I20260709 13:03:31.832477  1560 raft_consensus.cc:2393] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:31.833031  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 55268us
W20260709 13:03:31.834127  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.784588  1565 raft_consensus.cc:606] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:03:31.787030  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 85484.6us
W20260709 13:03:31.790387  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.835146  1565 raft_consensus.cc:614] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:31.823586  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:31.828750  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:31.833436  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:31.832770  1562 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:31.844861  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:31.734858  7208 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:31.848069  1569 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 11: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:31.847911  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:31.738330  7109 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:31.847548  6733 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.876080  6733 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.878297  6733 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:31.825060  6946 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:31.888123  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 52693.6us
I20260709 13:03:31.735538  7085 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:31.889397  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 65948.4us
I20260709 13:03:31.892026  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:31.828022  7160 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:31.888965  2269 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:31.898732  2269 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:31.899967  7146 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:31.900815  7240 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:31.901342  7240 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.901729  7240 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:31.903060  7207 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:31.903565  7207 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.903667  7037 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:31.904592  7139 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:31.905265  6827 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:31.905783  6827 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:31.905328  7139 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.906551  6827 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.872049  7069 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:31.907860  7069 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:31.909777  7069 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:31.904006  7207 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:31.904814  7037 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.897428  7106 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:03:31.912314  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 627528us
I20260709 13:03:31.923465  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:31.841699  6876 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:31.911111  6984 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:31.920069  1635 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:31.913602  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:31.848449  7200 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:31.890563  1561 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:31.904668  7126 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:31.874804  7178 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:31.931466  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 600670us
I20260709 13:03:31.746364  7036 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:31.924523  7235 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:03:31.943859  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 303176us
I20260709 13:03:31.940580  1640 raft_consensus.cc:2393] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:31.948767  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 601682us
W20260709 13:03:31.950646  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:31.947747  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:31.952637  1561 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:31.922482  7106 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:31.953991  1565 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:31.958297  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 381330us
I20260709 13:03:31.959007  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:31.935225  6745 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:31.919699  7240 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:31.950693  7156 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.534s	user 0.005s	sys 0.007s
I20260709 13:03:31.964128  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:31.972596  1635 raft_consensus.cc:2376] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18. Re-sending same reply.
I20260709 13:03:31.938539  7126 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.959499  1681 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:31.970629  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:31.974675  1641 raft_consensus.cc:2410] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:03:31.952193  7193 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:31.928399  7037 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.959959  7106 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:31.980058  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:31.934528  6876 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:31.986167  7139 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:31.977492  7071 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:31.995127  1564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:03:31.995958  1564 raft_consensus.cc:606] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:31.996606  1564 raft_consensus.cc:614] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:31.999737  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 76363.7us
I20260709 13:03:32.000712  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:32.002298  1534 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:32.003542  1639 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:32.007206  1681 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:32.008597  1681 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:32.009893  6876 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:32.010362  1681 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:32.010581  7009 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:32.011459  7009 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:31.976833  7212 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.015501  7212 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:32.016638  1563 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:32.019415  1639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:44380
I20260709 13:03:32.020785  1639 raft_consensus.cc:606] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:32.024372  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:31.984722  6951 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:03:31.975586  7126 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:31.950335  7178 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:32.032178  1643 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:32.032775  7178 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:32.033474  7113 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:32.034219  7113 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:32.035207  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:32.040282  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:32.035646  6984 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:32.042531  1635 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 15: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:32.038254  7210 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.050158  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:32.014012  7073 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:31.973973  6745 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.057976  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:32.039489  6959 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:32.034498  7240 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:32.026788  7036 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:32.045615  7037 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:32.066114  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:32.076428  1565 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
W20260709 13:03:32.071115  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 46835.9us
W20260709 13:03:32.078229  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 67954.3us
I20260709 13:03:32.072422  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:32.069541  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:32.081123  1635 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:32.070528  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:32.054447  7082 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.061403  1643 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 13: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:32.083288  2296 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:32.079859  1681 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:32.088603  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:32.089175  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.089524  1639 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:32.091364  1711 raft_consensus.cc:1275] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:03:32.093073  1711 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:32.093195  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:32.096302  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:32.094722  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:32.102085  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:32.103027  1714 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:32.113286  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:32.113962  1711 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:32.102666  1609 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:32.062301  7137 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.112299   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:03:32.062940  7168 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:32.123350  1709 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:32.126127  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:32.134532  2298 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:32.131502  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:32.132372  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.073709  6617 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:32.138679  1712 raft_consensus.cc:2410] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:03:32.141547  1714 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:32.143213  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:32.058874  1710 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.004s	sys 0.020s
I20260709 13:03:32.144107  1718 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
W20260709 13:03:32.145054  1710 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.005s	sys 0.021s
I20260709 13:03:32.110179  6712 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:32.169476  6712 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:32.170552  6712 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:32.171643  1642 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:03:32.173581  7145 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:32.175040  7207 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.097668  6876 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:32.152652  1532 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:32.218524  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 65969.6us
I20260709 13:03:32.219892  1532 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:32.154189  7246 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:32.154508  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.228790  1715 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:32.155938  7087 consensus_queue.cc:1048] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:32.149111  7137 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:32.110829  7082 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:32.163478  6617 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:32.125715  6733 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:32.061295  7210 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.152868  7168 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.180073  6745 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:32.181272  1639 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:32.245586  1682 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:32.185083  7106 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:32.236258  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 284939us
I20260709 13:03:32.186610  6827 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:32.230099  6632 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:32.153074  6393 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:32.189736  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.189543  1711 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
W20260709 13:03:32.151628  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 71852.6us
I20260709 13:03:32.193832  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:32.196266  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.197376  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.248741  1533 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:32.260288  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:32.278978  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.274300  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:32.279896  1640 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:32.260849  1638 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:32.285794  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:32.212481  1713 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
W20260709 13:03:32.214210  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 284000us
I20260709 13:03:32.230701  1532 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:32.288234  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:32.185127  6530 raft_consensus.cc:3055] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:32.236157  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.249154  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:32.306052  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 56978.7us
I20260709 13:03:32.308624  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:32.233388  7140 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:32.232483  6876 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:32.290035  7246 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:32.291440  7126 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.292487  7168 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:32.292841  6617 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.293937  7178 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.294489  7106 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:32.277067  7252 raft_consensus.cc:993] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: : Instructing follower 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
W20260709 13:03:32.314193  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 83603us
I20260709 13:03:32.313882  7073 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:32.315104  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 48474.5us
I20260709 13:03:32.339241  1681 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:32.254590  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:32.331607  6271 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:32.317584  1567 raft_consensus.cc:2410] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:03:32.322258  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:32.275887  7253 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:32.325511  7146 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:32.268392  6868 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:32.320164  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:32.266626  7137 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:32.269341  7207 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:32.265275  7082 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:32.375612  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 53473.8us
I20260709 13:03:32.259886  6733 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.314499  1712 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 13:03:32.379436  1712 pending_rounds.cc:85] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 6
I20260709 13:03:32.379777  1712 pending_rounds.cc:107] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 13:03:32.380066  1712 raft_consensus.cc:2889] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:32.265096  6393 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.301685  6530 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:32.313393  6745 raft_consensus.cc:3055] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:32.323300  7087 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:32.318836  7109 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:32.309653   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:32.335876  1532 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:32.393227  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 57480us
I20260709 13:03:32.270162  6632 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:32.302011  6959 consensus_queue.cc:1059] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:03:32.358243  1639 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:32.397042  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:32.397809  1639 raft_consensus.cc:2368] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
I20260709 13:03:32.371155  6946 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:32.335011  7246 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:32.325922  7071 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:03:32.337545  7178 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:32.383558  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 63521.5us
I20260709 13:03:32.343561  7085 raft_consensus.cc:1081] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Signalling peer 6208c2fa49f643b5a50ab8f6d292fab2 to start an election
I20260709 13:03:32.383572  1640 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:32.374722  7126 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:32.398727  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:32.385097  7106 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:32.345471  7073 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:32.388121  6745 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:32.397449  2269 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:32.382784  6868 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:32.399897  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 330795us
I20260709 13:03:32.449559  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1403) took 1233 ms. Trace:
I20260709 13:03:32.457700  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.384819  6733 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:32.440793  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.459519  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:32.371470  7253 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.444716  1709 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:03:32.461633  1607 leader_election.cc:400] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:32.460233  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:32.431008  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 52171.4us
I20260709 13:03:32.462190  1713 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
W20260709 13:03:32.462893  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 65534.3us
I20260709 13:03:32.463748  2268 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:32.464380  2269 leader_election.cc:400] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
I20260709 13:03:32.464845  2269 leader_election.cc:403] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:32.458782  2299 rpcz_store.cc:276] 0709 13:03:31.215339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.231252 (+ 15913us) service_pool.cc:224] Handling call
0709 13:03:31.236907 (+  5655us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.240260 (+  3353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.240294 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:31.240782 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 13:03:31.981111 (+740329us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b180022b2e0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:03:32.393432 (+412321us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:32.402147 (+  8715us) log.cc:844] Serialized 14 byte log entry
0709 13:03:32.430997 (+ 28850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.431085 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:32.431139 (+    54us) raft_consensus.cc:1685] finished
0709 13:03:32.448181 (+ 17042us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.448440 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36838,"prepare.queue_time_us":8482,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":32867,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":20285,"spinlock_wait_cycles":7220096,"thread_start_us":412128,"threads_started":1,"wal-append.queue_time_us":1161496}
I20260709 13:03:32.458288  1639 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:32.392056  7087 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.397928  7232 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:32.398365  7082 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:32.501060  7082 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:32.503080  7073 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:32.404290  6876 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:32.408159  6946 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.405318  7246 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:32.481657  7192 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:03:32.401533  6632 raft_consensus.cc:697] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:32.475657  7252 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:32.496601  7087 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:32.494590  7253 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:32.468063  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:32.524446  7200 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:32.525043  7200 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:32.499091  7139 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:32.450004  7182 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:03:32.508924  7232 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.458689  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:32.506609  6946 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:32.460179  1712 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:32.510210  1714 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:32.401141  7009 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:32.507601  6632 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:32.533684  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 65724.5us
I20260709 13:03:32.507710  6745 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:32.535284  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 196186us
I20260709 13:03:32.508577  7254 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:32.517663  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 1220) took 1330 ms. Trace:
I20260709 13:03:32.540187  1638 rpcz_store.cc:276] 0709 13:03:31.187580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.189357 (+  1777us) service_pool.cc:224] Handling call
0709 13:03:32.517392 (+1328035us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002d8a54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:32.517544 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2377635584}
I20260709 13:03:32.519562  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:32.541940  1641 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:32.495184  7210 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:32.482059  2269 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:32.529899  1640 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:32.530274  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:32.529053  7232 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:32.532317  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:32.511399  7036 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:32.552588  1713 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:32.529476  7161 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:32.536543  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:32.539583  1565 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:32.540458  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.519855  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:32.544636  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.536329  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:32.550268  1639 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
W20260709 13:03:32.551154  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 69185.1us
I20260709 13:03:32.554396  1564 raft_consensus.cc:2393] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:32.406347  6393 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:32.570266  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:32.576787  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:32.577970  1565 raft_consensus.cc:2410] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:03:32.557176  6745 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:32.557350  7106 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:32.567510  1712 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:32.582219  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:32.538185  7192 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.545419  7095 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:32.586767  6868 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:32.578212  2269 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:32.586311  1569 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:32.586558  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:32.587155  1565 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 11: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:32.584399  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:32.598385  1564 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 15: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:32.503859  7082 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.581146  7210 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:32.611588  1713 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:32.546291  7146 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:03:32.574934  7254 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:32.614079  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:32.585733  7106 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:32.615509  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:32.616990  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:32.592059  6712 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.618333  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:32.622490  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:32.614810  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:32.591446  7095 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.587462  7253 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.646937  1713 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:32.650568  1569 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:32.665292  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:32.620144  7185 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:32.622020  7210 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.666504  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:32.655833  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 178231us
I20260709 13:03:32.696357  1716 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:32.697619  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:32.667773  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 69015.7us
I20260709 13:03:32.593120  7192 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:32.604122  1681 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:32.655539  7095 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:32.704005  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 100011us
I20260709 13:03:32.656802  6271 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:32.681762  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.702844  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:32.653429  7069 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:32.654295  7085 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.707667  1568 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:32.707633  1681 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:32.724403  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:32.725044  1565 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:32.725852  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:32.726594  1565 raft_consensus.cc:2410] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
W20260709 13:03:32.727478  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.670957  7182 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.719303  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.720026  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.727928  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:32.738387  1561 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:32.738816  1713 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:32.655972  6712 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.695870  7185 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:32.733310  1560 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:32.667187  7253 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:32.699396  7210 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:32.729223  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:32.747654  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:32.700937  7256 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:32.733989  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:32.766291  1566 raft_consensus.cc:2376] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15. Re-sending same reply.
I20260709 13:03:32.769234  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:32.722471  6946 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:32.771498  1716 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:32.738875  7252 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:32.752564  7185 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:32.790854  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.732658  6271 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.732867  7069 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.776391  6934 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:32.772840  6876 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:32.732316  7085 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.784732  7269 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:32.746544  7182 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:32.781829  7252 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:32.824003  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 151159us
I20260709 13:03:32.812798  7166 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:32.789734  7273 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:32.752871  7106 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.748534  7254 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.789527  7246 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:03:32.847980  7252 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:32.863684  7212 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:32.864528  7069 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:32.865314  7069 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:32.843981  6876 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.871641  6393 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:32.872246  6393 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.872651  6868 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:32.874055  6876 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:32.873970  6868 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.875095  7208 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.879384  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1077) took 3054 ms. Trace:
I20260709 13:03:32.891289  1710 rpcz_store.cc:276] 0709 13:03:29.798390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:29.806218 (+  7828us) service_pool.cc:224] Handling call
0709 13:03:29.898836 (+ 92618us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:31.338599 (+1439763us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.338649 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:31.387772 (+ 49123us) log.cc:844] Serialized 52 byte log entry
0709 13:03:31.526256 (+138484us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:32.147280 (+621024us) log.cc:844] Serialized 14 byte log entry
0709 13:03:32.162628 (+ 15348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.848541 (+685913us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b54003dc7d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:32.848613 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:32.848679 (+    66us) raft_consensus.cc:1685] finished
0709 13:03:32.852125 (+  3446us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:32.852354 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":659314,"prepare.queue_time_us":193942,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":659963,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":234,"spinlock_wait_cycles":1822316800,"thread_start_us":686626,"threads_started":2,"wal-append.queue_time_us":193147}
I20260709 13:03:32.880288  1712 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:32.885558  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.895617  1713 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:32.887341  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:32.897421  1569 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
W20260709 13:03:32.899013  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 189445us
I20260709 13:03:32.826516  7087 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.882653  7009 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:32.888365  6959 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:32.852337  7270 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:03:32.904403  6393 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:32.919337  6990 raft_consensus.cc:3055] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:32.920401  6946 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:32.901786  7208 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:32.936013  7274 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:03:32.936105  7272 log.cc:927] Time spent T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:03:32.937048  7267 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:03:32.940757  7166 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:32.944130  7273 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:32.956866  7036 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:32.957511  7266 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:03:32.934690  7253 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:32.977283  7182 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:32.978114  7256 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:32.961315  7166 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:32.984897  7200 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:32.985880  7252 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:32.987224  7200 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:32.988269  7208 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:32.951680  7232 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:32.990569  7192 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:32.994246  7232 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:33.030300  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 128185us
I20260709 13:03:33.032459  2268 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:33.033248  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 255759us
I20260709 13:03:33.036254  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:33.039284  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:33.038520  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:33.041136  2299 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:33.041486  1710 raft_consensus.cc:2393] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:33.044577  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 1277) took 1694 ms. Trace:
I20260709 13:03:33.045145  2303 rpcz_store.cc:276] 0709 13:03:31.350064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.352146 (+  2082us) service_pool.cc:224] Handling call
0709 13:03:31.352483 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:31.356534 (+  4051us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:31.356580 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:31.357168 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 13:03:32.619281 (+1262113us) spinlock_profiling.cc:243] Waited 5.75 ms on lock 0x7b18003d07e0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:03:32.788440 (+169159us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:32.788492 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:32.789353 (+   861us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:33.043727 (+254374us) raft_consensus.cc:1685] finished
0709 13:03:33.043774 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.044255 (+   481us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3554,"raft.queue_time_us":3719,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":24965,"spinlock_wait_cycles":13223168,"thread_start_us":168965,"threads_started":1,"wal-append.queue_time_us":1550988}
I20260709 13:03:33.048359  1711 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:33.051491  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:33.052325  1711 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:32.944151  6868 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.054011  6617 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:33.054240  6868 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.054527  6271 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:33.055595  6271 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:33.055233  6868 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:33.056711  6822 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:33.056938  6712 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.063167  6975 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.074007  7168 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:33.075807  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:33.076963  1718 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:33.076982  7168 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:33.082728  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:33.057696  6822 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.084298  7210 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:33.059158  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.085090  7178 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.066227  7085 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.063141  6876 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.055181  6617 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.082944  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:33.062124  6868 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.056061  6271 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.069487  7087 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.086458  6822 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:32.935352  6990 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:03:33.096206  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 58021.9us
I20260709 13:03:33.100297  6617 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.100838  7241 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:33.101541  7241 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:33.092834  6271 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:33.074512  6975 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.107059  7232 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.094834  6876 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.108038  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:33.107932  7232 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.110371  6876 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.104279  6990 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.112459  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:33.113286  1710 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:33.113900  6990 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:33.113718  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.116204  2302 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:33.117206  7232 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.120297  6975 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.107434  7210 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.121860  1718 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:33.122444  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:33.124990  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:33.153507  2302 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEARNER]: Advancing to term 18
I20260709 13:03:33.102892  7241 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.130983  1710 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:33.131294  1711 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:33.131677  6946 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:33.136037  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2548) took 1350 ms. Trace:
I20260709 13:03:33.132887  7095 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.146673  7252 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.162837  1564 raft_consensus.cc:2368] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12.
I20260709 13:03:33.157516  1567 rpcz_store.cc:276] 0709 13:03:31.785807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.786098 (+   291us) service_pool.cc:224] Handling call
0709 13:03:33.135398 (+1349300us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b54002981d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:33.135722 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1019134720}
I20260709 13:03:33.157691  7232 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:03:33.172092  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 89759.8us
I20260709 13:03:33.161311  7287 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:33.177780  2268 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:33.174053  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 189561us
I20260709 13:03:33.178550  1682 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:33.181877  1567 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:03:33.185349  7095 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.185849  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:33.187325  6876 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:33.187856  6876 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:33.188504  6876 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:33.172333  7232 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.193452  7286 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.200055  7200 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:33.200237  7286 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:33.201097  7200 raft_consensus.cc:697] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:33.194324  7270 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:33.205704  7270 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:33.206372  7270 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:33.209681  7232 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:33.196954  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:33.211546  1642 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:33.198004  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:33.199208  7252 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:33.213974  7252 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.198961  1532 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:33.215638  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:33.216634  1639 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:33.217391  6975 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:33.218004  6975 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.218498  6975 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:33.209194  7200 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.205106  7286 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.224483  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:33.226099  1643 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:33.239161  7254 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:33.239414  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 53833.9us
I20260709 13:03:33.239970  7254 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.240365  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:33.226668  7252 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.241819  7252 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:33.246094  7254 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.249042  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.250907  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.252090  2298 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:33.253293  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.256276  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2501) took 1042 ms. Trace:
I20260709 13:03:33.256783  1714 rpcz_store.cc:276] 0709 13:03:32.214055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.214623 (+   568us) service_pool.cc:224] Handling call
0709 13:03:32.215610 (+   987us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.531542 (+315932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.255645 (+724103us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54003381d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:33.256023 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":677048064}
W20260709 13:03:33.260831  6733 consensus_meta.cc:287] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.782s	user 0.004s	sys 0.017s
I20260709 13:03:33.261492  6733 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.263147  1637 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 13: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:33.264578  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 1231) took 1013 ms. Trace:
I20260709 13:03:33.265158  2269 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:33.265080  1637 rpcz_store.cc:276] 0709 13:03:32.251196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.519545 (+268349us) service_pool.cc:224] Handling call
0709 13:03:33.264300 (+744755us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b54001e8a54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:33.264469 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1619669888}
W20260709 13:03:33.266990  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 44417.6us
I20260709 13:03:33.267419  7256 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:33.267974  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:33.267827  6530 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.269009  6868 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:33.264416  7087 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:03:33.271101  2304 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:33.274736  7145 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:33.275717  7145 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.267885  7256 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.276695  7256 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:33.254417  2301 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:53446
I20260709 13:03:33.278482  2301 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:33.278901  2301 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:33.279894  6637 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:33.280755  6637 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.283900  6637 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.269488  6868 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.284922  6868 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:33.285898  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2507) took 1197 ms. Trace:
I20260709 13:03:33.264847  1643 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:33.269412  6530 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:33.308952  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.317960  1532 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:33.318954  6733 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.319419  7087 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:33.320158  7087 raft_consensus.cc:697] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:33.321412  7087 consensus_queue.cc:237] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.325862  7232 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:33.327386  1563 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 2992) took 1776 ms (client timeout 2009 ms). Trace:
W20260709 13:03:33.328003  1563 rpcz_store.cc:269] 0709 13:03:31.551136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:31.552132 (+   996us) service_pool.cc:224] Handling call
0709 13:03:33.327002 (+1774870us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b5400200654. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:33.327187 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1022716288}
I20260709 13:03:33.327291  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:33.329510  1714 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:33.331218  1682 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:33.331641  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:33.332787  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:33.334132  7095 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:33.334596  7095 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.335091  7095 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:33.335803  1638 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:03:33.336925  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.337091  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:33.338133  6984 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:33.338461  7109 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:33.339001  7109 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:33.339483  7109 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:33.334611  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:33.345299  1636 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:03:33.307200  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 79222.6us
I20260709 13:03:33.345752  1636 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:33.337962  1638 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:33.278600  7270 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.358011  1643 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:03:33.353278  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:33.361066  2302 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:33.307005  7254 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:33.362187  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:33.338838  6984 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:33.291725  7145 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:33.375475  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 84161.3us
I20260709 13:03:33.286403  1644 rpcz_store.cc:276] 0709 13:03:32.088797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:32.103916 (+ 15119us) service_pool.cc:224] Handling call
0709 13:03:32.104221 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:32.338568 (+234347us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:32.338832 (+   264us) log.cc:844] Serialized 14 byte log entry
0709 13:03:32.592188 (+253356us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:32.592392 (+   204us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:32.592456 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:33.284569 (+692113us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b1800336280. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb822a6 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419
0709 13:03:33.285435 (+   866us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54002d8054. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:33.285489 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:33.285692 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1856,"prepare.queue_time_us":2022,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":11908,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":7020,"spinlock_wait_cycles":564680960,"thread_start_us":241111,"threads_started":1,"wal-append.queue_time_us":412572}
I20260709 13:03:33.380198  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:33.382371  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:33.383031  7137 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.019s	sys 0.004s
W20260709 13:03:33.383440  7137 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.019s	sys 0.004s
W20260709 13:03:33.391126  7161 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.810s	user 0.008s	sys 0.014s
I20260709 13:03:33.391520  1638 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:33.391671  7161 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
I20260709 13:03:33.392750  6733 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:33.393404  6733 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:33.396552  6951 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:33.397430  6951 raft_consensus.cc:697] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:33.403909  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:33.404873  1635 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:33.411398  2301 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:33.415342  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 57885us
W20260709 13:03:33.422904  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 48817.6us
I20260709 13:03:33.390446  7166 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.427922  1710 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:33.432577  7232 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.434165  7255 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:33.418322  7254 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.438308  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:33.394490  6733 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:33.461788  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:33.519758  2302 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:33.463167  1681 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:33.398556  6951 consensus_queue.cc:237] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:33.524026  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 61117.9us
W20260709 13:03:33.464694  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 918937us
I20260709 13:03:33.471086  6984 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:33.475989  6271 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:33.479000  7185 consensus_meta.cc:287] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.666s	user 0.015s	sys 0.003s
I20260709 13:03:33.480334  7270 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.459824  7255 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:33.482734  6975 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:33.486003  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:33.487025  2305 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:03:33.489714  1641 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:33.492780  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:33.493687  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.493904  7071 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:03:33.494712  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 73008.9us
I20260709 13:03:33.452417  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:33.440235  2297 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:33.462129  6955 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:33.504205  1639 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:33.505100  6617 consensus_queue.cc:1059] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:33.548532  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:33.438892  7254 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.505368  7256 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.421080  1636 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:33.509097  7137 consensus_queue.cc:1059] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:33.512019  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.531565  1714 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:33.395723   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:33.536253  2301 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:33.537309  2303 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 17: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:33.538550  2302 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:33.557512  6530 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:03:33.598047  1636 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:33.603258  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:33.603546  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:33.604167  1644 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:33.604302  1567 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:33.604804  7095 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:33.608307  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 186531us
I20260709 13:03:33.609390  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:33.611195  7178 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:33.612175  7265 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:33.614259  1561 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:33.616809  6876 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:33.598692  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.630214  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.630954  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:33.632133  1569 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 16: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:33.632377  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:33.632720  7109 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:33.616995  7255 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.640616  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.642138  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.643316  7095 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.643842  6876 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.644309  6975 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.644389  7254 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:33.645363  7254 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.647331  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.649101  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.650264  7146 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:33.650839  7146 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:33.651743  7146 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:33.652966  6868 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:33.655434  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.655939  1637 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:33.656929  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.657207  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:33.658263  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.659288  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:33.659988  1641 raft_consensus.cc:3055] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:33.659595  6712 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:33.660328  1641 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:33.661120  6712 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:33.661625  6712 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.662182  6712 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:33.662961  1644 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:33.663270  7166 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.663908  6990 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:33.664461  6990 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:33.665596  6990 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:33.667757  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.669548  6271 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.544507  7185 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:33.671980  1534 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.591s (SENT) [suppressed 14 similar messages]
I20260709 13:03:33.672276  7185 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:33.673875  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.675242  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.675897  2301 raft_consensus.cc:2368] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15.
I20260709 13:03:33.676743  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:33.677364  2301 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:33.650547  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:33.661293  1641 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.684119  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.635417  2298 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:33.638842  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:33.638842  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:33.688383  2304 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:03:33.688452  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 49676.4us
I20260709 13:03:33.685796  2300 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
W20260709 13:03:33.689488  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.686707  2305 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
W20260709 13:03:33.689926  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:33.691056  1569 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:03:33.691249  1609 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:33.691782  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 53054.5us
W20260709 13:03:33.691967  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 52172.4us
I20260709 13:03:33.694037  6955 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:33.694665  6617 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:33.691454  1569 pending_rounds.cc:85] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 12
I20260709 13:03:33.644671  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:33.701066  1567 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:33.702072  2302 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:33.702258  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:33.703714  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.703274  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:33.702574  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:33.707492  1681 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:33.709029  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:33.710538  1635 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 13: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:33.711030  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:33.712182  1644 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:03:33.711404  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.710119  1636 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:33.702515  6617 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:33.716562  2302 raft_consensus.cc:2410] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:03:33.716199  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:33.703230  7168 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:33.716634  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:33.695427  1569 pending_rounds.cc:107] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 17.13
I20260709 13:03:33.717331  7185 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.736119  2296 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:33.740609  1609 leader_election.cc:400] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15.
I20260709 13:03:33.741074  1609 leader_election.cc:403] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:33.740883  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:33.723096  7137 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:33.720966  6868 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.724426  6876 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:33.739974  1569 raft_consensus.cc:2917] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Aborting config change with OpId 17.13: Aborted: Op aborted by new leader
I20260709 13:03:33.743867  1681 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:33.725984  7168 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:33.719087  7166 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:03:33.763353  2302 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:33.739609  6990 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:33.779923  2296 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:33.718936  6271 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
W20260709 13:03:33.782102  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 76341.5us
I20260709 13:03:33.785503  1569 raft_consensus.cc:2889] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:33.776506  7168 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:33.794747  2296 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:33.795399  2298 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:03:33.799655  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 162941us
I20260709 13:03:33.802181  7230 consensus_queue.cc:1059] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 13:03:33.803601  2297 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:33.805426  1641 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:33.806947  7109 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:33.808784  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:33.809478  1636 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:33.809732  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:33.810369  1637 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:33.762780  6617 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.817657  1642 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:33.810402  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:33.821292  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 77563us
I20260709 13:03:33.829689  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:33.835386  6975 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.836069  6975 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:33.784789  7185 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.841503  7146 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:33.828390  6959 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:33.842891  7185 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.857082  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:33.860392  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:33.868203  6271 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:33.869158  6271 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.870518  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:33.871490  1637 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:33.873281  6271 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.838717  6975 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:33.862569  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:33.844313  1717 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:33.874596  1642 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:33.844480  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:33.875591  1712 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:33.877120  1641 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:33.886153  7073 raft_consensus.cc:3055] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:33.848999  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:33.887780  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:33.880178  1638 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:33.903954  1715 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:33.904415  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:33.845229  6959 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:33.903374  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:33.917299  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:33.797935  7270 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:33.918488  1717 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:33.888103  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:33.923815  1642 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:33.907972  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:33.927366  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:33.906976  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:33.928072  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:33.929212  1636 raft_consensus.cc:2368] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14.
I20260709 13:03:33.844195  7146 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:33.886497  6733 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:33.886687  7073 raft_consensus.cc:740] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:33.905856  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:33.962368  1716 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
W20260709 13:03:33.962498  7296 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.421s	user 0.003s	sys 0.003s
I20260709 13:03:33.980777  7146 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.981421  7146 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.983487  6959 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.986372  1569 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:33.990072  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 102155us
I20260709 13:03:33.905138  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:33.990280  7146 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:33.991353  1565 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:33.992050  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:33.992909  7256 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:33.994939  7069 raft_consensus.cc:2764] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:03:33.995687  7069 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:33.997192  6733 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:33.960498  2269 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:33.965742  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:34.001348  1644 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:33.961551  2301 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:34.003935  1638 raft_consensus.cc:2410] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:03:33.961948  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:34.009652  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 337871us
I20260709 13:03:34.058637  1712 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:34.061616  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:34.080673  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 128039us
I20260709 13:03:34.084055  1568 raft_consensus.cc:2368] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14.
I20260709 13:03:34.081660  2269 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:34.085579  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:33.959287  7071 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:34.086750  2300 raft_consensus.cc:2410] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:03:34.090004  1636 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
W20260709 13:03:34.095921  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.090881  1641 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:33.965335  6637 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.125s
I20260709 13:03:34.097738  1560 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:34.106060  1636 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:34.088480  6733 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:34.111346  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 346380us
I20260709 13:03:34.117476  7139 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15.
I20260709 13:03:34.098723  7069 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:34.112537  7255 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.143435  7069 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.143898  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:34.143599  7073 consensus_queue.cc:260] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.144686  7073 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:34.102069  7071 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.144775  7168 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.145975  7069 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:34.146960  7071 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:34.143527  7255 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.151687  7255 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:34.144554  1712 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:34.152976  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:34.155186  7106 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.155433  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:34.156034  7106 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.159699  7207 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:34.160529  7207 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:34.161778  7207 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.163203  7292 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:34.163359  7145 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:34.164193  7292 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.165571  7145 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:34.167095  7246 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:34.168007  7246 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.174989  7168 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:34.176551  1637 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:34.177628  1639 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:34.183439  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 60719.1us
I20260709 13:03:34.186084  1636 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:34.189925  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:34.195611  1569 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:34.118656  7126 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:34.119151  7254 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:34.204715  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 396618us
I20260709 13:03:34.193195  7106 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:34.214126  2296 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:34.215318  1712 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:34.205469  6712 consensus_meta.cc:287] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.540s	user 0.005s	sys 0.006s
I20260709 13:03:34.219280  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:34.203771  1641 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
W20260709 13:03:34.240693  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 86082.7us
I20260709 13:03:34.201278  7256 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:03:34.212452  7126 raft_consensus.cc:697] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:34.210470  7292 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:34.258934  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:34.267210  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.250185  7137 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:34.272101  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:34.275471  2300 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:34.239472  6712 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:34.293422  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 189563us
I20260709 13:03:34.296464  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:34.254204  7246 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:34.300364  7270 raft_consensus.cc:697] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:34.323998  7270 consensus_queue.cc:237] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:34.300689  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:34.270612  6990 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.327445  6990 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.328526  6712 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:34.329718  6990 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:34.331228  1718 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:34.276868  7292 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:34.332484  1683 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:34.332767  1710 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:34.270165  7106 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:34.305238  6822 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:34.333133  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:34.339352  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:34.341406  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:34.344055  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:34.344861  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 139074us
I20260709 13:03:34.350126  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1567) took 1292 ms. Trace:
I20260709 13:03:34.350590  2303 rpcz_store.cc:276] 0709 13:03:33.058326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.058572 (+   246us) service_pool.cc:224] Handling call
0709 13:03:34.349841 (+1291269us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b540059b5d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:34.350016 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1033902464}
I20260709 13:03:34.353322  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:34.354132  1643 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:34.334338  6822 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:34.360054  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:34.334348  7106 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:34.363011  2300 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:34.343400  7246 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:34.368407  1638 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 16: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:34.371441  1641 raft_consensus.cc:2410] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:03:34.375211  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:34.372823  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:34.332170  7292 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.370111  7246 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:34.395166  1712 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:34.397157  7106 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.400365  7106 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:34.407030  1609 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:34.407287  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:34.408982  1609 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:34.409337  2298 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:34.409865  6946 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:34.410290  6946 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.410486  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:34.411163  2300 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:03:34.410658  6946 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:34.411748  7168 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:34.412189  7168 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.412547  7168 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:34.420651  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:34.421367  7069 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:34.421983  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:34.422767  1609 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:34.423367  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 51469.8us
I20260709 13:03:34.423939  7069 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:34.424391  7069 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:34.424889  7069 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:34.424548  6946 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.425590  2268 leader_election.cc:400] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12.
I20260709 13:03:34.426976  6946 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:34.427713  6946 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.428366  6946 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.429555  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:34.430280  1638 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:34.430862  6946 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:34.433122  1642 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:03:34.429579  7305 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.229s	user 0.001s	sys 0.001s
I20260709 13:03:34.438318  6822 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.439361  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:34.439579  6822 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:34.439971  2299 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:34.438560  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:34.442674  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:34.443120  2298 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:34.444139  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:34.444842  7139 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.445611  1716 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:03:34.445379  7139 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:34.446686  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 209121us
W20260709 13:03:34.450098  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 191762us
I20260709 13:03:34.450429  7312 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:34.452431  7139 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:34.453059  7210 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:34.453699  1609 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:34.454207  7210 raft_consensus.cc:3055] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:34.454537  7139 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:34.455168  7139 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.455531  7139 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:34.457973  1681 leader_election.cc:400] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14.
I20260709 13:03:34.458328  1681 leader_election.cc:403] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:03:34.460572  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:34.462366  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:34.464663  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:34.469326  1532 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:34.471073  7145 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:03:34.472533  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 45829us
I20260709 13:03:34.389258  7246 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:34.440553  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:34.474331  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:34.471800  7145 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:34.476236  7145 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:34.479103  7254 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:34.479599  7254 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.479960  7254 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:34.483158  7185 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:34.483680  6975 consensus_queue.cc:1059] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:34.484112  7168 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.484884  6975 raft_consensus.cc:3055] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:34.485420  6975 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:34.479180  7310 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:34.487584  7310 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.487984  7310 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:34.489096  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:34.489933  1567 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:34.467854  1713 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:34.503613  1532 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:34.491889  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:34.510885  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:34.514597  2299 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:03:34.513360  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:34.519675  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 61849.5us
I20260709 13:03:34.366418  7126 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.531579  1713 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:03:34.523777  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:34.515803  1609 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:34.536098  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:34.526885  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:34.453070  1533 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:34.528911  1715 raft_consensus.cc:2410] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:03:34.543576  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:34.545032  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 71795.3us
I20260709 13:03:34.454619  7210 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:34.486786  7185 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:34.551214  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 82161.1us
I20260709 13:03:34.509413  6984 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.530678  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:34.545485  1683 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:34.557690  1560 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:03:34.486439  6975 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:34.546253  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:34.561326  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 120856us
I20260709 13:03:34.516470  6946 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:34.563369  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:34.569767  1682 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:34.570052  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:34.570551  1534 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:34.565900  1712 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:34.572479  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:34.533102  7087 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:34.553886  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:34.577749  1643 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:34.530565  6959 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14.
I20260709 13:03:34.581322  1567 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:34.550832  7178 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:34.586387  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:34.582998  1709 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:34.584408  1682 leader_election.cc:400] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14.
I20260709 13:03:34.569710  7312 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:34.592772  1682 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:34.595719  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:34.596228  6868 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:34.596570  6722 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:34.597339  6722 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.598382  7037 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:34.585924  7095 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:34.594313  7210 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:34.599787  1710 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:34.599884  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 54504.1us
I20260709 13:03:34.600703  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:34.603161  7232 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:34.603623  7232 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.604012  7232 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:34.600021  7210 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:34.573750  6946 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:34.598835  7037 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:34.636835  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 44189.4us
I20260709 13:03:34.590364  1607 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:34.616291  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 163311us
I20260709 13:03:34.641141  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:34.623214  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:34.581379  7230 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:34.634574  1713 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:34.635308  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:34.556018  7185 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.653285  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:34.654251  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:34.655480  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:34.661983  1560 raft_consensus.cc:2410] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:03:34.579061  6712 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:03:34.664460  1716 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:34.639249  6946 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:34.569003  7137 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:03:34.669742  1682 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:34.664913  1717 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:34.638687  6975 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:34.648198  7037 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:34.690430  2296 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:34.695828  7210 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:34.660646  7185 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:34.696920  1713 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:34.653730  6868 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:34.697970  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 127515us
I20260709 13:03:34.704766  1534 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:34.709545  1714 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:03:34.706811  1534 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:34.665678  6955 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:34.589653  6984 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.737069  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:34.690820  6271 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.689611  6530 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:34.684413  6712 consensus_queue.cc:1243] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:03:34.748063  6271 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:34.738638  7302 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 13:03:34.757055  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 77812.2us
I20260709 13:03:34.684064  7323 consensus_queue.cc:1059] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:03:34.712810  7230 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:34.762850  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:34.768147  1567 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:34.767230  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 60493.5us
I20260709 13:03:34.768985  6271 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:34.769780  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:34.761549  1564 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:03:34.746490  6955 raft_consensus.cc:697] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:03:34.764000  7326 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:03:34.765301  7286 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:34.766748  7146 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.781344  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:34.779512  6617 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:34.822860  1709 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:34.780649  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:34.790973  7194 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:34.792455  6984 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:34.799132  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:34.801113  1567 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:34.839360  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:34.842231  1709 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
W20260709 13:03:34.844444  7333 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:03:34.845089  7069 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:34.850129  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 268091us
I20260709 13:03:34.852109  7210 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:34.880563  1681 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:34.877871  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:34.867343  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:34.824715  6722 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:34.832755  7194 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:34.779775  6733 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.087s
I20260709 13:03:34.859792  1569 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:34.862462  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:34.933140  1563 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:34.895965  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 226310us
I20260709 13:03:34.899228  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:34.885360  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:34.934208  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:34.920768  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:34.885381  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:34.935475  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:34.838169  7178 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:34.948212  1567 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
W20260709 13:03:34.894749  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 114195us
I20260709 13:03:34.857955  7095 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:03:34.878741  7326 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:03:34.953084  1562 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:34.914285  7296 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 13:03:34.911221  7343 log.cc:927] Time spent T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:03:34.958613  1569 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:34.970860  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:34.970949  6959 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:34.970247  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:34.958729  7286 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:34.977749  6946 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:34.978801  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 96300.1us
W20260709 13:03:34.949268  7337 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:03:34.926908  6712 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:34.931874  7194 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:34.911406  7324 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:35.011044  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 56511.2us
I20260709 13:03:35.012296  1709 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:35.015856  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 45709.4us
I20260709 13:03:35.020879  6712 raft_consensus.cc:1064] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:03:34.980469  7178 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.983292  7292 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.023404  7292 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:35.023649  6946 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.023881  7292 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:35.024971  7292 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:35.027452  7270 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.028014  7270 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.030145  7270 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.031489  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:35.034444  7310 consensus_meta.cc:287] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.004s
I20260709 13:03:35.043061  1639 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:35.050722  2296 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:35.042387  6617 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:03:35.040899  6722 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:03:35.062865  6617 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.063437  6617 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.043540  7347 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:03:35.065562  6617 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.048740  6637 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:35.067694  6822 consensus_meta.cc:287] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.628s	user 0.012s	sys 0.000s
I20260709 13:03:35.067713  6637 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:34.963872  7315 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.314s
I20260709 13:03:35.015210  7069 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.075258  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:35.075362  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:35.129947  2299 raft_consensus.cc:2376] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13. Re-sending same reply.
W20260709 13:03:35.043583  7316 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:03:35.034900  7310 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:35.040194  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:34.858218  7146 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.120054  2298 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:35.074747  6530 raft_consensus.cc:3055] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:35.122681  2304 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:34.886622  6955 consensus_queue.cc:237] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:35.132427  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 47706us
I20260709 13:03:35.136759  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:35.144126  2304 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:35.148229  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:35.151887  1642 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:35.148679  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:35.156152  1641 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:35.136049  7106 consensus_meta.cc:287] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.735s	user 0.002s	sys 0.010s
W20260709 13:03:35.144701  7232 consensus_meta.cc:287] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.540s	user 0.013s	sys 0.000s
I20260709 13:03:35.164633  1643 pending_rounds.cc:85] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 7
I20260709 13:03:35.165710  1643 pending_rounds.cc:107] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:03:35.166000  1643 raft_consensus.cc:2889] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:35.153105  2304 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.164755  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:35.166988  2304 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:35.172818  1638 raft_consensus.cc:3055] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:35.173257  1638 raft_consensus.cc:740] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:35.168823  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:35.143600  6530 raft_consensus.cc:740] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:35.175841  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:35.147313  7315 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:35.177569  1644 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
W20260709 13:03:35.159235  7087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.026s	sys 0.059s
I20260709 13:03:35.146677  7185 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.176362  6946 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:35.177842  1567 raft_consensus.cc:2410] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:03:35.137370  7137 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:03:35.183110  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:35.187664  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 147805us
W20260709 13:03:35.188437  7087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.026s	sys 0.059s
I20260709 13:03:35.146872  7207 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:03:35.186877  1638 consensus_queue.cc:260] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:35.224314  7358 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:35.225982  7185 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:35.191440  7255 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:35.197398  7315 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:35.223176  6637 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.226197  7310 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:35.176714  7270 consensus_queue.cc:1059] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:03:35.173627  7232 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.264565  7146 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.266511  7315 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:35.266944  7358 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.267938  7360 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
W20260709 13:03:35.290197  7254 consensus_meta.cc:287] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.810s	user 0.008s	sys 0.004s
I20260709 13:03:35.272444  1607 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:35.330616  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 58244.1us
I20260709 13:03:35.195596  7087 raft_consensus.cc:697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:35.340000  7087 consensus_queue.cc:237] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:35.339337  1462 catalog_manager.cc:5697] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 11 to 13, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 13 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:03:35.277743  1563 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
W20260709 13:03:35.278824  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.276520  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:35.279343  1638 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:35.279595  2297 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:35.278839  7323 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.352409  7323 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.284113  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:35.244342  7246 consensus_meta.cc:287] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.731s	user 0.010s	sys 0.002s
I20260709 13:03:35.292613  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:35.296615  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:35.308153  7357 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:03:35.294036  7210 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:35.320696  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:35.321942  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:35.391628  1641 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:35.270185  7324 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:35.398049  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:35.291327  7037 consensus_meta.cc:287] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.548s	user 0.006s	sys 0.006s
I20260709 13:03:35.341804  1462 catalog_manager.cc:5697] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 10 to 12, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 12 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:35.351368  7232 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.351629  7095 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:35.344442  1681 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:35.359993  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:35.413228  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:35.363893  7185 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:35.203302  6712 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 12, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.356778  2300 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:35.385596  2304 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 12. (term mismatch)
I20260709 13:03:35.427129  2304 pending_rounds.cc:85] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 11
I20260709 13:03:35.427392  2304 pending_rounds.cc:107] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 11.12
W20260709 13:03:35.346804  7139 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.891s	user 0.012s	sys 0.001s
I20260709 13:03:35.362058  1639 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:35.412946  1638 raft_consensus.cc:1275] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:03:35.433655  1638 pending_rounds.cc:85] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 4
I20260709 13:03:35.434010  1638 pending_rounds.cc:107] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:03:35.434317  1638 raft_consensus.cc:2889] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:35.412070  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 127826us
I20260709 13:03:35.436255  1638 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:35.354885  7323 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.356822  6946 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:35.425556  1636 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:03:35.426635  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 66781.1us
I20260709 13:03:35.355909  7246 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.427666  2304 raft_consensus.cc:2889] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:35.428696  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 84346.1us
I20260709 13:03:35.431239  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:35.444897  2299 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:35.399806  7207 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:35.445200  1681 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:35.446601  1683 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:35.453805  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:35.455845  1636 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:03:35.457010  7312 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.457789  7312 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.356504  7254 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.460685  7254 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:35.463219  7246 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.464053  1715 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:35.464752  7323 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:35.474287  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:35.475416  6946 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:35.429188  7210 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.480690  7194 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.481851  7210 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.423177  7095 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:35.486610  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:35.487555  1716 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 13: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:35.489334  1717 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:35.489773  7346 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:35.412727  7324 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.490257  7346 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:35.490556  7324 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:35.491285  1718 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:03:35.491438  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:35.492311  1712 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:35.492959  7324 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:35.493732  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:35.494429  1718 raft_consensus.cc:2393] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:35.404740  7358 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:35.497826  6975 consensus_meta.cc:287] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.789s	user 0.010s	sys 0.000s
I20260709 13:03:35.450460  7178 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:35.416244  7037 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.486799  7168 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.505563  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:35.507258  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 60748.8us
W20260709 13:03:35.508634  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.510370  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:35.511426  2296 raft_consensus.cc:2368] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17.
I20260709 13:03:35.513146  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:35.504123  2299 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:35.516036  7194 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.518018  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:35.521185  7312 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.431463  7185 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.506842  7358 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:35.447418  7361 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:35.449661  7139 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:35.514449  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 62118.6us
I20260709 13:03:35.522892  1716 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:03:35.532729  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2512) took 1136 ms. Trace:
I20260709 13:03:35.550722  1714 rpcz_store.cc:276] 0709 13:03:34.396635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.396757 (+   122us) service_pool.cc:224] Handling call
0709 13:03:34.399683 (+  2926us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:34.710106 (+310423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.532321 (+822215us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b5400474354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:35.532610 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711800704}
I20260709 13:03:35.545450  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:35.525643  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 86883.6us
I20260709 13:03:35.557920  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:35.564633  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:35.564476  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:35.565811  1683 leader_election.cc:400] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17.
I20260709 13:03:35.566170  1683 leader_election.cc:403] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 13:03:35.567900  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 59480.8us
I20260709 13:03:35.568698  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:35.574883  1710 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:35.580402  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:35.583339  1642 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:35.577653  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:35.584511  1641 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:35.587397  1640 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:35.591707  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:35.599045  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:35.611711  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:35.612633  1709 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:35.536269  7037 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:35.534132  7185 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:35.607370  1712 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:35.539196  7139 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.559520  7255 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:35.556759  7256 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:35.625411  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:35.680042  1710 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:35.568243  7106 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:03:35.635566  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 47090.5us
W20260709 13:03:35.677744  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 109132us
I20260709 13:03:35.560705  7254 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.075s
W20260709 13:03:35.688648  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 129886us
I20260709 13:03:35.690325  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:35.565963  7207 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:03:35.626859  7178 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:35.534735  7358 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.677287  6990 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:35.684332  7367 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:35.715230  1709 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:35.717036  1642 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:35.679320  7137 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.683112  7255 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:35.523680  7346 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.679595  7256 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:35.707832  7324 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.687379  7357 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 13:03:35.741708  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.706470  7315 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:03:35.726064  6990 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:35.719193  7310 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
W20260709 13:03:35.749218  7364 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:03:35.749303  6990 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.764163  7367 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17.
I20260709 13:03:35.722863  1641 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:35.772774  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:35.773659  1641 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 13: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:35.774513  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:35.768900  1462 catalog_manager.cc:5697] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 10 to 11, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 11 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:35.730654  7255 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:35.704291  1644 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:35.783416  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:35.787714  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:35.788537  1563 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:35.779069  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:35.790030  1641 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:35.808298  1710 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:35.745528  7346 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:35.792662  1565 raft_consensus.cc:1275] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:03:35.811956  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:35.813211  1567 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:03:35.816152  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:35.815009  1641 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:35.724459  6975 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:03:35.725116  7137 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.817049  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:35.835484  1564 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:35.705703  7106 raft_consensus.cc:697] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:35.825140  1641 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:35.829437  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:35.821213  1567 pending_rounds.cc:85] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 8
I20260709 13:03:35.845165  1567 pending_rounds.cc:107] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:03:35.740134  7324 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:35.845504  1567 raft_consensus.cc:2889] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:35.845453  1641 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:35.867312  1641 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:35.782349  7194 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:35.780110  7312 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:03:35.876286  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:35.876919  1561 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:35.880398  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:35.885802  1569 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:35.873023  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:35.843252  7361 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:03:35.888293  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:35.889394  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:35.889907  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:35.890440  7106 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.895366  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:35.897058  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:35.779613  7210 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:03:35.899559  2299 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:35.900820  7358 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:35.900838  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 1258) took 1475 ms. Trace:
I20260709 13:03:35.828289  7323 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:35.846583  7369 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:03:35.903856  1639 rpcz_store.cc:276] 0709 13:03:34.425102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.473366 (+ 48264us) service_pool.cc:224] Handling call
0709 13:03:35.900656 (+1427290us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b5400155254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:35.900755 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204436864}
I20260709 13:03:35.903863  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:35.908396  1533 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:35.922745  1560 raft_consensus.cc:2368] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:35.937161  1563 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:03:35.939461  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:35.960146  6530 consensus_queue.cc:260] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.963204  1560 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 13:03:35.963698  1560 pending_rounds.cc:85] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 13
I20260709 13:03:35.963974  1560 pending_rounds.cc:107] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 13:03:35.964239  1560 raft_consensus.cc:2889] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:35.911170  7194 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:35.985795  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:35.988003  7194 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:35.990397  7324 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:35.990957  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:35.991556  1565 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:35.910210  7256 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:35.999888  7359 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:36.003037  1565 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
W20260709 13:03:36.010206  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 383033us
I20260709 13:03:36.011629  2268 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:36.014422  1641 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:36.023485  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:36.013586  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 285955us
I20260709 13:03:36.029889  1563 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:03:36.027967  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:36.032826  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:36.037654  2305 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:36.038118  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:36.006654  7310 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:36.026854  7371 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.208s	user 0.000s	sys 0.005s
I20260709 13:03:36.048339  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:36.049706  1532 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:03:36.037398  7087 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.038s	sys 0.055s
W20260709 13:03:36.053565  7087 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.039s	sys 0.056s
I20260709 13:03:36.055045  1681 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:36.031661  7380 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:36.034492  7361 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:03:36.069375  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 302005us
I20260709 13:03:36.115412  1561 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:03:36.031173  7324 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:03:36.124727  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:36.054100  6951 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:36.125085  1560 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:36.064313  7360 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:36.089733  7315 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.042701  6955 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:03:36.136307  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:36.042389  7254 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.055s
W20260709 13:03:36.200217  1459 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45142 (request call id 351) took 19051 ms (client timeout 15000 ms). Trace:
I20260709 13:03:36.200642  7200 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:03:36.201397  1534 leader_election.cc:336] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.591s (SENT)
I20260709 13:03:36.236021  1534 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:36.236268  1564 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:36.202234  1462 catalog_manager.cc:5697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 10 to 12, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 12 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:36.203346  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1427) took 1129 ms. Trace:
I20260709 13:03:36.205468  6530 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:36.211885  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 521669us
I20260709 13:03:36.212643  7139 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:36.213246  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2929) took 1063 ms. Trace:
W20260709 13:03:36.214699  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.243868  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:36.245843  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:36.215065  7178 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:36.215783  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2930) took 1065 ms. Trace:
I20260709 13:03:36.069631  7194 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:36.047048  7310 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:36.063601  7207 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:36.189229  7372 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 13:03:36.048000  7230 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:03:36.226722  1717 raft_consensus.cc:2410] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 13 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:03:36.199330  7315 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:36.237205  2297 rpcz_store.cc:276] 0709 13:03:35.073712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.074998 (+  1286us) service_pool.cc:224] Handling call
0709 13:03:36.188329 (+1113331us) spinlock_profiling.cc:243] Waited 38.8 ms on lock 0x7b54001112d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:36.203282 (+ 14953us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89280896}
I20260709 13:03:36.238273  6712 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.221242  6868 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.150836  6951 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:36.250543  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:36.310323  1638 raft_consensus.cc:2368] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17.
I20260709 13:03:36.252367  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:36.273945  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1434) took 1151 ms. Trace:
I20260709 13:03:36.225349  7323 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:36.224376  7354 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:36.243538  1643 rpcz_store.cc:276] 0709 13:03:35.149650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.158842 (+  9192us) service_pool.cc:224] Handling call
0709 13:03:35.159302 (+   460us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:35.173534 (+ 14232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:35.173565 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:35.176647 (+  3082us) log.cc:844] Serialized 52 byte log entry
0709 13:03:35.289740 (+113093us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:35.707179 (+417439us) log.cc:844] Serialized 14 byte log entry
0709 13:03:36.212685 (+505506us) log.cc:844] Serialized 14 byte log entry
0709 13:03:36.212892 (+   207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.212983 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:36.213025 (+    42us) raft_consensus.cc:1685] finished
0709 13:03:36.213038 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.213173 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":732680,"prepare.queue_time_us":241109,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":90,"raft.queue_time_us":509238,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":1558,"thread_start_us":276273,"threads_started":2,"wal-append.queue_time_us":175331}
W20260709 13:03:36.283748  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 375575us
I20260709 13:03:36.284794  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:36.276961  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:36.126679  7292 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.122s
I20260709 13:03:36.247354  1644 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:36.256088  1635 rpcz_store.cc:276] 0709 13:03:35.149913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.150229 (+   316us) service_pool.cc:224] Handling call
0709 13:03:35.164460 (+ 14231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:35.171685 (+  7225us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:35.705587 (+533902us) log.cc:844] Serialized 14 byte log entry
0709 13:03:36.215443 (+509856us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:36.215495 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:36.215538 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.215599 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.215723 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":482620,"prepare.queue_time_us":346631,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":154659,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":63,"thread_start_us":542135,"threads_started":2}
I20260709 13:03:36.295249  1462 catalog_manager.cc:5697] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:36.306124  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:36.311439  2301 rpcz_store.cc:276] 0709 13:03:35.122232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.122348 (+   116us) service_pool.cc:224] Handling call
0709 13:03:35.155114 (+ 32766us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:35.167939 (+ 12825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:35.167973 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:35.227929 (+ 59956us) log.cc:844] Serialized 26 byte log entry
0709 13:03:35.578781 (+350852us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:35.639102 (+ 60321us) log.cc:844] Serialized 14 byte log entry
0709 13:03:35.674426 (+ 35324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:35.674499 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:35.966434 (+291935us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:36.249097 (+282663us) raft_consensus.cc:1685] finished
0709 13:03:36.273708 (+ 24611us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.273870 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44553,"prepare.queue_time_us":60089,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":50958,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":26010,"spinlock_wait_cycles":11291904,"thread_start_us":271467,"threads_started":1,"wal-append.queue_time_us":460191}
I20260709 13:03:36.255190  7137 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:36.323582  1640 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:36.325136  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:36.326138  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:36.285033  7358 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.329701  1638 raft_consensus.cc:2410] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:03:36.334384  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:36.334455  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 243705us
I20260709 13:03:36.306411  7168 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:36.321467  6951 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:36.357463  1462 catalog_manager.cc:5697] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 3 to 11. New cstate: current_term: 11 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:36.374804  1642 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:03:36.324905  7109 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.293071  7286 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:03:36.326077  7367 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:03:36.399462  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 307486us
I20260709 13:03:36.328099  7069 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:36.307301  7178 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:36.405678  7178 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:36.406579  7178 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:36.393508  7095 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.910s	user 0.009s	sys 0.005s
I20260709 13:03:36.406888  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:36.332994  6946 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:36.363060  7312 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:36.407982  2268 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:36.314765  6712 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:36.321998  6868 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:36.327394  7323 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:36.451407  1462 catalog_manager.cc:5697] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 4 to 11, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 11 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:36.463521  1462 catalog_manager.cc:5697] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:36.468537  1462 catalog_manager.cc:5697] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 10 to 12, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 12 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:03:36.472258  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 64341.7us
I20260709 13:03:36.409065  7146 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:36.473166  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:36.327109  7354 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:36.395200  7210 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.090s
I20260709 13:03:36.360687  7358 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:36.490198  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.470727  1462 catalog_manager.cc:5697] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 8 to 10, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 10 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:03:36.500825  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 283050us
I20260709 13:03:36.403385  7109 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:36.516403  1709 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:36.432778  7367 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:36.528170  1562 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:36.495666  7037 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:03:36.517443  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 192397us
I20260709 13:03:36.466197  6946 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:36.536388  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 335033us
I20260709 13:03:36.533968  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:36.459452  7095 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:36.529892  1462 catalog_manager.cc:5697] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:03:36.541289  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 207037us
W20260709 13:03:36.535437  7374 log.cc:927] Time spent T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:03:36.521826  7361 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:36.524395  7360 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.534127  7194 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:36.562065  1462 catalog_manager.cc:5697] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 6 to 11, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 11 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:36.535256  7394 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:36.565258  1567 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:36.539762  7380 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:36.599068  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 60825.6us
I20260709 13:03:36.569362  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:36.599720  1462 catalog_manager.cc:5697] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 6 to 9, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:36.544644  7385 raft_consensus.cc:3055] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:36.605695  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:36.618659  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 49387.1us
I20260709 13:03:36.620077  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:36.621220  7139 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:36.541244  7350 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:36.623018  1607 leader_election.cc:400] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:36.602773  7210 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.541175  7390 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.672122  1607 leader_election.cc:403] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:03:36.555893  7387 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.616842  7292 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.612704  7385 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:36.649379  1462 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 10 to 11, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:03:36.564026  7373 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 13:03:36.556046  7069 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:36.621886  7315 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:36.682387  7382 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 13:03:36.698447  6712 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:36.560329  7361 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:36.679976  7210 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:36.701013  7361 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:36.701699  7210 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:36.617497  7380 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:36.702098  7385 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:36.703652  7385 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:36.704514  7380 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:36.560243  7360 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:36.705471  7380 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:36.707458  7360 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:36.687554  7387 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:36.598970  7109 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:36.623016  7286 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:36.795315  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 172483us
I20260709 13:03:36.809087  1607 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:36.802809  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3038) took 1020 ms. Trace:
I20260709 13:03:36.682365  6868 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:36.598114  7194 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:36.847358  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:36.848032  1642 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:36.849476  6946 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:36.820724  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:36.850155  6946 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:36.806326  1462 catalog_manager.cc:5697] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 10 to 11, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:36.687373  7292 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:36.856974  1709 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16. Ops: [13.7-15.8]
I20260709 13:03:36.855715  1682 leader_election.cc:400] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17.
I20260709 13:03:36.857372  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:36.852444  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.855724  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:36.803337  7139 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.851848  1462 catalog_manager.cc:5697] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:36.794361  7387 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:36.855687  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2565) took 2362 ms. Trace:
I20260709 13:03:36.874172  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:36.876520  7139 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:36.846521  1568 rpcz_store.cc:276] 0709 13:03:35.782240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.784392 (+  2152us) service_pool.cc:224] Handling call
0709 13:03:35.816676 (+ 32284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:36.001130 (+184454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.001181 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:36.001242 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:36.001291 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.721382 (+720091us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54000a78d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:36.802478 (+ 81096us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:36.802718 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423659264}
I20260709 13:03:36.878762  1566 rpcz_store.cc:276] 0709 13:03:34.492884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:34.493016 (+   132us) service_pool.cc:224] Handling call
0709 13:03:36.855471 (+2362455us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400200654. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:36.855610 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3376123264}
W20260709 13:03:36.231374  1459 rpcz_store.cc:269] 0709 13:03:17.148270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:17.154482 (+  6212us) service_pool.cc:224] Handling call
0709 13:03:36.199868 (+19045386us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:28.969882 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:29.066619 (+ 96737us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:29.066678 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:03:29.066699 (+    21us) tablet.cc:662] Decoding operations
0709 13:03:29.070049 (+  3350us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:29.070179 (+   130us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:29.070208 (+    29us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:03:29.192599 (+122391us) tablet.cc:701] Row locks acquired
0709 13:03:29.192658 (+    59us) write_op.cc:260] PREPARE: finished
0709 13:03:29.192838 (+   180us) write_op.cc:270] Start()
0709 13:03:29.192996 (+   158us) write_op.cc:276] Timestamp: P: 1783602209192768 usec, L: 0
0709 13:03:29.193036 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:03:29.193818 (+   782us) log.cc:844] Serialized 8219 byte log entry
0709 13:03:29.689289 (+495471us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:30.274661 (+585372us) op_driver.cc:464] REPLICATION: finished
0709 13:03:30.413168 (+138507us) write_op.cc:301] APPLY: starting
0709 13:03:30.413273 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:30.933147 (+519874us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:30.933201 (+    54us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:31.249399 (+316198us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:31.291459 (+ 42060us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 13:03:31.291538 (+    79us) write_op.cc:312] APPLY: finished
0709 13:03:31.303681 (+ 12143us) log.cc:844] Serialized 188 byte log entry
0709 13:03:31.568125 (+264444us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:31.582374 (+ 14249us) write_op.cc:454] Released schema lock
0709 13:03:31.604354 (+ 21980us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:31.622680 (+ 18326us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:31.622740 (+    60us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2202234,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":119660,"apply.run_cpu_time_us":56695,"apply.run_wall_time_us":1205026,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91949,"num_ops":21,"prepare.queue_time_us":217087,"prepare.run_cpu_time_us":116736,"prepare.run_wall_time_us":562165,"raft.queue_time_us":277057,"raft.run_cpu_time_us":31830,"raft.run_wall_time_us":64107,"replication_time_us":1081428,"spinlock_wait_cycles":3218688,"thread_start_us":890123,"threads_started":5,"wal-append.queue_time_us":557173,"wal-append.run_cpu_time_us":31526,"wal-append.run_wall_time_us":2801660}]]}
I20260709 13:03:36.877813  1682 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:36.894289  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:36.815791  7393 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:36.889338   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:36.841332  7069 raft_consensus.cc:697] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:36.855281  7286 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.852795  7398 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:36.904767  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:36.908213  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:36.875451  7323 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:36.842748  7402 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:36.914264  1640 raft_consensus.cc:2393] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 12: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:36.687801  7390 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:36.850744  6946 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:36.804491  7350 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:03:36.875202  7380 consensus_queue.cc:1059] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:03:36.898080  7168 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:03:36.918802  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 374288us
I20260709 13:03:36.918601  1643 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:03:36.920246  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 314657us
W20260709 13:03:36.919468  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 379654us
W20260709 13:03:36.925012  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 273720us
W20260709 13:03:36.921850  7388 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.394s	user 0.000s	sys 0.001s
I20260709 13:03:36.913609  6712 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:36.990320  7392 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.431s	user 0.000s	sys 0.001s
I20260709 13:03:36.914572  7358 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:36.991710  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:36.994444  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:36.995780  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.000560  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:37.020123  1710 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:37.001727  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:37.002875  6946 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:37.003541  7386 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.396s	user 0.000s	sys 0.001s
I20260709 13:03:37.003872  7404 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:36.998369  1533 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:37.012076  1640 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:03:36.974747  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:36.997679  7367 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:36.924463  7398 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:37.006222  7178 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:36.926314  7095 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:37.028656  7168 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:36.912981  7286 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:37.042254  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:37.054360  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 56082.1us
I20260709 13:03:37.019407  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:37.024456  7360 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:37.025136  7361 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:37.002892  7395 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:03:37.063509  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:37.072016  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:37.065255  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:37.084257  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:37.085338  1640 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 15: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:37.062191  7359 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:37.094305  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:37.055917  6946 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
W20260709 13:03:37.089143  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 63614.3us
W20260709 13:03:37.140832  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 53706.5us
I20260709 13:03:37.146121  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2507) took 1718 ms. Trace:
W20260709 13:03:37.082455  7255 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.267s	user 0.017s	sys 0.012s
W20260709 13:03:37.147425  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.150005  7412 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:37.145690  7087 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:03:37.151613  7087 consensus_queue.cc:1243] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:37.148386  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:37.146737  1713 rpcz_store.cc:276] 0709 13:03:35.427604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.427763 (+   159us) service_pool.cc:224] Handling call
0709 13:03:35.428112 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:35.428486 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:35.428518 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.008419 (+579901us) spinlock_profiling.cc:243] Waited 4.19 ms on lock 0x7b2c0010a498. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:03:36.014086 (+  5667us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.399536 (+385450us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:36.400130 (+   594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.401640 (+  1510us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.138216 (+736576us) raft_consensus.cc:1685] finished
0709 13:03:37.138240 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.146022 (+  7782us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5190,"raft.queue_time_us":11385,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":3241,"spinlock_wait_cycles":9640832,"thread_start_us":344089,"threads_started":1,"wal-append.queue_time_us":481065}
I20260709 13:03:37.155644  1644 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:37.060019  7178 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:37.072695  7402 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:37.038407  7069 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:37.160732  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 95534.9us
I20260709 13:03:37.166678  2301 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:37.083735  7207 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:37.072650  7360 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.188916  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:37.161942  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:37.144872  7367 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:37.196593  1534 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:37.140832  7095 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:03:37.061722  7404 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.197618  7359 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:37.209672  7408 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 13:03:37.224783  7037 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.005s	sys 0.009s
W20260709 13:03:37.208513  7416 log.cc:927] Time spent T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:03:37.297834  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2928) took 2137 ms. Trace:
W20260709 13:03:37.257694  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 61180.5us
W20260709 13:03:37.215317  7411 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:03:37.160513  7255 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.189883  7406 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:37.298298  1637 rpcz_store.cc:276] 0709 13:03:35.149361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.149553 (+   192us) service_pool.cc:224] Handling call
0709 13:03:35.164845 (+ 15292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:35.170751 (+  5906us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:35.497717 (+326966us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b18002fd160. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb2ec0a 00007fd7a826e219 00007fd79ebb6a40 00007fd79ebb14e4 00007fd79ebb1455 00007fd7a56993c7 00007fd7a1cab9a7 00007fd7a1cbb0bf 00007fd7a1cbb038
0709 13:03:35.819979 (+322262us) log.cc:844] Serialized 14 byte log entry
0709 13:03:36.331173 (+511194us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.355102 (+ 23929us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.355321 (+   219us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:36.356345 (+  1024us) log.cc:844] Serialized 14 byte log entry
0709 13:03:36.537854 (+181509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.537931 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.286671 (+748740us) raft_consensus.cc:1685] finished
0709 13:03:37.286700 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.286866 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36330,"prepare.queue_time_us":347473,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":69,"raft.queue_time_us":184786,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":24617,"spinlock_wait_cycles":4640000,"thread_start_us":881666,"threads_started":3,"wal-append.queue_time_us":571419}
I20260709 13:03:37.211669  7395 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:37.216460  7139 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:37.204654  7417 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:37.209723  7401 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:37.207111  7292 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:37.298067  7037 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.006s	sys 0.009s
I20260709 13:03:37.199002  7207 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:37.309160  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 147503us
I20260709 13:03:37.291792  6975 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:37.336760  7441 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:37.337852  7360 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:37.288751  7404 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:37.309638  7346 consensus_meta.cc:287] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.475s	user 0.014s	sys 0.000s
I20260709 13:03:37.365778  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:37.343675  2304 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:37.183815  7402 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.200837  7398 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.292273  7359 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.354089  7194 raft_consensus.cc:3055] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:37.333925  7139 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:37.365219  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:37.352265  6946 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:37.307446  7350 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:37.352339  7095 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:37.370390  7436 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:37.377909  7436 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:37.380191  7436 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:37.380236  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2523) took 1257 ms. Trace:
I20260709 13:03:37.380599  7402 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:37.381134  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3047) took 1055 ms. Trace:
I20260709 13:03:37.381460  1717 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18. Ops: []
W20260709 13:03:37.382349  7422 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.185s	user 0.000s	sys 0.000s
I20260709 13:03:37.376487  7350 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:37.383805  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:37.384747  2305 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:37.360014  7352 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.367713  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2939) took 1122 ms. Trace:
I20260709 13:03:37.368953  7447 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:03:37.372473  7414 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:03:37.372857  7407 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
W20260709 13:03:37.401252  7383 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 1.091s	user 0.002s	sys 0.004s
I20260709 13:03:37.397099  7404 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:37.386217  7185 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:37.402733  7404 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:37.403455  7256 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:37.404083  7185 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.404547  7185 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:37.405128  7404 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:37.374572  7194 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:37.353885  7037 raft_consensus.cc:697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:37.408279  7037 consensus_queue.cc:237] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.401302  7448 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:37.409966  7448 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.411438  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1604) took 1854 ms. Trace:
I20260709 13:03:37.412592  7448 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:37.409408  7414 raft_consensus.cc:697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:37.376241  6946 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:37.413936  6946 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:37.204367  7367 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.406744  7194 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.416086  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:37.377311  7095 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.416479  7367 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:37.315250  7395 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:37.416982  7438 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:37.417550  7438 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:37.418160  7350 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:37.418471  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:37.418701  7395 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:37.366791  7441 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.419798  7438 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:37.419500  7352 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.420403  7405 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.395789  1641 rpcz_store.cc:276] 0709 13:03:36.245602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.245726 (+   124us) service_pool.cc:224] Handling call
0709 13:03:36.294068 (+ 48342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.294487 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.294528 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.333652 (+ 39124us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.880921 (+547269us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:36.880976 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.881071 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.367466 (+486395us) raft_consensus.cc:1685] finished
0709 13:03:37.367496 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.367643 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":622,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":15215,"thread_start_us":428058,"threads_started":1,"wal-append.queue_time_us":646873}
I20260709 13:03:37.420929  7405 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.413810  7414 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.422106  7352 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:37.394649  7385 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.691s	user 0.007s	sys 0.015s
W20260709 13:03:37.423358  6951 consensus_meta.cc:287] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.022s	user 0.008s	sys 0.004s
I20260709 13:03:37.423820  6951 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.421375  7441 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:37.425160  2299 raft_consensus.cc:2410] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 13 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:03:37.426107  6951 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:37.427170  7359 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:37.429467  6530 consensus_meta.cc:287] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.175s	user 0.009s	sys 0.002s
I20260709 13:03:37.430238  1716 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:37.411993  2296 rpcz_store.cc:276] 0709 13:03:35.557319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.557616 (+   297us) service_pool.cc:224] Handling call
0709 13:03:35.558026 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:35.621287 (+ 63261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:35.621329 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:35.621919 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.045477 (+423558us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:36.045555 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.045812 (+   257us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.400795 (+1354983us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b24002c95b8. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79ea3a9c4 00007fd79ea38acb 00007fd79ebf3dc9 00007fd79ebf3a55 00007fd79ebf309d 00007fd7a182510e 00007fd7a183c88c 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221
0709 13:03:37.410984 (+ 10189us) raft_consensus.cc:1685] finished
0709 13:03:37.411006 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.411309 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33835,"raft.queue_time_us":68062,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":6277,"spinlock_wait_cycles":79042944,"thread_start_us":321912,"threads_started":1,"wal-append.queue_time_us":540890}
I20260709 13:03:37.431010  7323 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.431686  7323 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.380805  1711 rpcz_store.cc:276] 0709 13:03:36.122782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.122977 (+   195us) service_pool.cc:224] Handling call
0709 13:03:36.123470 (+   493us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:36.488893 (+365423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.488927 (+    34us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:36.524024 (+ 35097us) log.cc:844] Serialized 78 byte log entry
0709 13:03:36.888901 (+364877us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:36.909986 (+ 21085us) log.cc:844] Serialized 14 byte log entry
0709 13:03:36.910944 (+   958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.911239 (+   295us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.379989 (+468750us) raft_consensus.cc:1685] finished
0709 13:03:37.380018 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.380179 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":141689,"prepare.queue_time_us":20030,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":1019,"raft.queue_time_us":95153,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":11469,"thread_start_us":606892,"threads_started":2,"wal-append.queue_time_us":458449}
I20260709 13:03:37.335340  7417 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.418190  7095 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:37.399029  7360 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.381515  1564 rpcz_store.cc:276] 0709 13:03:36.313875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.314066 (+   191us) service_pool.cc:224] Handling call
0709 13:03:36.333065 (+ 18999us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.356631 (+ 23566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.356662 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.369423 (+ 12761us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.907074 (+537651us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:03:36.907133 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.907226 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.368803 (+461577us) raft_consensus.cc:1685] finished
0709 13:03:37.368828 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.368978 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":977,"raft.queue_time_us":63113,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":32356,"thread_start_us":442962,"threads_started":1,"wal-append.queue_time_us":656649}
I20260709 13:03:37.415237  6946 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:37.441979  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:37.444620  1711 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:37.429457  7412 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:37.445334  6951 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:37.448573  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:37.449326  7255 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:37.450346  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:37.448807  6951 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.453205  7146 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:37.453876  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:37.454195  7146 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:37.454730  1718 raft_consensus.cc:2393] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 16: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:37.454998  7310 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:37.457363  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3048) took 1119 ms. Trace:
I20260709 13:03:37.457770  1561 rpcz_store.cc:276] 0709 13:03:36.337563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.337708 (+   145us) service_pool.cc:224] Handling call
0709 13:03:36.376746 (+ 39038us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:36.397117 (+ 20371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.397150 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:36.410888 (+ 13738us) log.cc:844] Serialized 26 byte log entry
0709 13:03:36.876523 (+465635us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:36.876567 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:36.876639 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:37.457155 (+580516us) raft_consensus.cc:1685] finished
0709 13:03:37.457180 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:37.457306 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28398,"raft.queue_time_us":96709,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":3664,"thread_start_us":387607,"threads_started":1,"wal-append.queue_time_us":985338}
I20260709 13:03:37.461858  7404 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:37.463486  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:37.464236  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 70365.2us
I20260709 13:03:37.467202  7404 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:37.445194  7360 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.467795  7404 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:37.468236  7404 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:37.469805  7360 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:37.362879  7401 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.453245  7417 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:37.477450  1609 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:37.479893  7417 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:37.386965  7139 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.481789  1609 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:37.486011  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:37.492939  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:37.493903  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:37.494333  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:37.494715  1640 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:37.494930  1644 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
W20260709 13:03:37.495570  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.496665  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:37.496876  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:37.493856  1711 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:37.500059  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:37.501264  1533 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:37.501760  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:37.503602  1609 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:37.503811  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:37.504711  1644 pending_rounds.cc:85] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 8
I20260709 13:03:37.504951  1644 pending_rounds.cc:107] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:03:37.505096  1644 raft_consensus.cc:2889] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:37.505031  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:37.506332  1533 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:37.507951  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:37.510167  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:37.510980  2301 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:37.482868  7139 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:37.516726  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:37.520008  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:37.520367  1609 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:37.521049  1637 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:03:37.497794  7417 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:37.498103  7095 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:37.526160  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.529490  1609 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:37.510838  7384 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:37.514956  7246 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.538691  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:37.516950  7312 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:37.514534  7069 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:37.533423  7346 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:37.555949  7246 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.534603  7095 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:03:37.551224  7431 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:03:37.538070  7384 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.556171  7312 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.550679  6951 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:37.602766  7384 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:37.609385  7312 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:37.624675  7095 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:37.625028  7401 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:37.626269  7246 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:37.627460  7185 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.630118  7185 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:37.630282  7146 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:37.654070  7146 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:37.692004  7450 log.cc:927] Time spent T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 13:03:38.028878  7404 consensus_meta.cc:287] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.004s
W20260709 13:03:38.029440  7454 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:03:38.088889  7460 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.368s	user 0.001s	sys 0.000s
W20260709 13:03:38.154055  7456 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.149s	user 0.000s	sys 0.003s
W20260709 13:03:38.161438  7461 log.cc:927] Time spent T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
W20260709 13:03:38.274376  7468 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.242s	user 0.000s	sys 0.000s
W20260709 13:03:38.281275  6946 consensus_meta.cc:287] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.821s	user 0.010s	sys 0.002s
I20260709 13:03:38.056290  7384 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:38.151412  7404 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:38.715531  7469 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.273s	user 0.002s	sys 0.000s
W20260709 13:03:38.719192  7470 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
W20260709 13:03:38.674187  7312 consensus_meta.cc:287] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.041s	user 0.013s	sys 0.001s
I20260709 13:03:38.743249  7384 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:38.808049  7312 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:38.986806  7404 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:39.018482  1534 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.174s	user 0.011s	sys 0.013s
W20260709 13:03:39.017560  6975 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.324s	user 0.002s	sys 0.021s
W20260709 13:03:39.019035  1534 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.174s	user 0.011s	sys 0.013s
W20260709 13:03:39.019688  1607 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 3.358s (SENT) [suppressed 4 similar messages]
W20260709 13:03:39.019970  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 1.17541e+06us
W20260709 13:03:39.016644  7456 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.005s	sys 0.014s
W20260709 13:03:39.019338  7467 log.cc:927] Time spent T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 13:03:39.021359  7465 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:39.020634  6975 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.327s	user 0.002s	sys 0.021s
I20260709 13:03:39.021687  7095 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:39.023138  7465 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:39.023412  7095 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:39.025050  7465 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:39.025091  7095 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:39.025614  7465 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:39.026057  7465 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:39.021291  7456 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.006s	sys 0.014s
I20260709 13:03:39.029192  7312 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:39.031185  7465 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:39.022199  7384 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:39.032977  7384 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:39.035755  7384 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:39.041905  7474 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:39.043236  7474 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:39.049127  7474 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:39.084198  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:39.085026  1711 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2537) took 1594 ms (client timeout 2071 ms). Trace:
W20260709 13:03:39.085477  1711 rpcz_store.cc:269] 0709 13:03:37.491072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.083869 (+1592797us) service_pool.cc:224] Handling call
0709 13:03:39.084965 (+  1096us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:39.087286  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:39.088155  1711 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2541) took 1588 ms (client timeout 2065 ms). Trace:
W20260709 13:03:39.088507  1711 rpcz_store.cc:269] 0709 13:03:37.499450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.086425 (+1586975us) service_pool.cc:224] Handling call
0709 13:03:39.088110 (+  1685us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:39.090025  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2535) took 1604 ms. Trace:
I20260709 13:03:39.090386  1711 rpcz_store.cc:276] 0709 13:03:37.485282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.089171 (+1603889us) service_pool.cc:224] Handling call
0709 13:03:39.089457 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.089666 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.089692 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:39.089733 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:39.089763 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.089818 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.089948 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:39.092288  1711 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:03:39.093017  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2536) took 1602 ms. Trace:
I20260709 13:03:39.093338  1711 rpcz_store.cc:276] 0709 13:03:37.490634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.091664 (+1601030us) service_pool.cc:224] Handling call
0709 13:03:39.091953 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.092628 (+   675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.092964 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:39.095548  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2538) took 1604 ms. Trace:
I20260709 13:03:39.095898  1711 rpcz_store.cc:276] 0709 13:03:37.491396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.094715 (+1603319us) service_pool.cc:224] Handling call
0709 13:03:39.095025 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.095205 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.095228 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:39.095266 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:39.095311 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.095378 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.095498 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:39.097945  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2539) took 1606 ms. Trace:
I20260709 13:03:39.098275  1711 rpcz_store.cc:276] 0709 13:03:37.491960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.097098 (+1605138us) service_pool.cc:224] Handling call
0709 13:03:39.097408 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.097600 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.097631 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:39.097682 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:39.097713 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.097768 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.097891 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:39.102058  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2540) took 1606 ms. Trace:
I20260709 13:03:39.102437  1711 rpcz_store.cc:276] 0709 13:03:37.495459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.101057 (+1605598us) service_pool.cc:224] Handling call
0709 13:03:39.101519 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.101726 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:39.101755 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:39.101798 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:39.101829 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.101881 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.102004 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:39.131194  7390 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:39.178978  6990 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:39.179714  6990 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:39.182554  6990 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:39.183235  6990 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:39.183848  6990 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:39.186064  6990 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:39.190210  6990 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:39.190968  6990 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:39.194242  6990 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:39.194859  6990 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.201706  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1149) took 2029 ms. Trace:
I20260709 13:03:39.204000  7413 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:39.205286  1533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.686s	user 0.003s	sys 0.019s
I20260709 13:03:39.204873  7413 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:39.205780  1533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.686s	user 0.003s	sys 0.019s
I20260709 13:03:39.207836  7413 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:39.214182  1567 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:03:39.214699  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.215368  1566 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:39.215377  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.216349  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.216565  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.216684  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.217234  1562 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:39.217375  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.218159  1566 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:39.202194  1710 rpcz_store.cc:276] 0709 13:03:37.172233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.172722 (+   489us) service_pool.cc:224] Handling call
0709 13:03:37.172998 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.254866 (+ 81868us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:03:37.305281 (+ 50415us) log.cc:844] Serialized 14 byte log entry
0709 13:03:37.312606 (+  7325us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:37.312670 (+    64us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:37.312716 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.201433 (+1888717us) spinlock_profiling.cc:243] Waited 81.6 ms on lock 0x7b54003395d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:39.201501 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:39.201642 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":52441,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":456,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":187683200,"thread_start_us":6875,"threads_started":1,"wal-append.queue_time_us":145276,"wal-append.run_cpu_time_us":1079,"wal-append.run_wall_time_us":1001110}
I20260709 13:03:39.217339  1567 raft_consensus.cc:2368] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16.
I20260709 13:03:39.222090  1533 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:39.227074  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:39.228286  7354 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:39.227128  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.196151  6990 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:39.230664  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.231174  7474 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:39.236408  1566 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:39.237905  7479 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:39.238765  1566 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:39.239928  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.240530  1641 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 13:03:39.241042  1641 pending_rounds.cc:85] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 7
I20260709 13:03:39.240607  1566 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:39.241349  1641 pending_rounds.cc:107] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:03:39.241639  1641 raft_consensus.cc:2889] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:39.241896  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.242545  1566 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:39.243850  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.244714  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 1307) took 3362 ms. Trace:
I20260709 13:03:39.245201  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.245134  2304 rpcz_store.cc:276] 0709 13:03:35.882566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.883256 (+   690us) service_pool.cc:224] Handling call
0709 13:03:35.889533 (+  6277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.366527 (+1476994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.244444 (+1877917us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400643f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:39.244630 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3343901056}
I20260709 13:03:39.238485  7479 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:39.248968  7479 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:39.249468  7479 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:39.246706  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.250587  1566 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:39.240898  7478 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.258785  7478 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:39.261206  7478 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:39.261780  7478 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:39.262182  7478 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:39.263274  7478 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:39.264225  1561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:03:39.263908  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.264904  1561 raft_consensus.cc:606] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:39.265246  1561 raft_consensus.cc:614] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:39.265626  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.266155  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:39.266721  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:39.267315  1566 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:39.267534  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:40.031575  1561 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:03:40.033476  1561 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:39.268352  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2557) took 1769 ms. Trace:
W20260709 13:03:39.231823  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:39.233418  6990 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.034945  1641 rpcz_store.cc:276] 0709 13:03:37.499109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.517980 (+ 18871us) service_pool.cc:224] Handling call
0709 13:03:37.541001 (+ 23021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:39.242094 (+1701093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:39.266946 (+ 24852us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b540032bed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:39.268289 (+  1343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3908031232}
W20260709 13:03:40.040297  1681 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.002s	sys 0.000s
W20260709 13:03:40.040848  1681 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.002s	sys 0.000s
W20260709 13:03:40.041283  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 770149us
I20260709 13:03:40.019272  7286 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:40.021224  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2520) took 3167 ms. Trace:
I20260709 13:03:40.044317  1717 rpcz_store.cc:276] 0709 13:03:36.854236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.855831 (+  1595us) service_pool.cc:224] Handling call
0709 13:03:37.381260 (+525429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.381944 (+   684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.020875 (+2638931us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b5400268ed0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:40.021097 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1207640448}
W20260709 13:03:40.024328  1568 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.010s	sys 0.029s
W20260709 13:03:40.045526  1568 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.010s	sys 0.029s
W20260709 13:03:40.051432  1565 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.534s	user 0.009s	sys 0.030s
W20260709 13:03:40.052242  1565 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.537s	user 0.010s	sys 0.031s
I20260709 13:03:40.052964  7396 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:40.055676  7489 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.056272  7489 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.058286  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 990) took 3560 ms. Trace:
I20260709 13:03:40.058781  1565 rpcz_store.cc:276] 0709 13:03:36.497722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.498679 (+   957us) service_pool.cc:224] Handling call
0709 13:03:36.498989 (+   310us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:36.534113 (+ 35124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:36.534141 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:36.534810 (+   669us) log.cc:844] Serialized 52 byte log entry
0709 13:03:37.414902 (+880092us) spinlock_profiling.cc:243] Waited 7.72 ms on lock 0x7b18003eb5a0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:03:37.500690 (+ 85788us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:37.514794 (+ 14104us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.052710 (+2537916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.057897 (+  5187us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.057961 (+    64us) raft_consensus.cc:1685] finished
0709 13:03:40.058080 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.058211 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20444,"prepare.queue_time_us":13938,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":144241,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":52903,"spinlock_wait_cycles":17758080,"thread_start_us":2598283,"threads_started":2,"wal-append.queue_time_us":1056714,"wal-append.run_cpu_time_us":6079,"wal-append.run_wall_time_us":1923012}
I20260709 13:03:40.091329  7489 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:40.093005  1682 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.003s	sys 0.006s
W20260709 13:03:40.094087  1682 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.004s	sys 0.006s
W20260709 13:03:40.095580  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 690821us
I20260709 13:03:40.100281  7486 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.100972  7486 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.042218  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:03:40.111856  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 69762us
I20260709 13:03:40.113343  7483 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:40.113895  7483 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.114224  7483 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:40.121654  7458 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.122264  7458 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.123781  7483 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:40.124634  7458 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:40.130085  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.136016  1713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.691s	user 0.017s	sys 0.017s
W20260709 13:03:40.137337  1713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.696s	user 0.017s	sys 0.017s
I20260709 13:03:40.138041  1681 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:03:40.139110  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.141201  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1445) took 3144 ms. Trace:
I20260709 13:03:40.147181  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:40.147631  7482 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.160583  7482 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:40.162250  1532 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.651s	user 0.020s	sys 0.037s
I20260709 13:03:40.162983  7482 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:40.162782  1532 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.651s	user 0.020s	sys 0.038s
I20260709 13:03:40.150403  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1148) took 2978 ms. Trace:
W20260709 13:03:40.164072  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 2.65267e+06us
I20260709 13:03:40.166342  1532 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:40.169726  1532 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:40.155829  1681 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:40.171813  1681 leader_election.cc:400] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16.
I20260709 13:03:40.172139  1681 leader_election.cc:403] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:40.156608  1564 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:40.164837  1713 rpcz_store.cc:276] 0709 13:03:37.171874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.172058 (+   184us) service_pool.cc:224] Handling call
0709 13:03:37.172402 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.349116 (+176714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.349208 (+    92us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:37.368071 (+ 18863us) log.cc:844] Serialized 26 byte log entry
0709 13:03:40.137750 (+2769679us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:40.140004 (+  2254us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.150059 (+ 10055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.150164 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.150200 (+    36us) raft_consensus.cc:1685] finished
0709 13:03:40.150214 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.150335 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11293,"prepare.queue_time_us":166,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":43176,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":209,"thread_start_us":2871037,"threads_started":3,"wal-append.queue_time_us":255629,"wal-append.run_cpu_time_us":4922,"wal-append.run_wall_time_us":1838429}
I20260709 13:03:40.177016  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1150) took 2959 ms. Trace:
I20260709 13:03:40.177615  1709 rpcz_store.cc:276] 0709 13:03:37.218158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.218336 (+   178us) service_pool.cc:224] Handling call
0709 13:03:37.218705 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.436393 (+217688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.436434 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:37.436487 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:37.436531 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.176731 (+2740200us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54003386d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:40.176791 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.176947 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500058880}
I20260709 13:03:40.156273  2303 rpcz_store.cc:276] 0709 13:03:36.997188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.997533 (+   345us) service_pool.cc:224] Handling call
0709 13:03:37.001844 (+  4311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.025006 (+ 23162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.025045 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:37.036255 (+ 11210us) log.cc:844] Serialized 26 byte log entry
0709 13:03:37.087828 (+ 51573us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:37.087879 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:37.087966 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.140961 (+3052995us) raft_consensus.cc:1685] finished
0709 13:03:40.140981 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.141141 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111398,"raft.queue_time_us":50065,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":9954,"spinlock_wait_cycles":32384,"thread_start_us":51234,"threads_started":1,"wal-append.queue_time_us":250138,"wal-append.run_cpu_time_us":9042,"wal-append.run_wall_time_us":1339172}
I20260709 13:03:40.187496  1532 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:40.126349  7483 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:40.155977  7479 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:40.155202  7486 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.188285  7482 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:40.198444  7491 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:03:40.152546  7480 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.882s	user 0.006s	sys 0.047s
I20260709 13:03:40.184664  7399 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:40.204777  2300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.646s	user 0.015s	sys 0.051s
W20260709 13:03:40.207836  7480 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.006s	sys 0.047s
I20260709 13:03:40.208235  7399 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:40.208904  2300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.653s	user 0.015s	sys 0.052s
I20260709 13:03:40.209478  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 1319) took 2708 ms. Trace:
I20260709 13:03:40.200062  7479 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.210278  2300 rpcz_store.cc:276] 0709 13:03:37.500960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.501390 (+   430us) service_pool.cc:224] Handling call
0709 13:03:37.501669 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.508113 (+  6444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.508144 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:37.518264 (+ 10120us) log.cc:844] Serialized 26 byte log entry
0709 13:03:40.209234 (+2690970us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:40.209263 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.209307 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.209335 (+    28us) raft_consensus.cc:1685] finished
0709 13:03:40.209342 (+     7us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.209439 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4628,"raft.queue_time_us":4964,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":11070,"thread_start_us":2653730,"threads_started":1,"wal-append.queue_time_us":2667970}
I20260709 13:03:40.201354  7483 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:03:40.207406  7486 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:40.208559  7491 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:03:40.219167  7495 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:03:40.219762  2299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1442) took 3228 ms (client timeout 2579 ms). Trace:
I20260709 13:03:40.219913  7495 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:40.220233  2299 rpcz_store.cc:269] 0709 13:03:36.991629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.991753 (+   124us) service_pool.cc:224] Handling call
0709 13:03:40.219525 (+3227772us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b5400600ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:40.219665 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":932276480}
I20260709 13:03:40.219815  7497 consensus_queue.cc:1059] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:03:40.221627  7497 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:40.222257  7495 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:40.207296  7482 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:40.223941  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3050) took 1009 ms. Trace:
I20260709 13:03:40.208726  7399 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.224344  1569 rpcz_store.cc:276] 0709 13:03:39.214010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.216311 (+1002301us) service_pool.cc:224] Handling call
0709 13:03:40.223402 (+  7091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:40.223590 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:40.223618 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:40.223664 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:40.223727 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.223782 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.223884 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.231086  7482 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:40.215582  7479 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:40.226591  7399 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:40.249185  1635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 2949) took 3429 ms (client timeout 1711 ms). Trace:
W20260709 13:03:40.249644  1635 rpcz_store.cc:269] 0709 13:03:36.819267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.820201 (+   934us) service_pool.cc:224] Handling call
0709 13:03:40.248874 (+3428673us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b5400247554. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:40.249078 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1161486720}
W20260709 13:03:40.254685  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 47779.5us
I20260709 13:03:40.256788  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:40.259981  7495 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:40.260649  7495 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.261420  6990 raft_consensus.cc:3055] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:40.261854  6990 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:40.263048  7495 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:40.262820  6990 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:40.263902  6990 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:40.271325  7479 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:40.276094  1640 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.734s	user 0.003s	sys 0.032s
I20260709 13:03:40.278869  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:40.280771  1569 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
W20260709 13:03:40.279275  1640 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.737s	user 0.004s	sys 0.032s
I20260709 13:03:40.278621  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:40.282572  1565 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:40.282907  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2554) took 2784 ms. Trace:
W20260709 13:03:40.278584  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 69587.7us
I20260709 13:03:40.273814  7479 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.291159  7503 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.276247  7483 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16.
I20260709 13:03:40.288486  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:40.285285  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.291705  1717 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:40.292294  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.283456  1640 rpcz_store.cc:276] 0709 13:03:37.498328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.498449 (+   121us) service_pool.cc:224] Handling call
0709 13:03:37.498737 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:37.512948 (+ 14211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.512996 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:37.513584 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 13:03:40.282451 (+2768867us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:40.282520 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.282612 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.282666 (+    54us) raft_consensus.cc:1685] finished
0709 13:03:40.282683 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.282836 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7797,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":6445,"thread_start_us":2740175,"threads_started":1,"wal-append.queue_time_us":105404,"wal-append.run_cpu_time_us":2930,"wal-append.run_wall_time_us":1076192}
I20260709 13:03:40.337340  1565 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:40.290459  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:40.341511  1681 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:40.330381  7498 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.327811  7458 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:40.346655  7498 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.346993  7458 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.347890  1718 raft_consensus.cc:2410] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:40.338055  7502 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:40.354014  6990 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:40.353996  7482 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.355297  6990 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.355717  6990 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:40.357157  7482 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.358119  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1284) took 3366 ms. Trace:
I20260709 13:03:40.359266  7498 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.359830  7502 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.365545  7458 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.369747  7393 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:40.358584  1639 rpcz_store.cc:276] 0709 13:03:36.991229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.154713 (+163484us) service_pool.cc:224] Handling call
0709 13:03:37.188536 (+ 33823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.471990 (+283454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:37.472042 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:37.472105 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:37.472157 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.357862 (+2885705us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54001e8f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:40.357921 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.358058 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":651324800}
I20260709 13:03:40.370234  7393 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:40.371914  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.372862  7478 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:40.372555  7393 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.373947  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.375844  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.376459  1714 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:40.377368  1711 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:40.373764  7393 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.382956  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.351411  7503 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.386658  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:40.387336  1567 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:40.387857  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 3052) took 1173 ms. Trace:
I20260709 13:03:40.388159  1567 rpcz_store.cc:276] 0709 13:03:39.214632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.223494 (+  8862us) service_pool.cc:224] Handling call
0709 13:03:40.386474 (+1162980us) spinlock_profiling.cc:243] Waited 3.51 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:03:40.387776 (+  1302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8062336}
I20260709 13:03:40.389820  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.394485  7503 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.395635  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.401932  1714 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:40.403005  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.407335  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:40.408485  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 3061) took 1191 ms. Trace:
I20260709 13:03:40.408972  1560 rpcz_store.cc:276] 0709 13:03:39.216914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.220255 (+  3341us) service_pool.cc:224] Handling call
0709 13:03:40.407160 (+1186905us) spinlock_profiling.cc:243] Waited 7.16 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:03:40.408403 (+  1243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16474880}
W20260709 13:03:40.416325  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 2.88695e+06us
W20260709 13:03:40.418002  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.418387  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.418715  1607 connection.cc:256] RPC call timeout handler was delayed by -1.0114320099999645s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 13:03:40.420972  1607 connection.cc:256] RPC call timeout handler was delayed by -1.2788789509999674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
W20260709 13:03:40.423638  7451 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 2.817s	user 0.003s	sys 0.019s
I20260709 13:03:40.427142  1712 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
W20260709 13:03:40.437767  1607 connection.cc:256] RPC call timeout handler was delayed by -1.2324945039999875s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19630000000000003s for a response
W20260709 13:03:40.438798  1607 connection.cc:256] RPC call timeout handler was delayed by -1.2146843099999387s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21640000000000004s for a response
W20260709 13:03:40.439406  1607 connection.cc:256] RPC call timeout handler was delayed by -1.1214401479999765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2017s for a response
W20260709 13:03:40.439757  1607 connection.cc:256] RPC call timeout handler was delayed by -1.0196258039999293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2163s for a response
I20260709 13:03:40.446499  6990 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.426738  7502 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.448972  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.449945  1713 raft_consensus.cc:2410] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:40.451251  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:40.451936  1713 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:40.454700  7323 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.457594  1709 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:03:40.459388  1714 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:40.459625  1713 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:40.460877  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.463080  7286 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:40.463465  1713 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:03:40.463649  7286 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:40.464167  7286 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:40.464996  6990 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.466051  7346 consensus_queue.cc:1059] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:40.466589  7354 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:40.467152  7354 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:40.467927  7346 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:40.468511  7346 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:40.469194  7354 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:40.469718  7354 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:40.470135  7354 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:40.472308  7354 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:40.472946  7354 raft_consensus.cc:3055] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:40.473248  7354 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:40.470613  7346 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:40.474820  7354 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.473667  7286 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.481724  7286 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.484077  7178 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:40.496951  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.497810  1711 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:40.502151  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.502846  2303 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:40.503280  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.503507  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.504083  2300 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:40.503899  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.505026  2305 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:40.505407  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.506086  2297 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:40.509658  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.510103  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.513900  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.514446  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.515209  2297 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:40.510565  1532 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:40.516255  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:40.516883  2300 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:40.518015  7458 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:40.518465  7458 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.519168  7385 consensus_queue.cc:1059] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:40.521555  7412 raft_consensus.cc:3055] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:40.521981  7412 raft_consensus.cc:740] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:40.522634  1532 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:40.523775  7458 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:40.524358  7361 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:40.525012  7361 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.527248  7396 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:40.527367  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.527879  7396 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:40.528784  7396 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:40.529606  1610 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.228s	user 0.016s	sys 0.039s
W20260709 13:03:40.529944  1610 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.229s	user 0.016s	sys 0.039s
I20260709 13:03:40.530596  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.518268  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:40.514595  2303 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:40.533388  1563 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:40.521373  7385 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:40.534034  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 994) took 1324 ms. Trace:
I20260709 13:03:40.534608  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.535055  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.513958  2304 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:40.535485  2301 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:40.534451  1563 rpcz_store.cc:276] 0709 13:03:39.209945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.517952 (+1308007us) service_pool.cc:224] Handling call
0709 13:03:40.533939 (+ 15987us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:40.543865  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1144) took 4043 ms. Trace:
I20260709 13:03:40.545878  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:40.544528  1715 rpcz_store.cc:276] 0709 13:03:36.500329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.500523 (+   194us) service_pool.cc:224] Handling call
0709 13:03:36.500962 (+   439us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:36.794782 (+293820us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:36.847161 (+ 52379us) log.cc:844] Serialized 14 byte log entry
0709 13:03:37.508171 (+661010us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b1800341740. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a 00007fd7a1caba2a
0709 13:03:40.247055 (+2738884us) log.cc:844] Serialized 14 byte log entry
0709 13:03:40.443363 (+196308us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:40.455801 (+ 12438us) log.cc:844] Serialized 52 byte log entry
0709 13:03:40.455955 (+   154us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:40.455987 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.456065 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:40.537022 (+ 80957us) spinlock_profiling.cc:243] Waited 37.2 ms on lock 0x7b1800341740. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:03:40.543552 (+  6530us) raft_consensus.cc:1685] finished
0709 13:03:40.543585 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:40.543783 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122666,"prepare.queue_time_us":40105,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":94,"raft.queue_time_us":31790,"raft.run_cpu_time_us":421,"raft.run_wall_time_us":21947,"spinlock_wait_cycles":112533632,"thread_start_us":509559,"threads_started":4,"wal-append.queue_time_us":906444,"wal-append.run_cpu_time_us":1725,"wal-append.run_wall_time_us":1011769}
I20260709 13:03:40.546751  1710 raft_consensus.cc:2368] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
I20260709 13:03:40.547277  7412 consensus_queue.cc:260] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.548938  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:40.549671  1718 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:40.545454  2300 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:40.554699  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:40.556653  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:40.563997  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.594375  7393 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:40.595700  7500 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:40.596290  7500 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.598423  7500 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:40.598886  7495 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:40.599428  7495 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:40.628137  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.642761  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:40.643553  2300 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:40.548956  7412 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:40.642829  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 60105.8us
I20260709 13:03:40.645594  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.648036  2301 raft_consensus.cc:2410] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:03:40.652192  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:40.654260  1718 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:40.656684  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.646085  7406 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:40.658964  7406 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.661204  7406 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.653313  7478 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:40.662039  7478 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:40.664304  7478 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:40.667059  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.657562  2303 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:40.683197  2300 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:40.686269  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:40.686966  2269 connection.cc:256] RPC call timeout handler was delayed by -1.5861316339999689s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
W20260709 13:03:40.687577  2269 connection.cc:256] RPC call timeout handler was delayed by -1.1104842650000251s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2217s for a response
I20260709 13:03:40.689978  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:40.704012  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:40.708422  7497 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:40.709010  7497 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.709451  7497 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:40.704262  7510 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:40.711323  7387 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:03:40.724807  7510 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.684877  7502 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.706605  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.707885  1683 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:40.644719  7393 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:40.705245  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 74395.6us
I20260709 13:03:40.728682  1683 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:40.706324  7508 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:40.707988  6990 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:40.743223  6990 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.739553  7502 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.743837  6990 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:40.746140  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.711498  7495 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:40.742162  7483 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:40.729105  7503 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:40.748042  7508 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.748888  7503 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:40.748059  7483 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.751636  7483 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.751837  7508 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.743665  7292 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:40.738780  7479 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:40.747464  7502 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.740798  1683 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:40.770071  7510 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.772388  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:40.773236  7503 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:40.778272  1717 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:40.770148  1712 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2531) took 3285 ms (client timeout 2017 ms). Trace:
W20260709 13:03:40.779704  1712 rpcz_store.cc:269] 0709 13:03:37.484866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.560445 (+ 75579us) service_pool.cc:224] Handling call
0709 13:03:40.769915 (+3209470us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b54003dc2d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:40.770053 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99818752}
W20260709 13:03:40.780149  7511 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:03:40.787595  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:40.794036  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 72656.1us
I20260709 13:03:40.791628  7361 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:40.765147  7500 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.803138  1710 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:40.802878  7500 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.805173  7500 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:40.760180  7292 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.811775  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.759181  7479 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:40.817641  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.760208  7495 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:40.821393  7393 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:40.823129  7495 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.824285  2302 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
W20260709 13:03:40.820241  7394 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 4.141s	user 0.014s	sys 0.023s
I20260709 13:03:40.808295  7492 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:03:40.786952  7508 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:40.824724  6990 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:40.852846  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 67683.8us
I20260709 13:03:40.825362  7495 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:40.826042  2269 leader_election.cc:336] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 2.696s (SENT)
I20260709 13:03:40.826521  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.760803  7458 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.863166  2303 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:40.819526  7479 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:40.827664  7394 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.154s	user 0.014s	sys 0.023s
I20260709 13:03:40.900677  7394 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:40.901810  7394 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.902493  7403 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:40.903000  7403 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:40.903820  7403 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:40.864952  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.866442  7361 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:40.866966  7292 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.868180  7458 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.869882  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.872856  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:40.827273  7508 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:03:40.876045  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.917798  1683 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:40.876179  7497 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.878669  7405 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:40.883168  2268 connection.cc:256] RPC call timeout handler was delayed by -1.4997720970000046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
I20260709 13:03:40.923624  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:40.884492  7207 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:40.883226  7495 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.828498  7483 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.926517  7495 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.927213  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:40.928390  7483 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.889273  6990 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:40.928836  7495 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:40.929256  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:40.929956  1568 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:40.924232  7508 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:40.933732  7522 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:40.899147  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1278) took 4011 ms. Trace:
I20260709 13:03:40.907428  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:40.909061  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:40.937326  1643 rpcz_store.cc:276] 0709 13:03:36.887954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.888107 (+   153us) service_pool.cc:224] Handling call
0709 13:03:36.903867 (+ 15760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:36.924269 (+ 20402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:40.898840 (+3974571us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b54003efb54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:40.899078 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25918208}
I20260709 13:03:40.938475  1565 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:40.937602  1683 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:40.909487  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:40.910388  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:40.905014  7403 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:40.915212  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:40.919515  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:40.948473  2303 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:40.893728  7479 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:40.948984  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:40.906976  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:03:40.863198  1534 leader_election.cc:336] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.659s (SENT)
W20260709 13:03:40.896420  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 1139) took 5298 ms (client timeout 1919 ms). Trace:
I20260709 13:03:40.955751  1560 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:40.955909  1716 rpcz_store.cc:269] 0709 13:03:35.598340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:35.598555 (+   215us) service_pool.cc:224] Handling call
0709 13:03:40.896160 (+5297605us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54000b3bd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:40.896286 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4207675136}
I20260709 13:03:40.956235  1683 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:40.947356  1567 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:40.958726  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:40.959837  1713 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:40.959942  1712 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:40.958848  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:40.955863  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 92720.4us
I20260709 13:03:40.947978  7361 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:03:40.962826  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 55949.2us
I20260709 13:03:40.964658  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:40.967052  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:40.967936  1560 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:40.973098  1565 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:40.955452  6990 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:40.976972  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:40.978446  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.985381  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:40.986233  1567 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:40.981889  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:40.982456  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.988914  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:40.989665  2298 raft_consensus.cc:2368] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15.
I20260709 13:03:40.985428  2302 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.990197  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:40.991555  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:40.992619  1565 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:40.980901  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.993791  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:40.994437  1560 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:40.989264  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.963979  7292 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:40.962188  7510 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:41.004537  1569 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:40.950157  7483 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:03:41.004724  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:41.006286  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:41.007138  2296 raft_consensus.cc:2393] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 15: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:41.008194  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:41.014424  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:40.988991  7406 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:41.017287  7406 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:41.017710  7406 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:40.965973  7502 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:03:41.021176  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.012753  2298 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:41.010130  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:41.029769  7519 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:03:40.961591  7106 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:03:41.009241  7510 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.038872  7510 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:40.989262  7500 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:41.040285  7500 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:41.040686  7500 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:40.984980  6990 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:41.041214  6990 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:40.988948  7361 raft_consensus.cc:697] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:41.020774  7292 raft_consensus.cc:697] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:41.043028  7292 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.050760  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:41.052011  2299 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:41.053925  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:41.054257  2302 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:03:41.055531  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.057785  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:41.059113  2304 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:41.057811  7502 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:41.060894  1565 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:41.061051  7502 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:41.061929  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.062548  1565 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:03:41.059106  1568 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:41.063794  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.008848  7497 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:41.068305  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:41.065924  7528 log.cc:927] Time spent T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:03:41.071564  1567 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:41.072463  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.070092  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:41.073510  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.073369  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:41.073546  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.071101  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.076157  1565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:03:41.077646  1567 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:41.081158  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:41.081077  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:41.082139  2302 raft_consensus.cc:2410] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:03:41.083314  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:41.084117  1568 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:41.086587  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:41.089174  1681 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:41.090045  7497 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:41.090571  7497 raft_consensus.cc:697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:41.091506  7497 consensus_queue.cc:237] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.043045  7361 consensus_queue.cc:237] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.095988  7510 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.102655  7406 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.105399  7510 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:41.080379  7479 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:41.105436  1683 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:41.108805  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:41.131264  2302 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:41.132648  7194 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:41.136040  7479 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:41.136618  7479 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:41.136703  1714 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:41.137523  7479 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.140898  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:41.141840  2300 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:41.113940  2303 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:41.109313  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:41.126186  1709 raft_consensus.cc:1240] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 10. Current term is 11. Ops: []
W20260709 13:03:41.130195  2326 debug-util.cc:398] Leaking SignalData structure 0x7b080030dc20 after lost signal to thread 434
W20260709 13:03:41.148936  2326 debug-util.cc:398] Leaking SignalData structure 0x7b08002d0760 after lost signal to thread 1501
I20260709 13:03:41.149570  2302 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
W20260709 13:03:41.150645  2326 debug-util.cc:398] Leaking SignalData structure 0x7b08002ecea0 after lost signal to thread 1593
W20260709 13:03:41.156656  2326 debug-util.cc:398] Leaking SignalData structure 0x7b08002ec640 after lost signal to thread 1668
W20260709 13:03:41.157781  2326 debug-util.cc:398] Leaking SignalData structure 0x7b08002ec620 after lost signal to thread 1669
W20260709 13:03:41.159725  2326 debug-util.cc:398] Leaking SignalData structure 0x7b08003fac40 after lost signal to thread 1742
W20260709 13:03:41.161417  2326 debug-util.cc:398] Leaking SignalData structure 0x7b0800233e80 after lost signal to thread 2329
W20260709 13:03:41.174625  1534 leader_election.cc:336] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 2.121s (SENT)
I20260709 13:03:41.175292  1534 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:41.177338  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:41.127501  7406 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:41.194893  2300 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:41.190418  7503 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:41.210696  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.211419  1561 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:41.212056  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:41.171906  6990 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.221050  6990 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:41.191490  7397 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:03:41.243201  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:41.244011  2304 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:41.244451  2300 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:41.245826  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:41.246028  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:41.246755  2298 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:41.247051  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:41.249192  2268 leader_election.cc:400] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
I20260709 13:03:41.249521  2268 leader_election.cc:403] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:03:41.249567  2326 debug-util.cc:398] Leaking SignalData structure 0x7b080030be20 after lost signal to thread 7401
W20260709 13:03:41.249972  2326 debug-util.cc:398] Leaking SignalData structure 0x7b0800318940 after lost signal to thread 7404
W20260709 13:03:41.250309  2326 debug-util.cc:398] Leaking SignalData structure 0x7b08002291a0 after lost signal to thread 7417
W20260709 13:03:41.250764  2326 debug-util.cc:398] Leaking SignalData structure 0x7b08003c67e0 after lost signal to thread 7455
I20260709 13:03:41.254755  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:41.260414  1560 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:41.260890  1565 raft_consensus.cc:606] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:41.261251  1565 raft_consensus.cc:614] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:41.264448  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.265098  7106 raft_consensus.cc:2764] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 13:03:41.268072  7526 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.260s	user 0.005s	sys 0.000s
I20260709 13:03:41.268555  7106 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
I20260709 13:03:41.272871  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.269524  1681 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:41.272410  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:41.276836  1564 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:41.290532  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:41.291239  1717 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:41.299173  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:41.307597  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:41.278019  7109 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:41.311697  7109 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:41.311900  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.288945  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:41.312649  7109 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.313153  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.313679  1568 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:41.315814  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.316602  1561 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:41.318077  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:41.226884  7500 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.323899  1712 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:41.324456  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:41.325263  1715 raft_consensus.cc:2368] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
I20260709 13:03:41.325693  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:41.326130  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.327824  7393 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:41.270684  7385 consensus_queue.cc:1059] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
W20260709 13:03:41.328830  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 904223us
I20260709 13:03:41.329173  7385 raft_consensus.cc:3055] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:41.329622  7385 raft_consensus.cc:740] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:41.333117  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.333751  1635 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
W20260709 13:03:41.334388  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 892377us
I20260709 13:03:41.334532  6946 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.335688  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.324638  7500 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:41.335589  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:41.336150  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:41.336354  1640 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:41.371477  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2548) took 3879 ms. Trace:
I20260709 13:03:41.375494  2298 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:41.377286  2305 raft_consensus.cc:2368] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17.
I20260709 13:03:41.379308  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.380228  1640 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:41.382269  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.388129  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.390008  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.383492  1638 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:41.389001  1644 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:41.399261  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:41.371918  1636 rpcz_store.cc:276] 0709 13:03:37.492519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.493033 (+   514us) service_pool.cc:224] Handling call
0709 13:03:37.507496 (+ 14463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:37.508076 (+   580us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:37.529926 (+ 21850us) log.cc:844] Serialized 14 byte log entry
0709 13:03:41.371164 (+3841238us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:41.371214 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:41.371254 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:41.371317 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:41.371421 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":317801,"prepare.queue_time_us":14471,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":490,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":130715,"threads_started":1,"wal-append.queue_time_us":133933}
I20260709 13:03:41.400440  1643 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:41.400637  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:41.315822  7106 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:41.404148  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.406891  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.408742  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.410035  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.415449  2299 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:41.404258  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:41.416390  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.417125  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.422178  1639 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:41.419066  1644 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:41.423964  1636 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:41.420890  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 133992us
I20260709 13:03:41.419958  1635 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:41.426098  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:41.339308  7401 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.351267  7139 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:41.339390  7404 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:41.278549  7508 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.418339  1641 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:41.335194  6946 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.399152  7417 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:41.350524  7396 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:03:41.442976  1716 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:41.424897  7412 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:41.438122  1609 leader_election.cc:400] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17.
I20260709 13:03:41.448658  1643 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:41.429849  7178 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.423516  7405 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:41.404989  7286 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:03:41.459571  7405 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:41.462127  7405 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:41.462442  7346 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.451864  1609 leader_election.cc:403] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:03:41.454262  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.486661  1639 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:41.456696  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.416409  7455 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:41.490141  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 58685.2us
I20260709 13:03:41.464033  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:41.470393  7385 consensus_queue.cc:260] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.472236  7354 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:41.439232  7106 raft_consensus.cc:697] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:41.435590  7139 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.511298  7385 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:41.481411  6946 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:41.483390  1562 raft_consensus.cc:2410] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:03:41.488320  1643 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
W20260709 13:03:41.489380  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.443323  7210 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:41.439936  7404 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.508854  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:41.509935  7455 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.510483  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:41.523766  1635 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:41.434028  7401 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:41.530269  1639 raft_consensus.cc:2376] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16. Re-sending same reply.
W20260709 13:03:41.605121  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 279514us
I20260709 13:03:41.457155  7493 consensus_queue.cc:1059] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:03:41.519629  7405 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:41.609731  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:41.443979  7417 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.518460  7486 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.483845  7178 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.635224  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:41.616861  7530 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:03:41.638090  1642 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
W20260709 13:03:41.645057  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 219835us
I20260709 13:03:41.638033  1637 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:41.641151  1641 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:41.500073  7346 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.616065  6946 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:41.646245  7486 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.641978  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:41.525327  7069 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.115s
I20260709 13:03:41.660669  1533 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:41.639140  7495 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:41.669255  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.667959  2268 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:41.672878  1641 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:41.674526  7522 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:41.677574  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.685802  1638 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
W20260709 13:03:41.686203  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 260172us
I20260709 13:03:41.686942  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.672598  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:41.690016  1568 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:41.690060  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:41.690819  1565 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:41.691673  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 356607us
I20260709 13:03:41.687019  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:03:41.689309  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 332084us
W20260709 13:03:41.683844  1562 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2579) took 2466 ms (client timeout 1758 ms). Trace:
I20260709 13:03:41.692998  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:41.691435  1533 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:41.688180  1638 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:41.700383  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.700824  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:03:41.698210  1562 rpcz_store.cc:269] 0709 13:03:39.217449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:39.217851 (+   402us) service_pool.cc:224] Handling call
0709 13:03:41.673570 (+2455719us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b5400297cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:41.683748 (+ 10178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5187293440}
I20260709 13:03:41.701165  1638 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
W20260709 13:03:41.697393  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.704768  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:41.671501  7499 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:41.687830  2268 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:41.709513  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.708209  1565 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:41.711826  1607 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:41.713011  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.718708  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:41.683041  7069 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:41.744763  1638 raft_consensus.cc:2393] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 18: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:41.745160  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.745049  7069 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.745657  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.746294  1638 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:41.748021  7069 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:41.752025  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.752861  1644 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:41.753852  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.754489  1644 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:41.685138  7522 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:41.757416  7522 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:41.740293  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.760033  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.760687  1637 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:03:41.761631  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.755156  1636 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:41.732589  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:41.763662  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.764420  1635 raft_consensus.cc:2368] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16.
I20260709 13:03:41.765218  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.765810  1635 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:41.673290  7520 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:03:41.762285  1637 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:41.766614  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:41.782446  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 63793.5us
I20260709 13:03:41.668673  6946 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.708618  7396 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:41.788128  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:41.806056  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 94346.6us
I20260709 13:03:41.806859  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:41.789819  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 102903us
I20260709 13:03:41.803413  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.809063  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:41.809837  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 118453us
I20260709 13:03:41.819553  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.822377  1533 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:41.822417  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:41.823885  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:41.824229  1534 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:41.821043  1644 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:41.826095  1683 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:41.827363  1635 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:41.731233  7499 raft_consensus.cc:697] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:41.669420  7106 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.828794  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:41.803326  7513 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:41.803150  7394 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:03:41.825057  7069 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:41.837270  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:41.839893  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.834012  7540 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:41.848147  7403 consensus_meta.cc:287] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.898s	user 0.009s	sys 0.004s
I20260709 13:03:41.808183  7544 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:41.831815  7479 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:41.829996  7491 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:41.829595  7495 raft_consensus.cc:3055] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:41.860857  1644 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:03:41.858618  7550 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:41.864377  7455 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:41.866129  7499 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:41.843880  7513 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.834249  7323 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:41.831028  7545 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:41.868089  7522 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:41.876557  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 54234.9us
I20260709 13:03:41.909754  1533 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:41.877401  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:41.879191  2305 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:03:41.881727  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 55746.8us
W20260709 13:03:41.884079  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 77301.6us
I20260709 13:03:41.888801  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.912920  1644 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:41.914095  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.915014  1644 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 14: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:03:41.891055  7537 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:03:41.892182  1640 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:41.870374  7559 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:41.900511  7346 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:41.901160  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 77012.7us
I20260709 13:03:41.876559  7491 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:41.876222  7495 raft_consensus.cc:740] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:41.936000  1534 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:41.901873  7551 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:41.903195  7210 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:41.904232  6946 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:41.904717  7417 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:41.905097  7178 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:41.871701  7557 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:41.916453  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:41.953001  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:41.954130  1607 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:41.911715  1683 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:41.954824  7069 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:41.885118  7546 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:41.955229  7069 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:41.955946  7069 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:41.956063  6946 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:41.876452  7550 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.956808  6946 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:41.956910  7550 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:41.957254  6946 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:41.957758  6946 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:41.961342  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.964773  7069 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:41.967486  7069 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:41.971705  7522 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:41.972635  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.973886  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:41.974468  1644 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:41.976253  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.976542  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:41.977322  1566 raft_consensus.cc:2410] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
W20260709 13:03:41.979034  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 67415.8us
I20260709 13:03:41.980020  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:41.981387  7508 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:41.933934  1563 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:41.917342  7545 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:41.887244  7479 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:42.000640  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.001336  1638 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:42.002296  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:41.908250  7513 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:41.948632  7403 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:41.955426  7546 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:41.858222  7520 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:03:42.010573  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:41.939643  7491 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:41.941622  7551 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:41.968231  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.921922  7559 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:41.934711  7323 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:41.970450  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.983292  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.983805  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:41.984503  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.955135  7548 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:41.985203  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:42.023137  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:42.023861  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:42.024176  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:41.986992  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:42.024199  1644 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:03:42.025144  1569 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:42.025336  1565 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 19: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:41.988198  7194 raft_consensus.cc:3055] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:42.025998  7194 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:42.026793  7194 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:42.027796  7194 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:42.028059  7502 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.028600  7405 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17.
I20260709 13:03:42.028570  7502 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:42.002561  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:41.991178  2304 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.001s	sys 0.009s
I20260709 13:03:41.944744  1534 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:42.030889  2304 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.002s	sys 0.009s
W20260709 13:03:42.031373  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 86738.3us
I20260709 13:03:42.011655  1638 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:42.016455  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:42.018759  1562 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:41.982085  7508 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:42.035431  1716 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 11. Current term is 12. Ops: []
W20260709 13:03:41.939648  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 112090us
I20260709 13:03:42.008801  7397 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:03:42.019423  7292 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:03:42.022120  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.037387  2268 leader_election.cc:400] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
I20260709 13:03:42.037833  2268 leader_election.cc:403] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:42.037874  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.021970  7323 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.028846  7396 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:42.003123  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.020112  7559 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:42.034420  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:42.006704  7557 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:42.007319  7545 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:42.039644  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.040525  1639 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 16: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:42.095928  1637 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:42.015921  7546 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:42.020023  7520 consensus_queue.cc:1243] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:42.146171  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:42.101289  1715 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:42.018896  7551 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:42.149860  7401 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:42.104614  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.151582  1718 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:03:42.151160  6946 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.105485  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.153258  1644 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 16: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:42.153476  6946 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:42.039722  7405 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:42.113704  7508 raft_consensus.cc:697] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:42.154980  7508 consensus_queue.cc:237] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.156229  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.033780  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:42.158427  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 124730us
I20260709 13:03:42.161528  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:42.125749  7540 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:42.126602  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.127092  7396 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:42.167779  7479 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.128376  7397 consensus_queue.cc:1059] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:42.030933  7547 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.097s
I20260709 13:03:42.172462  7547 raft_consensus.cc:3055] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:42.035454  7548 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:42.163391  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:42.164132  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:42.172538  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:42.135159  7502 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:42.154618  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2591) took 1217 ms. Trace:
I20260709 13:03:42.182320  1563 rpcz_store.cc:276] 0709 13:03:40.937279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:40.948704 (+ 11425us) service_pool.cc:224] Handling call
0709 13:03:42.154459 (+1205755us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b54001ce154. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:42.154561 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2118655872}
I20260709 13:03:42.179210  1710 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:42.181336  1640 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:42.124316  2269 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:42.175275  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:42.174117  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:42.186713  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.122697  7106 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
W20260709 13:03:42.191058  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 66852.4us
I20260709 13:03:42.192785  2269 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:42.178676  7565 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:42.194049  2269 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:42.194350  7540 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:42.194836  7540 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:42.195201  7393 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:42.195273  7540 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:42.195582  7500 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:42.199746  7559 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.178838  7401 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:42.203979  7540 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.205821  7139 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:42.206326  7393 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:42.208204  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:42.208827  1635 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:42.213508  7574 log.cc:927] Time spent T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.088s	user 0.002s	sys 0.000s
I20260709 13:03:42.224488  7562 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.225373  7540 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:42.219460  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.225080  7562 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.227820  1718 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:42.228159  7562 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:42.229017  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.215421  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.238224  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:42.239557  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:42.240221  1637 raft_consensus.cc:2393] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 18: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:42.178006  7547 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:42.240906  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:42.243204  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.253305  1637 raft_consensus.cc:2376] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13. Re-sending same reply.
I20260709 13:03:42.244134  1711 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:42.254176  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.244769  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.242542  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.245424  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.257443  1710 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:03:42.257999  7540 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:42.258500  7518 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.246706  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.258622  7540 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.259061  7518 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.259591  7498 raft_consensus.cc:3055] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:42.260008  7498 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:42.260896  7498 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.261332  7518 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:42.257241  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.262172  7498 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:42.262392  7540 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:42.262993  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.263625  1637 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:42.264498  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.246364  7550 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.267725  7550 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:42.265535  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.271230  1637 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:42.272179  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.272759  1637 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
I20260709 13:03:42.273504  1637 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:44380
I20260709 13:03:42.273998  1637 raft_consensus.cc:606] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:42.274284  1637 raft_consensus.cc:614] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:42.275594  1533 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:42.276374  7493 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:42.276744  7493 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:42.276825  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:42.277192  7493 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:42.277952  1533 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:42.278680  7562 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:42.279085  7562 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:42.279462  7562 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:42.283128  7458 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:42.283587  7458 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:42.283963  7458 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:42.288549  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 251462us
I20260709 13:03:42.290940  1569 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:03:42.291913  1640 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:03:42.292904  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.293466  1640 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:42.288707  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.295356  1533 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:42.295521  1637 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:42.296561  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.296770  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.298946  1640 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:42.247009  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.304809  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:42.238245  1639 raft_consensus.cc:2376] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17. Re-sending same reply.
I20260709 13:03:42.305528  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:42.309293  1565 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:42.325390  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:42.332501  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:42.333246  2300 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:42.337741  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:42.334848  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:42.338804  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:42.341663  2269 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:42.346884  1635 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:42.348924  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:42.353983  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 63794.1us
I20260709 13:03:42.354382  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:42.355690  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:42.354507  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.357173  1682 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:42.357811  1637 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:42.358052  1568 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 19: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:42.359375  7396 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.361472  1568 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: [10.10-12.11]
I20260709 13:03:42.358314  7394 consensus_queue.cc:1059] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:42.363324  7540 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:42.363777  7540 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:42.363489  7491 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.364750  7540 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.366109  7540 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:42.357173  7493 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.369436  7513 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.370508  7458 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:42.373304  7562 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.374488  7458 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:42.371534  7551 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:42.376287  1682 leader_election.cc:400] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16.
I20260709 13:03:42.356258  1638 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:42.376650  7406 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:03:42.377226  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 54351.9us
I20260709 13:03:42.398382  1635 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:03:42.376019  7545 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:42.379345  1560 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:42.379477  1682 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:42.379786  1569 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:42.400611  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:42.401150  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 59603.6us
I20260709 13:03:42.380021  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.380199  1638 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:42.380666  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.381372  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:42.402659  1709 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 10.
I20260709 13:03:42.403234  1715 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:42.364120  2304 raft_consensus.cc:2393] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:03:42.387879  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 92614.6us
I20260709 13:03:42.372212  7493 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:42.391360  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:42.409390  1568 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:42.411350  1682 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:42.411758  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:42.417466  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:42.428750  1718 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:42.429669  1682 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:42.429991  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:42.432075  1564 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:03:42.437216  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:42.439146  2269 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:42.241212  7547 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.410523  7194 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:42.442942  7194 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.443414  7194 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:42.447803  7513 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:42.406837  7412 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.454929  1641 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:42.441210  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:42.407440  7491 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:42.487840  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 313798us
I20260709 13:03:42.488837  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:42.397157  7559 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:42.409319  7361 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:42.412003  7106 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:03:42.464473  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 482632us
I20260709 13:03:42.510267  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:42.510550  2299 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:03:42.398617  7565 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:42.412024  7394 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:42.472404  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 123588us
I20260709 13:03:42.511061  2299 pending_rounds.cc:85] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 13
I20260709 13:03:42.459262  7518 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:42.485450  1564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1256ff04335423286d7c081ec776061"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:57836
W20260709 13:03:42.494554  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 385920us
I20260709 13:03:42.511554  2299 pending_rounds.cc:107] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:03:42.516669  1564 raft_consensus.cc:606] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:42.516852  2299 raft_consensus.cc:2889] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:42.517102  1564 raft_consensus.cc:614] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:42.454108  7168 raft_consensus.cc:3055] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:42.522614  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:42.459806  7497 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:42.530234  2305 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:42.540149  1635 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:42.515237  7394 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:42.274811  7546 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.412024  7502 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 13:03:42.526491  2297 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:03:42.536604  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.538067  2300 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:42.509613  7361 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:42.555950  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:42.542419  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:42.514312  7401 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:42.564144  2299 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
W20260709 13:03:42.573559  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.494019  7550 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:42.651552  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 109282us
I20260709 13:03:42.516139  7551 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:42.533470  7168 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:03:42.654455  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.534469  7106 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:42.674621  1565 raft_consensus.cc:2368] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19.
I20260709 13:03:42.493906  7412 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:42.539464  7497 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:42.542299  7394 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:42.523679  7139 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 13:03:42.531988  7572 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.150s
I20260709 13:03:42.699882  1635 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:42.649642  7396 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:42.654601  7361 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:42.649295  7518 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:42.687204  7562 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:42.717136  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:42.677083  7578 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:42.743307  7578 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:42.743796  7578 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:42.744460  7546 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:42.680518  7406 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:42.750407  7106 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.751384  7406 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:42.715493  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:42.742357  7412 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:42.717810  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:42.757506  1635 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:42.757411  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:42.755219  2301 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:42.759550  7545 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:42.654209  6959 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:42.728051  7479 raft_consensus.cc:3055] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:42.761797  7479 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:42.762809  7479 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.687225  7497 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:42.793408  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:42.718662  7168 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:42.801232  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 372464us
I20260709 13:03:42.802013  1635 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
W20260709 13:03:42.804595  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:42.804960  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:42.819660  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.800328  7508 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.823575  2269 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:42.820482  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 390898us
I20260709 13:03:42.825637  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:42.830175  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:42.831354  1635 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:42.838187  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:42.841418  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:42.850342  2269 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:42.852280  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:42.857195  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:42.860844  1638 raft_consensus.cc:1240] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
W20260709 13:03:42.862685  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:42.862473  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 186058us
W20260709 13:03:42.860729  1641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2586) took 1448 ms (client timeout 1907 ms). Trace:
W20260709 13:03:42.867017  1641 rpcz_store.cc:269] 0709 13:03:41.412634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.717205 (+304571us) service_pool.cc:224] Handling call
0709 13:03:42.860494 (+1143289us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b5400154854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:42.860671 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1485556480}
I20260709 13:03:42.837560  7508 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.835753  7587 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:42.871596  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1294) took 1253 ms. Trace:
I20260709 13:03:42.877923  1635 rpcz_store.cc:276] 0709 13:03:41.638490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.866882 (+1228392us) service_pool.cc:224] Handling call
0709 13:03:42.870860 (+  3978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.871132 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.871171 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.871232 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:42.871287 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.871370 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.871519 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:42.850311  7567 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:42.883958  1635 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:42.871553  7406 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:42.875910  7557 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:42.820761  7518 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.889621  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1295) took 1250 ms. Trace:
I20260709 13:03:42.893913  1641 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:42.895016  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2962) took 1276 ms. Trace:
I20260709 13:03:42.892575  1635 rpcz_store.cc:276] 0709 13:03:41.639723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.881573 (+1241850us) service_pool.cc:224] Handling call
0709 13:03:42.883600 (+  2027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.888704 (+  5104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.889546 (+   842us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:42.895613  1641 rpcz_store.cc:276] 0709 13:03:41.618406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.874093 (+1255687us) service_pool.cc:224] Handling call
0709 13:03:42.893694 (+ 19601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.894578 (+   884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.894909 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:42.900840  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1297) took 1236 ms. Trace:
I20260709 13:03:42.901435  1635 rpcz_store.cc:276] 0709 13:03:41.664100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.899055 (+1234955us) service_pool.cc:224] Handling call
0709 13:03:42.900176 (+  1121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.900408 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.900437 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.900498 (+    61us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:42.900543 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.900616 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.900755 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:42.686254  7566 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.202s
I20260709 13:03:42.880702  7593 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:42.888105  7568 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:42.914410  7593 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:42.914963  7593 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:42.915407  7361 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.916847  1717 raft_consensus.cc:2368] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20.
I20260709 13:03:42.921397  7508 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:42.912668  1635 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:03:42.923974  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1301) took 1209 ms. Trace:
I20260709 13:03:42.741559  7565 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:42.929363  7579 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:03:42.886651  7587 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:42.926148  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1298) took 1256 ms. Trace:
I20260709 13:03:42.935223  7361 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:42.935372  6959 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:42.936277  6959 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:42.936170  7497 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:42.938706  7584 log.cc:927] Time spent T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:03:42.939429  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 82344.7us
I20260709 13:03:42.940299  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1291) took 1329 ms. Trace:
I20260709 13:03:42.940913  1682 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:42.940675  1637 rpcz_store.cc:276] 0709 13:03:41.611153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.360503 (+749350us) service_pool.cc:224] Handling call
0709 13:03:42.360780 (+   277us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:42.361119 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.361148 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:42.361700 (+   552us) log.cc:844] Serialized 52 byte log entry
0709 13:03:42.728677 (+366977us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:42.728722 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.728793 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:42.940056 (+211263us) raft_consensus.cc:1685] finished
0709 13:03:42.940079 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.940229 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6216,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"thread_start_us":366590,"threads_started":1,"wal-append.queue_time_us":437410}
I20260709 13:03:42.943068  1637 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:42.943751  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2981) took 1226 ms. Trace:
I20260709 13:03:42.944051  1637 rpcz_store.cc:276] 0709 13:03:41.716984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.942547 (+1225563us) service_pool.cc:224] Handling call
0709 13:03:42.942861 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.943499 (+   638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.943698 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:42.945389  7597 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:42.945901  7597 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:42.946319  7597 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:42.942448  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:42.946830  1642 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:42.933862  1641 rpcz_store.cc:276] 0709 13:03:41.669725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.899660 (+1229935us) service_pool.cc:224] Handling call
0709 13:03:42.925539 (+ 25879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.925749 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.925798 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.925851 (+    53us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:42.925893 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.925959 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.926082 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:42.924435  1635 rpcz_store.cc:276] 0709 13:03:41.714383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.911975 (+1197592us) service_pool.cc:224] Handling call
0709 13:03:42.912376 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.923604 (+ 11228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.923902 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:42.893453  7557 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:42.891623  7599 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:42.914213  7586 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:03:42.957899  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1306) took 1110 ms. Trace:
I20260709 13:03:42.939718  7497 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:42.956640  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2958) took 1339 ms. Trace:
I20260709 13:03:42.962567  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:42.893108  7540 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:03:42.904956  7210 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:42.967422  1642 rpcz_store.cc:276] 0709 13:03:41.847127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.946311 (+1099184us) service_pool.cc:224] Handling call
0709 13:03:42.946656 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.957628 (+ 10972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.957846 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:42.968972  1638 rpcz_store.cc:276] 0709 13:03:41.616991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.416389 (+799398us) service_pool.cc:224] Handling call
0709 13:03:42.860641 (+444252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.861370 (+   729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.956379 (+ 95009us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b5400154850. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:42.956533 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1020343680}
I20260709 13:03:42.971705  1682 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:42.960101  7503 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:42.973078  7503 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:42.973973  7503 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.935317  7194 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:42.975131  7503 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:42.975919  7406 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:42.976891  7194 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:42.976998  7210 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:42.979702  7210 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:42.983633  2296 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:42.949522  7361 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:42.988193  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1307) took 1124 ms. Trace:
I20260709 13:03:42.964346  7557 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:42.990888  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:42.967391  7508 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:42.991828  7406 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:42.987592  6959 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:42.989163  7486 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:42.999567  1682 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:43.001088  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2954) took 1393 ms. Trace:
I20260709 13:03:42.992408  1641 rpcz_store.cc:276] 0709 13:03:41.847422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.969117 (+1121695us) service_pool.cc:224] Handling call
0709 13:03:42.970847 (+  1730us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.971046 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.971071 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:42.971124 (+    53us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:42.971155 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.971208 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:42.971312 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:43.001655  1640 rpcz_store.cc:276] 0709 13:03:41.607920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.377056 (+769136us) service_pool.cc:224] Handling call
0709 13:03:42.377426 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.377766 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.377798 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:42.378244 (+   446us) log.cc:844] Serialized 26 byte log entry
0709 13:03:42.638671 (+260427us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:42.638740 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.638861 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.000806 (+361945us) raft_consensus.cc:1685] finished
0709 13:03:43.000837 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.001011 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19456,"raft.queue_time_us":9415,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":15362,"thread_start_us":259909,"threads_started":1,"wal-append.queue_time_us":411864}
I20260709 13:03:43.004500  1640 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:43.005276  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2589) took 1074 ms. Trace:
I20260709 13:03:43.005684  1640 rpcz_store.cc:276] 0709 13:03:41.931226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.003860 (+1072634us) service_pool.cc:224] Handling call
0709 13:03:43.004252 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:43.004982 (+   730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.005197 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:42.998379  7598 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:42.994690  7508 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.012920  7508 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:42.994793  7518 consensus_queue.cc:1059] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:03:42.986846  7568 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:03:42.993923  7361 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:43.025293  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:43.026255  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2591) took 1049 ms. Trace:
I20260709 13:03:42.993477  7393 consensus_queue.cc:1059] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:03:43.027781  1641 rpcz_store.cc:276] 0709 13:03:41.976498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.024407 (+1047909us) service_pool.cc:224] Handling call
0709 13:03:43.025041 (+   634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:43.025380 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.025415 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:43.025462 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:43.025494 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.025551 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.026202 (+   651us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:43.023927  6959 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:43.032384  1682 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:43.031347  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2592) took 1054 ms. Trace:
I20260709 13:03:43.035419  1641 rpcz_store.cc:276] 0709 13:03:41.976763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.029826 (+1053063us) service_pool.cc:224] Handling call
0709 13:03:43.030887 (+  1061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:43.031057 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.031081 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:43.031119 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:43.031148 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.031197 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.031298 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:43.027892  7600 raft_consensus.cc:3055] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:43.033483  7510 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:43.033949  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 169725us
I20260709 13:03:43.043062  1682 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:43.045535  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:43.047873  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:43.053591  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2593) took 1069 ms. Trace:
I20260709 13:03:43.054064  1641 rpcz_store.cc:276] 0709 13:03:41.984282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.038106 (+1053824us) service_pool.cc:224] Handling call
0709 13:03:43.053103 (+ 14997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:43.053278 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.053300 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:43.053344 (+    44us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:43.053372 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.053422 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.053532 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:43.054349  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:43.037691  7600 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:43.063284  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:43.038875  7497 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:03:43.059626  7109 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:43.053719  7602 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:43.082701  7602 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:43.083330  7602 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:43.084328  7593 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:43.084815  7404 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:43.085387  7404 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:43.055665  7545 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:43.087437  7601 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:03:43.088021  7479 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:43.088025  1643 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:43.090692  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2994) took 1089 ms. Trace:
I20260709 13:03:43.091015  1643 rpcz_store.cc:276] 0709 13:03:42.001859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.087359 (+1085500us) service_pool.cc:224] Handling call
0709 13:03:43.087709 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:43.090429 (+  2720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.090639 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:43.091574  7600 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.092450  1643 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:43.093051  7602 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:43.095069  7602 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:43.095834  1643 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:43.066180  7361 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:43.065215  7587 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:43.065779  7510 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:03:43.066793  7401 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:43.100200  7508 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.068301  7406 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:03:43.101495  7109 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:43.076437  7394 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.106312  1636 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:03:43.055902  6959 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:43.102560  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:43.098379  7599 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:43.096427  7593 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:43.137941  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:43.114987  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 73088.7us
W20260709 13:03:43.120074  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 56904.9us
I20260709 13:03:43.147965  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:43.126904  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2992) took 1358 ms. Trace:
I20260709 13:03:43.148955  1637 rpcz_store.cc:276] 0709 13:03:41.768497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.944859 (+1176362us) service_pool.cc:224] Handling call
0709 13:03:42.945150 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.945354 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:03:43.034678 (+ 89324us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.126452 (+ 91774us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:43.126520 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:03:43.126578 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.126678 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.126836 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":173814,"raft.queue_time_us":100,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":2346,"thread_start_us":177690,"threads_started":2,"wal-append.queue_time_us":87004}
I20260709 13:03:43.125633  7393 consensus_queue.cc:1059] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:03:43.114391  7500 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:43.152837  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:43.154036  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:43.155720  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:43.162701  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:43.163291  2302 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:43.136165  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 80903.5us
I20260709 13:03:43.158879  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:43.160565  1637 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:43.162701  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:43.167419  2300 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:43.163386  2297 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:43.168339  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:43.132289  7109 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:43.173663  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:43.170579  2303 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:43.131527  7520 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.172362  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:43.175823  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:43.175910  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:43.133066  6959 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.176919  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:43.178243  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:43.133210  7106 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:43.137118  7503 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:43.160821  7404 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:43.179160  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:43.181772  2297 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:43.179934  2302 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:03:43.166427  7458 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:43.126217  7602 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:03:43.177351  7590 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.183283  7397 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:43.206552  7508 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:43.209424  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2590) took 1278 ms. Trace:
I20260709 13:03:43.169107  7557 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.219065  7557 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:43.183998  6959 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:43.213692  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 54929.2us
I20260709 13:03:43.220431  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:43.215685  1640 rpcz_store.cc:276] 0709 13:03:41.931449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.006628 (+1075179us) service_pool.cc:224] Handling call
0709 13:03:43.008421 (+  1793us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:43.038120 (+ 29699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:43.038159 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:43.040218 (+  2059us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.111804 (+ 71586us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:43.182703 (+ 70899us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.184864 (+  2161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.184938 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.209148 (+ 24210us) raft_consensus.cc:1685] finished
0709 13:03:43.209176 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.209344 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15109,"prepare.queue_time_us":16070,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"thread_start_us":127473,"threads_started":2,"wal-append.queue_time_us":92499}
I20260709 13:03:43.192067  7503 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.204208  7403 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:43.174866  7500 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.226222  2297 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:03:43.237002  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:43.171541  7566 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:03:43.178800  7591 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:03:43.189599  7545 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:03:43.212549  7397 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.447290  7545 consensus_queue.cc:1243] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:43.447326  7397 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:43.452149  7547 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:43.331622  7612 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:03:43.347851  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:43.458250  1715 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:43.460515  7629 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:03:43.342253  1609 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:43.462133  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 119970us
I20260709 13:03:43.464789  1609 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:43.209815  7578 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.347711  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:43.465935  7623 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:43.239850  7403 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.216642  7590 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.384760  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2600) took 1408 ms. Trace:
I20260709 13:03:43.417184  7597 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:43.473757  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 306787us
I20260709 13:03:43.222134  7604 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 13:03:43.402873  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:43.333223  7615 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:43.384842  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:43.516729  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:43.368208  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:43.355261  1714 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:43.517534  1635 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:43.518534  1710 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:43.519543  7286 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.351403  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:43.333005  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.520085  7286 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.521680  7590 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:43.239310  7503 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:43.522248  7286 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:43.223759  7404 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:43.191352  7106 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.474998  7403 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:43.525302  7572 consensus_queue.cc:1059] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:43.525882  7106 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:43.305222  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:43.526425  7572 raft_consensus.cc:3055] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:43.526851  7572 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:43.360052  1713 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:03:43.529562  7585 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.769s	user 0.004s	sys 0.000s
I20260709 13:03:43.383771  7566 raft_consensus.cc:3055] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:43.530170  7566 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:43.530781  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2955) took 1914 ms. Trace:
I20260709 13:03:43.436400  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 2993) took 1435 ms. Trace:
I20260709 13:03:43.327728  7565 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:43.532188  1641 rpcz_store.cc:276] 0709 13:03:42.001052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.055210 (+1054158us) service_pool.cc:224] Handling call
0709 13:03:43.061243 (+  6033us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:43.061425 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:43.123390 (+ 61965us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.436061 (+312671us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:43.436113 (+    52us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:43.436154 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.436227 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.436338 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65909,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":111,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":123,"spinlock_wait_cycles":1865472,"thread_start_us":365185,"threads_started":2,"wal-append.queue_time_us":230628}
I20260709 13:03:43.438331  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2587) took 1507 ms. Trace:
I20260709 13:03:43.443185  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2588) took 1512 ms. Trace:
I20260709 13:03:43.534749  1715 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:43.535912  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:43.536790  1643 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
W20260709 13:03:43.416006  7616 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:03:43.444221  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 97424.9us
I20260709 13:03:43.326995  7458 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:43.457607  7583 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.696s	user 0.001s	sys 0.002s
W20260709 13:03:43.296943  1682 leader_election.cc:336] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 2.351s (SENT)
W20260709 13:03:43.545841  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 248947us
I20260709 13:03:43.478812  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:43.479405  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.483553  7550 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:43.484663  7518 raft_consensus.cc:3055] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:43.551239  2268 leader_election.cc:400] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20.
I20260709 13:03:43.552770  2268 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:43.488098  7567 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:43.553653  7567 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:43.555706  7403 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.556850  7168 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:43.558327  7403 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:43.488955  2305 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:03:43.492725  7109 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.561539  7109 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:43.474222  7500 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:43.496363  1718 raft_consensus.cc:1240] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:43.497279  7597 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:43.502619  7495 consensus_queue.cc:260] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.555720  7550 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:43.564184  7495 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:43.564626  7550 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:43.565860  7550 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:43.566593  7569 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:43.568994  7569 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:43.505133  6959 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.570489  7569 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:43.571972  1709 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 20: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:43.572254  6959 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:43.517081  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:43.526081  7582 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.766s	user 0.000s	sys 0.003s
I20260709 13:03:43.181023  7520 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.576514  7600 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:43.577768  7520 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:43.502717  1716 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:43.549081  7572 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.580641  2298 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:03:43.581091  2298 pending_rounds.cc:85] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 9
I20260709 13:03:43.581365  2298 pending_rounds.cc:107] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:03:43.581636  2298 raft_consensus.cc:2889] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:43.581955  2298 pending_rounds.cc:107] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:03:43.582522  2298 raft_consensus.cc:2889] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:43.583340  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:43.583958  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.585003  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:43.581110  7572 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:43.551146  7518 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:43.587795  7518 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.589224  7518 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:43.486451  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 1327) took 1182 ms. Trace:
I20260709 13:03:43.590736  7567 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.591071  2304 rpcz_store.cc:276] 0709 13:03:42.304311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.304550 (+   239us) service_pool.cc:224] Handling call
0709 13:03:43.486211 (+1181661us) spinlock_profiling.cc:243] Waited 58.5 ms on lock 0x7b5400168354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:43.486381 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134644480}
I20260709 13:03:43.591687  1711 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:43.562841  7458 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:43.586683  7569 raft_consensus.cc:697] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:43.593845  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:43.596338  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:43.597270  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:43.598203  1718 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:43.602698  1718 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: [10.10-12.11]
I20260709 13:03:43.594272  7569 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.604964  7502 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:43.606422  7502 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:43.606875  7502 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:43.608116  7502 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.611217  1711 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:03:43.611338  7502 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:43.613953  1711 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:43.614056  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:43.581877  7578 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:43.583851  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1656) took 1703 ms. Trace:
I20260709 13:03:43.457710  7397 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.618886  1718 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:43.531252  7566 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.614889  2298 rpcz_store.cc:276] 0709 13:03:41.880039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.880197 (+   158us) service_pool.cc:224] Handling call
0709 13:03:41.880564 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:43.583069 (+1702505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.583632 (+   563us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b5400686ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:43.583796 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3909242752}
I20260709 13:03:43.437865  7394 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:43.630016  1709 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:03:43.475765  7634 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:43.652006  7397 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:43.532675  7545 consensus_queue.cc:1059] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.100s
W20260709 13:03:43.656524  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.658948  7503 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.630729  7591 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.476514  7619 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.174s
I20260709 13:03:43.669087  1682 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:43.675797  1709 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:43.665783  7572 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:03:43.664713  7637 consensus_queue.cc:1059] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:03:43.692981  1716 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:43.665912  7106 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:03:43.699733  7589 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:43.699935  7567 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:43.700314  7589 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:43.700757  7589 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:43.676868  7499 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:03:43.674048  6959 consensus_queue.cc:1059] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:03:43.677266  7520 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:03:43.673981  7394 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:03:43.706766  6959 raft_consensus.cc:3055] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:43.690199  7458 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:43.677516  7500 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:03:43.691994  7503 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:43.670769  7591 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:43.710035  7401 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.586s	user 0.010s	sys 0.004s
I20260709 13:03:43.709481  7566 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:43.720754  7401 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:43.728191  7591 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:43.728513  7540 raft_consensus.cc:3055] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:43.729022  7540 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:43.728434  7599 raft_consensus.cc:3055] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:43.733716  7599 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:43.734808  7599 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.735307  7589 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:43.736246  7599 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:43.737897  7589 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:43.729063  7604 raft_consensus.cc:3055] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:43.739280  7604 raft_consensus.cc:740] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:43.472607  1562 rpcz_store.cc:276] 0709 13:03:41.976171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:41.977932 (+  1761us) service_pool.cc:224] Handling call
0709 13:03:41.978247 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.032847 (+ 54600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.384509 (+1351662us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b5400276854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:43.384692 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92387712}
W20260709 13:03:43.282881  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.534608  1642 rpcz_store.cc:276] 0709 13:03:41.930614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.969790 (+1039176us) service_pool.cc:224] Handling call
0709 13:03:42.972809 (+  3019us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.973166 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.973199 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:42.973682 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:03:43.063671 (+ 89989us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:43.436946 (+373275us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.438011 (+  1065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.438086 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.438125 (+    39us) raft_consensus.cc:1685] finished
0709 13:03:43.438137 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.438265 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6408,"prepare.queue_time_us":63957,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":3721,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":162,"thread_start_us":407283,"threads_started":2,"wal-append.queue_time_us":99619}
I20260709 13:03:43.779524  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:43.519776  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:43.478899  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:43.786088  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:43.793107  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:43.523444  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 301722us
I20260709 13:03:43.535175  1638 rpcz_store.cc:276] 0709 13:03:41.930970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.970044 (+1039074us) service_pool.cc:224] Handling call
0709 13:03:42.970379 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:42.970570 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:43.056673 (+ 86103us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.183274 (+126601us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.440745 (+257471us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b18001b7a20. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb1455 00007fd7a1c99060 00007fd7a1cab716 00007fd7a1cbc287
0709 13:03:43.440840 (+    95us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:43.440886 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:43.440928 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.440989 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.443105 (+  2116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25948,"prepare.queue_time_us":208728,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":67,"raft.queue_time_us":146864,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":7043,"spinlock_wait_cycles":67967488,"thread_start_us":180958,"threads_started":3,"wal-append.queue_time_us":134324}
I20260709 13:03:43.799418  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:43.800530  1638 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:43.802050  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:43.531224  1639 rpcz_store.cc:276] 0709 13:03:41.616031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.378808 (+762777us) service_pool.cc:224] Handling call
0709 13:03:42.399384 (+ 20576us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:42.399746 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:42.399775 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:42.415354 (+ 15579us) log.cc:844] Serialized 564 byte log entry
0709 13:03:42.690256 (+274902us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:42.690300 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:42.690407 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:43.528941 (+838534us) spinlock_profiling.cc:243] Waited 5.53 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:43.530500 (+  1559us) raft_consensus.cc:1685] finished
0709 13:03:43.530521 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.530724 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12713728,"thread_start_us":274505,"threads_started":1,"wal-append.queue_time_us":345208}
I20260709 13:03:43.804751  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:43.808777  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1320) took 1265 ms. Trace:
I20260709 13:03:43.809183  1533 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:43.813580  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:43.814391  1460 catalog_manager.cc:5697] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:03:43.816622  1533 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:43.816812  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:43.818472  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:43.825459  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:43.825667  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:43.826138  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:43.806908  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:43.711854  6959 raft_consensus.cc:740] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:43.834906  7109 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:43.836328  1636 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:43.691525  6990 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:03:43.730787  7568 raft_consensus.cc:3055] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:43.851059  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:43.851783  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:43.852895  1534 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:43.853607  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:43.860389  7406 consensus_queue.cc:1059] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:43.863842  7361 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:43.868954  7394 raft_consensus.cc:3055] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:03:43.872332  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 49630.4us
I20260709 13:03:43.875227  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:43.794322  1566 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:43.876863  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:43.879484  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 45449.3us
I20260709 13:03:43.880295  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:43.882302  2304 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:43.888752  7499 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:43.889734  7539 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.891283  7520 consensus_queue.cc:1059] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:43.810403  1644 rpcz_store.cc:276] 0709 13:03:42.543262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:43.130248 (+586986us) service_pool.cc:224] Handling call
0709 13:03:43.158515 (+ 28267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:43.158717 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:43.178097 (+ 19380us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.534601 (+356504us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.797366 (+262765us) log.cc:844] Serialized 14 byte log entry
0709 13:03:43.807311 (+  9945us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:43.807420 (+   109us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:43.807632 (+   212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:43.807963 (+   331us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:43.808447 (+   484us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32174,"prepare.queue_time_us":114170,"prepare.run_cpu_time_us":127,"prepare.run_wall_time_us":130,"raft.queue_time_us":116407,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"thread_start_us":298337,"threads_started":2,"wal-append.queue_time_us":236043}
I20260709 13:03:43.895301  7500 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:43.900266  7508 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:03:43.901266  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 83314.9us
I20260709 13:03:43.902340  7567 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:43.905169  2269 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:43.907191  7635 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:43.897241  7565 raft_consensus.cc:3055] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:43.984774  7565 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:43.850903  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:43.986742  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:43.987342  7597 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:43.988485  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:43.910372  7540 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.906718  7393 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:03:43.989048  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.989634  7393 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.990334  7547 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:43.990490  7540 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:43.991147  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 165765us
I20260709 13:03:43.991575  7292 raft_consensus.cc:3055] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:43.992101  7292 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:43.992319  1682 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:43.992671  7393 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:43.879494  7106 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:43.993673  7106 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:43.993788  7292 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:43.957352  2301 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:43.958526  7520 raft_consensus.cc:3055] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:43.963953  6959 consensus_queue.cc:260] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:43.964303  7604 consensus_queue.cc:260] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.965626  7499 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:43.967020  1566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 3106) took 1638 ms (client timeout 1821 ms). Trace:
I20260709 13:03:43.967947  2305 raft_consensus.cc:2393] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 19: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:43.975108  7403 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.026311  7403 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.027868  7385 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.028412  7385 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.029385  7403 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:43.975270  7458 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:44.030810  7458 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:44.031435  7458 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:44.034057  1636 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:03:44.035233  6990 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.035976  6990 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.037496  7499 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:44.019656  7520 raft_consensus.cc:740] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:44.038545  6990 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:43.977394  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:43.977641  7486 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:43.978276  1638 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:43.934031  7394 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:43.980224  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 163853us
I20260709 13:03:43.995004  7106 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:43.932591  7361 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:43.995424  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:43.996388  7497 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:44.092095  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:43.999578  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:43.999617  7393 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:44.000006  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 95232.4us
I20260709 13:03:44.000795  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:44.131812  1640 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:43.950381  7568 raft_consensus.cc:740] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:44.003275  7397 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:44.144871  7106 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:44.145460  7397 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:44.145934  7568 consensus_queue.cc:260] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.003963  2268 leader_election.cc:400] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19.
I20260709 13:03:44.146983  7393 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.147087  2268 leader_election.cc:403] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:03:44.147522  7393 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:44.147953  7397 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:44.147570  7394 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:44.148494  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 144904us
I20260709 13:03:44.149540  7394 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:44.004405  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:44.004880  7547 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:43.944240  7539 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.005445  7399 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.005978  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:44.010463  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:44.011296  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:44.011412  7513 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:43.950515  7500 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:43.955825  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 103446us
I20260709 13:03:43.995343  7589 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:44.024339  6959 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:44.024192  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:44.025705  7604 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:44.166323  7539 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:44.170802  7502 consensus_meta.cc:287] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.559s	user 0.015s	sys 0.005s
I20260709 13:03:44.172116  7361 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:44.173746  7361 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:44.175138  7513 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:44.175688  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 180449us
I20260709 13:03:44.176496  7550 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:44.177798  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:44.178746  7550 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:44.039455  7520 consensus_queue.cc:260] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.179198  7550 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:44.179730  7550 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:44.179912  7520 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:43.975888  1460 catalog_manager.cc:5697] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:44.184330  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:44.184463  1460 catalog_manager.cc:5697] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 11 to 13, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 13 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:44.021222  1566 rpcz_store.cc:269] 0709 13:03:42.308577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.362541 (+ 53964us) service_pool.cc:224] Handling call
0709 13:03:43.792805 (+1430264us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b5800002a98. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ceee 0000000000375939 00007fd7a6bf01e0 00007fd7a6cb90c4 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:03:43.947306 (+154501us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3849600}
I20260709 13:03:44.030697  7385 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:44.047169  7503 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.187646  7503 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.188387  7500 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.189910  7500 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:44.190274  7503 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:44.048303  7292 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:43.986320  7565 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.060566  1682 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:44.088762  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.191797  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 170695us
I20260709 13:03:44.193902  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:44.091917  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.194980  7572 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:03:44.126679  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.196424  7572 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:44.128696  7486 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:44.137699  7497 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:03:44.143239  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.143451  2303 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:44.134446  1533 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:44.144891  1644 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:44.199931  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:44.151412  7547 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.151412  7599 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19.
W20260709 13:03:44.195089  1681 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.821s (SENT) [suppressed 39 similar messages]
I20260709 13:03:43.998890  7576 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:44.198894  7567 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.206496  7576 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.206734  7567 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.207445  7168 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:44.200340  7497 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:44.209110  7576 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:44.209350  7567 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:44.205845  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.090879  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 84331.1us
I20260709 13:03:44.200095  7486 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.199996  7399 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.211853  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:44.226769  2302 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:44.227509  1718 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:03:44.153016  7397 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:44.234318  1462 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60118 (request call id 231) took 45243 ms (client timeout 15000 ms). Trace:
W20260709 13:03:44.236269  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.246279  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 52547.1us
I20260709 13:03:44.246009  7397 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.247162  2303 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:44.247298  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:44.246457  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:44.250384  7397 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:44.250319  1682 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:44.177798  7513 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:44.251114  7572 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:44.252584  7399 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:44.254698  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:44.255419  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 71275.2us
I20260709 13:03:44.257745  7458 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.262694  1715 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:03:44.263616  1566 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:44.264329  7502 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:44.267831  1562 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:44.268427  7540 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:44.269449  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:44.271903  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:44.246013  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.273571  1643 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:03:44.277995  1564 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:44.278679  1565 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:44.279673  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.280926  1560 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:44.281908  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.271776  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:44.287858  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 153595us
I20260709 13:03:44.288537  1568 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:44.288374  1638 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:44.289239  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.326598  1716 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:44.290418  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:44.291020  7503 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:03:44.291512  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 64635.1us
I20260709 13:03:44.291972  7385 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:44.292593  7397 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.293458  7649 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:44.293964  2303 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
W20260709 13:03:44.238811  1462 rpcz_store.cc:269] 0709 13:02:58.973477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.973700 (+   223us) service_pool.cc:224] Handling call
0709 13:03:44.202098 (+45228398us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:37.468033 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:37.469004 (+   971us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:37.469067 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:03:37.469092 (+    25us) tablet.cc:662] Decoding operations
0709 13:03:37.470806 (+  1714us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:37.472568 (+  1762us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:37.473715 (+  1147us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:03:37.477119 (+  3404us) tablet.cc:701] Row locks acquired
0709 13:03:37.477176 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:03:37.477387 (+   211us) write_op.cc:270] Start()
0709 13:03:37.477549 (+   162us) write_op.cc:276] Timestamp: P: 1783602217477316 usec, L: 0
0709 13:03:37.477586 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:03:37.478397 (+   811us) log.cc:844] Serialized 6345 byte log entry
0709 13:03:39.022990 (+1544593us) op_driver.cc:464] REPLICATION: finished
0709 13:03:39.061468 (+ 38478us) write_op.cc:301] APPLY: starting
0709 13:03:39.061571 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:39.072721 (+ 11150us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:39.072868 (+   147us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:39.081262 (+  8394us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:39.081548 (+   286us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 13:03:39.081592 (+    44us) write_op.cc:312] APPLY: finished
0709 13:03:39.081915 (+   323us) log.cc:844] Serialized 148 byte log entry
0709 13:03:39.082149 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:39.082307 (+   158us) write_op.cc:454] Released schema lock
0709 13:03:39.082375 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:39.082832 (+   457us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:39.082849 (+    17us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4348328,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":38152,"apply.run_cpu_time_us":20012,"apply.run_wall_time_us":21351,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71039,"num_ops":16,"prepare.queue_time_us":251954,"prepare.run_cpu_time_us":36707,"prepare.run_wall_time_us":152700,"raft.queue_time_us":866675,"raft.run_cpu_time_us":18799,"raft.run_wall_time_us":38419,"replication_time_us":1545228,"thread_start_us":1169795,"threads_started":4,"wal-append.queue_time_us":173574,"wal-append.run_cpu_time_us":27987,"wal-append.run_wall_time_us":2425864}]]}
I20260709 13:03:44.305327  7497 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:44.307253  7568 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:44.301172  7565 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:44.250355  1709 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:03:44.290124  7518 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.700s	user 0.010s	sys 0.008s
I20260709 13:03:44.326042  1561 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:44.398301  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:44.399540  1561 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 20: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:44.400704  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.401515  1561 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:44.312875  7547 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:44.334885  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 44572.4us
I20260709 13:03:44.350575  1569 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:44.350744  6959 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:44.350939  1717 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 12: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:03:44.405876  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 75131.4us
I20260709 13:03:44.340416  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.331468  7503 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:44.356242  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:44.353595  1563 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
W20260709 13:03:44.359699  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.375990  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.340147  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.405105  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:44.351792  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.417680  1562 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:44.417894  1566 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:44.418236  1567 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:44.403739  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.351316  7540 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.403088  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.424068  1565 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:44.429903  1562 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:44.431161  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:44.432540  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 182410us
I20260709 13:03:44.432791  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.434172  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:44.435133  7572 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:44.435525  1533 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:44.436020  7393 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.438441  7393 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:44.439083  7397 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:44.440807  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:44.415347  7385 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.441515  7397 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:44.443300  7385 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:44.445286  2296 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:44.434818  1562 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:44.405498  7399 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:03:44.448233  7458 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:44.422591  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.450529  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:44.428412  1568 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:44.451804  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:44.451963  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.452513  7650 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:44.452723  1568 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:03:44.453191  7650 raft_consensus.cc:697] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:44.453220  1560 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:44.396586  1460 catalog_manager.cc:5697] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 8 to 9, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 9 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:44.455500  1562 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:44.456152  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.457289  1561 raft_consensus.cc:2410] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 16 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:03:44.457125  1460 catalog_manager.cc:5697] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 11 to 13, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 13 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:44.460541  7550 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.434775  1682 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:44.441540  1564 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:44.419008  7503 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:44.467998  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.446031  7540 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:44.463299  1683 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:44.468977  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.471293  1566 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:44.478456  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 43807.9us
W20260709 13:03:44.481197  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 61799.8us
I20260709 13:03:44.472978  1532 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:44.475104  7397 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:44.501900  1637 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:03:44.503875  1641 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:44.508298  1568 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:03:44.508455  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 73096.1us
I20260709 13:03:44.509097  7404 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:44.510859  7615 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:44.541909  7497 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:44.514222  7638 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:44.514456  7604 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:44.517426  7502 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:44.544518  7502 raft_consensus.cc:697] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:44.544703  7403 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:44.545110  7403 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:44.545576  7502 consensus_queue.cc:237] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:44.518272  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:44.519277  2300 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:44.547646  1566 raft_consensus.cc:2393] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:44.550271  1460 catalog_manager.cc:5697] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:44.552255  1569 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:03:44.552902  7632 log.cc:927] Time spent T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:03:44.552776  7403 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:44.454584  7650 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.554198  7403 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:44.557292  7550 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:44.558494  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:44.559377  1638 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:03:44.559769  1638 pending_rounds.cc:85] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 10
I20260709 13:03:44.531665  1644 pending_rounds.cc:85] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 8
W20260709 13:03:44.539367  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 76232us
I20260709 13:03:44.540266  1565 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:03:44.550323  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:44.565809  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.567103  1683 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:44.559993  1638 pending_rounds.cc:107] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:03:44.572679  1638 raft_consensus.cc:2889] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:44.560750  1644 pending_rounds.cc:107] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:03:44.573462  1644 raft_consensus.cc:2889] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:44.532915  6959 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:03:44.568651  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:44.577391  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:44.577968  7649 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:44.578253  1566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:03:44.592612  1566 raft_consensus.cc:606] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:44.592973  1566 raft_consensus.cc:614] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:44.575305  7397 raft_consensus.cc:697] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:44.572347  7361 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:44.578017  2268 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:44.589083  2302 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 10.
W20260709 13:03:44.594267  7652 log.cc:927] Time spent T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:03:44.594885  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 264220us
W20260709 13:03:44.598764  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 172780us
I20260709 13:03:44.600529  7545 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:44.600826  7292 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:44.601207  7545 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:44.604628  1564 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:44.587111  7497 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:44.614305  7397 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.596177  7361 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:44.619232  7361 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:44.623443  7497 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:44.625838  7497 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:44.628312  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 50435.1us
I20260709 13:03:44.630600  7393 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:44.632406  7599 consensus_queue.cc:1059] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:44.633098  7518 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:44.635088  1533 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:44.635438  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:44.648959  2304 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:44.488577  7635 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.570407  7404 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:44.656409  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.656952  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:44.670903  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.671869  1566 raft_consensus.cc:2393] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 16: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:44.672326  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:44.609754  7550 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.671825  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:44.679073  1635 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:44.679764  7139 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:44.680399  7394 consensus_meta.cc:287] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.530s	user 0.014s	sys 0.000s
I20260709 13:03:44.680979  7615 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.681622  7615 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.684049  7615 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:44.684195  7520 raft_consensus.cc:3055] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:03:44.684671  7520 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:44.685719  7520 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.676563  7168 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:44.687788  7518 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:44.687924  7168 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:44.688386  7503 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:44.688534  7518 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.689507  1562 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:44.690181  7168 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:44.690586  7503 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:44.673269  7638 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
W20260709 13:03:44.710893  1681 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.070s	sys 0.118s
W20260709 13:03:44.711412  1681 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.070s	sys 0.118s
I20260709 13:03:44.691672  7649 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:44.691538  7550 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:44.692974  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.672396  7635 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.693943  2300 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:03:44.696051  7520 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:44.671768   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:44.700596  1635 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:44.652236  7292 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:03:44.702097  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:44.702847  7404 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:44.702653  7604 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:44.702121  7599 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:03:44.703320  7394 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:44.704619  1718 raft_consensus.cc:2393] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 16: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:44.705895  7545 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:44.711825  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.712435  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 517394us
I20260709 13:03:44.663064  7393 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:44.691313  1715 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:03:44.734184  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 57425.6us
I20260709 13:03:44.552237  1460 catalog_manager.cc:5697] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 11 to 13, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 13 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:44.680574  7139 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:03:44.758656  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 123731us
W20260709 13:03:44.760773  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 125409us
I20260709 13:03:44.788108  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:44.802973  2304 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 15: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:44.746974  7497 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.802805  1609 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:44.808380   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:44.809273  7139 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.801766  7565 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.812073  1533 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:44.812837  7565 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:44.815327  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:44.816496  7168 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:44.816915  7168 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:44.816846  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:44.817569  7168 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:44.821795  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:44.765236  7637 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:03:44.852730  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.883683  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:44.885368  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.818713  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:44.854238  7497 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:44.857260  7393 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:44.867108  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 264856us
W20260709 13:03:44.872315  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 71407.1us
I20260709 13:03:44.867401  7565 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:44.859054  7637 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:44.861265  7547 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:44.858714  7545 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:03:44.883683  7648 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 13:03:44.902287  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 154533us
W20260709 13:03:44.906068  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:44.906724  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 204725us
I20260709 13:03:44.914551  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:44.886308  6959 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.893354  7403 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:44.909626  7637 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:44.903916  7615 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:44.923525  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:44.908787  7638 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.933815  1682 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:44.937583  1532 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:44.921130  7394 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:03:44.926558  7545 raft_consensus.cc:697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:44.947091  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:44.941533  7599 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:44.947785  1532 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:44.941196  7394 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:44.948913  7638 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.949409  7589 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:44.950263  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:44.951007  7604 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:44.952536  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:44.968343  1562 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:44.933735  7565 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:44.929039  7403 raft_consensus.cc:697] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:44.952886  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:44.926966  6959 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:44.953820  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 135245us
I20260709 13:03:44.954380  7572 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:44.955163  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:44.956136  7497 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:44.957316  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:44.943061  7568 consensus_meta.cc:287] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.002s
I20260709 13:03:44.953908  7399 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:44.944818  7139 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:44.967463  7394 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:03:44.987959  1532 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:44.969343  7604 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:44.993943  1635 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:45.004354  7497 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:45.004892  7497 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:45.004495  7545 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.005182  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.005826  1638 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:45.005954  7497 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.008213  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:44.979105  7500 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:44.977288  7649 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:45.007184  7486 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:44.967463  7599 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:44.978951  7550 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:44.994701  7572 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:45.074759  7599 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:45.075973  1717 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:45.078979  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.079715  1638 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:45.081054  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:45.082504  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:45.083433  2298 raft_consensus.cc:2410] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
W20260709 13:03:45.074847  7677 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:03:45.008045  7604 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:45.088405  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 92629.7us
I20260709 13:03:45.092137  2269 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:45.093565  1566 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18. Ops: [15.12-17.13]
I20260709 13:03:45.072173  7139 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:45.100658  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.007308  7399 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:45.100520  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:45.044932  7500 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.107816  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:45.119014  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.117746  1567 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:03:45.129142  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:45.129912  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 215866us
I20260709 13:03:45.131294  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:45.134286  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:45.108680  7568 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:45.137573  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.138402  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:45.143905  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:45.122219  7500 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:45.147642  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 151510us
I20260709 13:03:45.148845  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:45.144766  1565 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:03:45.147773  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.153687  1683 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:45.151366  1715 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
W20260709 13:03:45.146374  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 250172us
I20260709 13:03:45.151413  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:45.136534  7576 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:45.160681  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:45.161356  1567 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:03:45.161793  1681 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:45.158545  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:45.130636  7518 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:45.143395  7597 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.179503  7597 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.181931  7597 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:45.143257  7568 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.166451  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.169430  6959 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:45.147989  7139 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:45.169978  1460 catalog_manager.cc:5697] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:45.164562  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:45.193749  1709 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
W20260709 13:03:45.208967  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 53427.2us
I20260709 13:03:45.209694  7497 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:45.210501  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:45.210664  7638 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:45.211275  7597 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:45.211758  7393 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:45.212195  7393 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:45.212594  7393 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:45.217211  7637 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.219789  7637 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:45.184804  7572 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:45.122674  7403 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.201423  7568 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:45.185851  7576 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:45.235551  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.280872  1715 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
W20260709 13:03:45.246639  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 82228.5us
I20260709 13:03:45.250625  1716 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:03:45.283983  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:45.263396  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:45.268399  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.244647  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:45.298466  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.239326  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.326839  1718 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
W20260709 13:03:45.252156  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.270282  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:45.329329  1711 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:45.248490  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.279282  1713 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
W20260709 13:03:45.246234  7361 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.626s	user 0.010s	sys 0.000s
W20260709 13:03:45.244976  7645 log.cc:927] Time spent T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:03:45.254611  1532 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:03:45.342103  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 87555us
W20260709 13:03:45.342604  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.342994  1532 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:45.264155  7576 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:45.351526  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:03:45.227567  7665 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:03:45.336529  7599 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.349227  7486 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:45.340006  7497 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:45.349936  6959 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:45.235925  7139 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:45.267987  7404 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:45.268103  7545 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:45.254753  7503 consensus_queue.cc:1059] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:03:45.232326  7635 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:45.362020  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 151634us
I20260709 13:03:45.357097  7567 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.269109  7687 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.380707  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:45.381160  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 111151us
I20260709 13:03:45.382208  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:45.383682  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.383980  1682 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:45.384908  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.385126  7635 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:45.385726  7106 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:45.383985  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:45.366537  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.222867  7604 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.263870  7394 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:45.364161  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 211078us
I20260709 13:03:45.277444  7500 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.277444  7168 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.362967  7486 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.395051  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:45.269745  7615 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.095s
I20260709 13:03:45.380113  7567 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:45.362743  7599 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.381011  7687 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.257619  7547 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:45.386189  7106 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:45.378508  6959 raft_consensus.cc:697] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:45.237581  7399 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.423605  7599 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:45.354151  7385 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:45.425081  7385 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.425220  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.425550  7385 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:45.346166  7361 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.386799  7676 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:45.270551  1460 catalog_manager.cc:5697] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 12 to 14, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:45.398519  1681 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:45.398799  1532 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:45.392835  7497 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:45.400498  1683 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:45.405643  1644 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:03:45.405917  7604 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:45.406476  7637 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.407312  7687 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:45.396075  7486 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:45.408942  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:45.409317  7567 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:45.409946  6959 consensus_queue.cc:237] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.411159  7500 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:45.411809  7168 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:45.413530  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:45.415072  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3110) took 1122 ms. Trace:
I20260709 13:03:45.417188  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:45.417550  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:45.418219  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:45.418506  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:45.419129  7684 log.cc:927] Time spent T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:03:45.419478  7681 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:45.419580  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:45.420248  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:45.404263  7106 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:45.420899  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.402328  7545 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:45.428686  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 93690.2us
W20260709 13:03:45.431593  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 47678.8us
I20260709 13:03:45.387699  7404 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:45.431249  7635 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:45.446049  7503 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:45.452479  1635 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:45.458079  2298 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 20: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:45.458302  1641 raft_consensus.cc:2393] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:45.438750  7497 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:45.462129  1637 raft_consensus.cc:2393] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:45.466152  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.445698  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.457741  7649 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.491704  1640 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:45.473721  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.475901  7508 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:03:45.476226  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 77813.7us
I20260709 13:03:45.460635  7681 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.476753  7292 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.479127  6990 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:45.476145  1713 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:45.450551  1563 rpcz_store.cc:276] 0709 13:03:44.292703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.423341 (+130638us) service_pool.cc:224] Handling call
0709 13:03:44.440765 (+ 17424us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:44.517102 (+ 76337us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b180028fc00. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb1455 00007fd7a1c99060 00007fd7a1c99bfb 00007fd7a1ca464b 00007fd7a1ca0d8f
0709 13:03:44.916466 (+399364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:44.916494 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:44.940969 (+ 24475us) log.cc:844] Serialized 52 byte log entry
0709 13:03:45.144790 (+203821us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:45.355421 (+210631us) log.cc:844] Serialized 14 byte log entry
0709 13:03:45.356502 (+  1081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:45.356572 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:45.414703 (+ 58131us) raft_consensus.cc:1685] finished
0709 13:03:45.414822 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:45.415007 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37511,"prepare.queue_time_us":268505,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":39531,"raft.run_cpu_time_us":402,"raft.run_wall_time_us":25918,"spinlock_wait_cycles":5471104,"thread_start_us":752449,"threads_started":3,"wal-append.queue_time_us":411128}
I20260709 13:03:45.481988  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:45.504761  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:45.505970  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:45.483170  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:45.485553  2300 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 11.
I20260709 13:03:45.485944  7683 raft_consensus.cc:3055] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:45.487057  7394 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:45.488322  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.447872  7637 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.509352  7568 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.497632  1636 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:45.477258  7589 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.500553  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.502661  1681 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:45.521660  2269 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:45.509244  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 108835us
I20260709 13:03:45.513939  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:45.515156  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:45.470670  7503 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.496708  7497 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:45.512806  7676 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:45.607147  2302 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:45.608683  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.513964  7683 raft_consensus.cc:740] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:45.611306  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 89741.2us
I20260709 13:03:45.614401  2300 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
W20260709 13:03:45.618358  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 104506us
I20260709 13:03:45.620468  2298 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:45.587462  7503 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:45.633992  1682 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:45.645001  7589 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.651258  7674 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:45.651777  7674 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.652144  7674 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:45.618677  7599 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:45.653720  7599 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:45.654183  7599 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:45.658890  7637 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:45.665076  7568 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:45.619017  7502 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:45.668713  7139 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:45.668358  7683 consensus_queue.cc:260] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:45.669611  7139 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:45.670373  7683 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:45.670182  7139 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.597035  7550 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:45.596721  7168 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:45.671451  7361 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:45.503453  7649 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:03:45.502193  7681 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:45.691852  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:45.696607  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.613667  7520 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.517405  7500 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:45.699338  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.443053  1460 catalog_manager.cc:5697] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 10 to 11, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:45.706821  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.648219  7589 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:45.710889  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 208311us
I20260709 13:03:45.528846  7687 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:45.671906  7394 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:45.523455  7508 raft_consensus.cc:697] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:45.714892  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:45.513376  7292 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.735879  1640 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:45.539188  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.739543  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:45.741631  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:45.689350  7550 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:45.744203  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:45.749926  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:45.750530  1639 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:45.711623  7565 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:45.751277  1644 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:45.758795  1642 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:03:45.713053  7635 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:45.759090  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:45.702904  7568 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 13:03:45.767117  1681 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:45.769855  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:45.741312  7361 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:45.745985  7394 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:45.754385  7550 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:45.780537  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:45.782968  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:45.794417  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 279390us
I20260709 13:03:45.795280  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:45.728646  7520 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.803046  6990 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:45.804229  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:45.805347  2296 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:45.803334  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:45.807180  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:45.808665  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:45.809456  1561 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:03:45.809798   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:45.776790  7565 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.811492  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:45.812134  2305 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:45.813237  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:45.813334  7565 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:45.823206  7399 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:45.824169  7399 consensus_queue.cc:1243] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:03:45.826395  1638 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:45.814626  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:45.833439  1640 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:45.806665  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:45.836349  2305 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:45.766588  7638 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:45.837111  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 70096.8us
I20260709 13:03:45.839545  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:45.844164  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:45.845999  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:45.841801  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:45.848201  1568 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:45.849900  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:45.853106  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:45.834661  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:45.856678  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.858112  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:45.779055  7394 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:45.841997  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.855955  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.872303  2305 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 13:03:45.874178  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 78970us
I20260709 13:03:45.874995  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.883653  1718 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:03:45.792853  7589 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.888346  1716 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:03:45.895541  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 49665.4us
I20260709 13:03:45.897048  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:45.898556  2305 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: []
W20260709 13:03:45.860374  7690 log.cc:927] Time spent T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:03:45.907177  1635 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:03:45.908004  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:45.900435  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 160583us
I20260709 13:03:45.908865  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:45.902074  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:45.883850  1710 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:45.843019  7403 raft_consensus.cc:3055] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:45.906738  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.912959  1562 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:45.915162  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:45.915410  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:45.835989  7635 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:45.840994  7404 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:45.922691  1711 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 11: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:45.923655  1713 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 20: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:45.927484  1565 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:45.860054  7649 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:45.884404  7568 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.784909  7361 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:45.946545  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:45.860603  7572 consensus_queue.cc:1059] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 13:03:45.899730  7702 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:45.879520  7638 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:45.917580  7403 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:45.872618  6990 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:03:45.983680  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:45.964078  1638 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:45.890039  7597 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:03:45.925918  7518 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.916154  7589 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:45.978619  1714 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:03:45.989274  1639 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:46.008569  1562 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:46.013950  1642 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:45.990031  1643 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:03:45.975945  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.039403  1711 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 15: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:03:45.998538  7385 consensus_meta.cc:287] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.572s	user 0.006s	sys 0.002s
I20260709 13:03:46.003144  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:45.925592  7486 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:46.077735  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 180794us
I20260709 13:03:46.065270  7545 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:45.947921  7682 consensus_queue.cc:1059] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 13:03:45.941429  7635 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:45.988438  7707 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:45.957823  7568 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.093138  1718 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
W20260709 13:03:46.094596  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.014163  7502 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:46.107759  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:46.098459  1609 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:46.043670  7710 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:45.998723  7497 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:46.115391  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:46.118054  2300 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:03:46.113072  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:46.123062  2305 raft_consensus.cc:2393] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 15: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:03:46.103443  7393 consensus_meta.cc:287] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.890s	user 0.013s	sys 0.000s
I20260709 13:03:45.959893  7508 consensus_queue.cc:237] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.062542  7385 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.015749  7518 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:46.133808  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 215489us
I20260709 13:03:46.114161  2269 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:46.100620  7687 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.103286  7520 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:46.142086  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:46.166131  1642 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 21 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:03:46.122152  7292 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:46.166668  2299 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:03:46.003558  7403 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:46.112016  7139 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:46.115599  7545 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:03:46.112080  7713 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:45.989938  7702 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.123407  6959 raft_consensus.cc:3055] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:46.183187  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:46.191813  1562 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 20: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:46.173352  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:46.197281  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:46.197875  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:46.198295  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:46.198837  1564 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:46.199007  1563 raft_consensus.cc:2368] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12.
I20260709 13:03:46.199110  1568 raft_consensus.cc:2393] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:46.216143  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 235773us
I20260709 13:03:46.123868  7500 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:03:46.262180  1638 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:46.267560  7385 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:46.268213  7518 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:46.270040  7497 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:46.187084  6959 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:46.274788  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:46.260912  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 65704.2us
I20260709 13:03:46.284608  7637 raft_consensus.cc:3055] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:46.285248  7637 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:46.138934  7393 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.289247  7393 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:46.208666  7520 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:46.103166  7707 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.185515  7649 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:46.291581  2303 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:46.281036  1561 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:46.296669  1681 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:46.167421  7687 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:46.305372  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:46.273726  7545 raft_consensus.cc:697] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:03:46.293018  7599 consensus_meta.cc:287] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.638s	user 0.009s	sys 0.003s
I20260709 13:03:46.315356  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:46.304628  7403 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:46.308605  7393 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:45.946327  1460 catalog_manager.cc:5697] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 9 to 12, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 12 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:46.320077  6990 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.311642  7404 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:46.337631  1641 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
W20260709 13:03:46.350062  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.296515  7385 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.319314  7520 raft_consensus.cc:697] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:46.331118  7726 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:46.371493  7726 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:46.372345  7687 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:46.372596  7726 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.374185  7726 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:46.209393  7713 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.376508  6990 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.377020  7713 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:46.293895  7615 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:03:46.368536  7486 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:46.286628  7637 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.368780  7589 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:46.369061  7404 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.370426  7520 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.317716  7649 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.340018  7599 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.353906  7393 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:46.376538  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 102971us
I20260709 13:03:46.398406  1636 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:03:46.339833  7714 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:46.396139  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:46.380162  7637 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:46.405409  7712 log.cc:927] Time spent T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
W20260709 13:03:46.383915  7676 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.760s	user 0.009s	sys 0.003s
W20260709 13:03:46.405053  7723 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:03:46.442945  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:46.406342  7708 log.cc:927] Time spent T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.351s	user 0.000s	sys 0.001s
I20260709 13:03:46.382067  7404 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:46.387207  7649 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:46.389263  7599 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:46.378880  6990 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:46.458870  7486 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:46.461954  7734 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:46.462507  7486 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.462772  7734 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:46.464802  7486 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:46.448949  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 52903us
I20260709 13:03:46.382129  7168 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:46.453779  7713 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.465741  7168 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:46.466096  7713 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.467250  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:46.468159  7404 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:46.468568  1460 catalog_manager.cc:5697] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 9 to 13, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 13 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:46.476082  1533 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:46.476622  7711 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.477206  7711 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.477617  7486 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:46.477694  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:46.393034  7393 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:46.479146  7704 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.338s	user 0.001s	sys 0.000s
I20260709 13:03:46.479501  1533 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:46.480428  7729 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:46.483165  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:46.483407  7106 consensus_meta.cc:287] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.011s	user 0.008s	sys 0.004s
I20260709 13:03:46.483934  7729 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:46.483834  7106 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.484742  7676 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:46.485764  1635 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:46.485697  1533 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:46.486408  7106 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:46.486812  7729 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:46.404352  7545 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.444043  7687 raft_consensus.cc:2764] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:03:46.405471  7696 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:46.379710  7361 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:46.393584  7385 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:46.469569  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:46.504508  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 189921us
W20260709 13:03:46.513842  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 44349.1us
I20260709 13:03:46.515676  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:46.361479  6959 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.478883  7486 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:46.478881  7550 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.653s	user 0.007s	sys 0.002s
I20260709 13:03:46.521970  6959 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:46.519030  7615 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:46.525532  7715 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:46.526459  7674 consensus_meta.cc:287] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.874s	user 0.009s	sys 0.000s
I20260709 13:03:46.526145  7687 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:46.483868  1639 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:46.488479  7721 raft_consensus.cc:3055] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:46.528587  7702 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:46.528825  7721 raft_consensus.cc:740] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:46.529863  7721 consensus_queue.cc:260] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.522342  7550 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.532873  7550 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:46.533547  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:46.533644  1644 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:46.534577  7394 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.635s	user 0.010s	sys 0.001s
I20260709 13:03:46.535048  7394 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.536422  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:46.537122  2300 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:03:46.537532  7394 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:46.538010  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:46.538857  2305 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
W20260709 13:03:46.539031  7503 consensus_meta.cc:287] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.841s	user 0.010s	sys 0.002s
I20260709 13:03:46.539558  7503 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:46.539939  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:46.406122  7717 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.533686  1460 catalog_manager.cc:5697] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 11 to 13, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 13 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:46.541409  7717 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.542320  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:46.543110  1565 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:46.543948  7717 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:46.544147  7720 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:03:46.544885  2304 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 15: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:46.545436  7721 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:46.542659  1460 catalog_manager.cc:5697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 9 to 13, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 13 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:46.552322  7572 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.553169  7572 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.553999  1460 catalog_manager.cc:5697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 2 to 11. New cstate: current_term: 11 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:46.556137  7572 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:46.536643  2296 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:46.505131  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 57665.8us
W20260709 13:03:46.551453  7725 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:03:46.526146  7715 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:46.556888  1609 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:46.444034  7638 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:46.527801  7707 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:46.574110  1609 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:46.561728  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.562093  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:46.563882  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.565203  7550 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:46.566689  1711 raft_consensus.cc:1240] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: []
W20260709 13:03:46.567407  7576 consensus_meta.cc:287] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.204s	user 0.014s	sys 0.000s
I20260709 13:03:46.567498  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:46.578321  1717 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:46.571247  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:46.579097  1644 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:03:46.563604  2269 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:46.460647  7635 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:46.586308  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:46.587312  1639 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:46.590085  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:46.591441  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:46.592413  1635 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:46.491591  1533 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:46.590996  1644 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:46.593228  2269 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:46.598474  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:46.599515  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:46.600157  1569 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:46.602769  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:46.604295  1607 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:46.594866  7635 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.582370  7734 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:46.611501  7615 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:46.612481  7713 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:46.576638  7568 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:46.582696  7702 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:46.603479  1560 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:46.591823  7707 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:46.610249  7635 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.618011  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.618643  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:46.591531  7638 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:46.608249  7687 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:46.649492  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:46.696189  1711 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:46.648219  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:46.665333  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 56602.4us
I20260709 13:03:46.620035  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.667716  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.668893  1713 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:46.598147  7361 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:03:46.556826  1460 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 11 to 12, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 12 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:46.603051  7717 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:46.710968  1635 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
W20260709 13:03:46.702708  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 84162.3us
W20260709 13:03:46.618844  7683 consensus_meta.cc:287] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.931s	user 0.006s	sys 0.007s
I20260709 13:03:46.704660  1715 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:46.694686  2296 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:03:46.645466  2269 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:46.712801  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:46.713465  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.708875  7550 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:46.571532  7674 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:46.718663  1716 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:46.589253  7576 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.620369  7615 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.127s
I20260709 13:03:46.686731  7731 consensus_queue.cc:1059] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:03:46.682126  7168 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:46.790017  1566 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:46.791543  7404 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:46.735181  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:46.792313  7404 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:46.792898  7737 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:46.793414  7737 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.793824  7737 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:46.700899  7638 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.654313  7385 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:46.759078  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 46361.1us
W20260709 13:03:46.733081  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 87707.8us
I20260709 13:03:46.744598  1718 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:03:46.689446  7707 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:46.796386  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.011s	sys 0.017s
I20260709 13:03:46.616940  7604 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.147s
I20260709 13:03:46.649009  7503 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:46.715109  7676 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:46.751262  7139 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:46.794884  7404 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:46.682562  7711 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:46.802143  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 67059.9us
W20260709 13:03:46.803385  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.012s	sys 0.017s
I20260709 13:03:46.747287  7520 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:46.798489  2269 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:46.821363  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:46.827538  1639 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:46.746978  7734 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 13:03:46.617619  7702 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:46.715499  7717 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.702452  1714 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:46.844864  1635 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:46.845187  1683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.006s	sys 0.029s
I20260709 13:03:46.811721  7615 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:46.685920  7399 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:46.719385  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 227893us
I20260709 13:03:46.856032  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:46.809999  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 106355us
I20260709 13:03:46.797878  7638 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:46.683269  7727 consensus_queue.cc:1059] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.187s
I20260709 13:03:46.798489  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:46.857800  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 59405us
I20260709 13:03:46.832913  1717 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:46.806191  7707 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:46.845631  1683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.006s	sys 0.029s
I20260709 13:03:46.820029  2269 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:46.836742  7361 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.839362  7702 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:46.846242  7604 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:46.849182  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:46.814725  7520 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.871908  7520 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:46.853173  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:46.876438  1640 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 11.
I20260709 13:03:46.859841  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:46.837306  7597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.000s	sys 0.031s
W20260709 13:03:46.856022  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.818s	user 0.008s	sys 0.029s
I20260709 13:03:46.863559  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:46.880199  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:46.858670  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:46.863806  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:46.864372  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 44483.5us
I20260709 13:03:46.720078  7550 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:46.842480  7717 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:46.882287  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.843664  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:46.882369  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.883255  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:46.884394  1718 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:46.884711  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:46.861557  7139 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:46.850814  7615 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:46.881567  2269 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:46.887372  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.895107  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.887430  1566 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:46.884891  1532 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:46.889437  1533 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:46.891378  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.008s	sys 0.030s
I20260709 13:03:46.890343  7683 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:46.898123  7518 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.898384  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:46.898789  7635 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:46.898633  7518 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.885356  7729 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:46.900023  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:46.900466  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:46.900836  7518 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:46.901152  2296 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:46.901522  7168 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:46.839694  7676 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:46.901273  7576 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:46.879220  7597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.000s	sys 0.032s
I20260709 13:03:46.891077  1560 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:46.902662  2268 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:46.889469  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:46.903765  1609 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:46.870687  7604 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.902122  7106 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:46.890343  7545 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:46.927201  1639 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:46.753921  1460 catalog_manager.cc:5697] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:46.914369  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.937472  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:46.919374  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:46.932114  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.908689  2296 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:03:46.932087  2297 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:46.885330  7550 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:46.924075  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.925127  2268 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:46.894436  7139 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.936614  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:46.909963  7539 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:03:46.890569  7404 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:46.905881  7735 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:46.926239  7604 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:46.929702  7635 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:46.930150  7547 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:46.926548  7399 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:46.930702  7734 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:46.923314  7394 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:46.964778  1460 catalog_manager.cc:5697] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 9 to 12, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 12 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:46.926180  7168 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:46.932193  7568 raft_consensus.cc:3055] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:46.967133  7168 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.967196  7568 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:46.967691  7168 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:46.932893  7385 raft_consensus.cc:3055] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:46.968364  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1698) took 1362 ms. Trace:
I20260709 13:03:46.968542  7385 raft_consensus.cc:740] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:46.968878  2297 rpcz_store.cc:276] 0709 13:03:45.606170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.606325 (+   155us) service_pool.cc:224] Handling call
0709 13:03:45.630703 (+ 24378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:46.939961 (+1309258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:46.968086 (+ 28125us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54005dc0d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:46.968280 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2708689664}
I20260709 13:03:46.969509  7385 consensus_queue.cc:260] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:46.970494  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:46.970721  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.934240  7674 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:46.926882  7393 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:46.926941  1532 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:46.942404  1715 raft_consensus.cc:2393] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:46.906167  7361 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.910300  7736 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:46.915287  7500 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:46.949009  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:46.972137  7399 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:46.929138  7599 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:46.952378  1533 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:46.954044  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:46.957473  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.938144  7715 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:46.938488  7545 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:46.971998  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:46.995784  7385 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:47.015345  7545 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:46.975997  2305 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:03:46.976912  2296 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:46.963537  7106 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:03:46.971875  1683 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:47.019974  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 48177.8us
I20260709 13:03:47.020908  1683 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:46.962769  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:46.968359  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:46.927582  7714 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:03:46.953639  7735 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.029269  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:46.994479  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:47.030001  1716 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 15: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:46.994797  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:46.995658  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:46.952189  7710 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:46.997020  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 44292.2us
I20260709 13:03:46.949186  7139 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:46.951696  7518 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:46.998268  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:46.998504  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 71678.5us
I20260709 13:03:47.000942  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:47.001156  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:47.039347  2300 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:47.039292  1569 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:47.001729  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:47.040156  1565 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:46.972934  7497 raft_consensus.cc:3055] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:47.003664  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:46.972617  7635 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:47.002519  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:46.951308  7486 consensus_queue.cc:1059] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:03:47.009091  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:46.974849  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:46.977474  1710 raft_consensus.cc:2368] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
I20260709 13:03:47.048583  1568 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:47.049361  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:46.957893  7404 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.050552  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:47.050752  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:47.051453  2298 raft_consensus.cc:2393] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:47.051613  1710 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 15: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:47.051854  7404 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:47.052426  7404 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:47.018292  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 66053.8us
W20260709 13:03:47.006111  7681 consensus_meta.cc:287] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.258s	user 0.014s	sys 0.000s
W20260709 13:03:47.029429  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 66782.8us
I20260709 13:03:47.053390  1533 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:47.053220  7681 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.054132  7547 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:03:46.994328  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:47.055241  1712 raft_consensus.cc:2368] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12.
I20260709 13:03:47.055712  7734 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:47.056264  7734 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:47.056509  7681 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:47.057260  7711 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.057350  7734 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.004731  7736 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.058696  7736 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:47.059026  7741 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:47.059033  7713 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 13:03:47.052935  7404 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:47.004187  7729 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:47.030010  2296 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:47.063251  7729 raft_consensus.cc:697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:47.064386  7404 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:47.064172  7729 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.065009  7404 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:47.036413  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:47.066120  2305 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:47.038107  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:47.009656  7599 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:46.977867  7743 consensus_queue.cc:1059] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:03:47.067210  1709 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:47.044842  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:47.067894  7681 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:47.068320  7681 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:47.049293  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:47.069636  1715 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:47.017620  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:47.067232  7599 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:47.057894  7711 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:47.072968  7711 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:47.073391  7736 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.074180  2305 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:47.008929  7399 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:47.046882  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:47.074257  7681 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:46.995038  7393 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.075606  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:47.079806  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:47.078474  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:47.079360  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:47.076545  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 74103.6us
I20260709 13:03:47.083487  1639 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:47.086069  2300 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:47.035758  7744 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:47.035969  6959 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:47.076318  2297 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:47.085443  1718 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:47.035691  7715 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:47.075106  7733 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:47.042237  7710 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.092482  7681 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:47.093628  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 1212) took 1241 ms. Trace:
I20260709 13:03:47.094452  7292 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:03:47.099493  7745 log.cc:927] Time spent T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:03:47.099347  1565 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:47.101379  7736 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:47.007993  7696 consensus_queue.cc:1059] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
W20260709 13:03:47.075752  7637 consensus_meta.cc:287] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.001s
I20260709 13:03:47.076383  7635 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:47.111933  7676 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:47.113775  7738 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:03:47.101985  7568 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.115149  7749 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:47.118062  7599 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:46.979229  1460 catalog_manager.cc:5697] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 7 to 9, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:47.121960  7399 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.113767  1568 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:47.126561  2296 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
W20260709 13:03:47.139891  7747 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:47.034080  7735 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:47.143059  7403 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.152539  7361 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:47.157326  7393 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:47.042933  7497 raft_consensus.cc:740] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:47.038774  7518 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.142267  6990 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:47.087776  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:47.180936  1532 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:03:47.189644  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 145573us
I20260709 13:03:47.196220  1561 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:47.192893  1712 rpcz_store.cc:276] 0709 13:03:45.852643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:45.852833 (+   190us) service_pool.cc:224] Handling call
0709 13:03:47.093408 (+1240575us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54001356d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:47.093545 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2763094912}
I20260709 13:03:47.085795  7743 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:47.199781  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:47.206280  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:47.206967  1711 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:47.013675  7615 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:47.206107  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:47.204921  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:47.222252  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:47.221822  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:47.224291  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 277921us
I20260709 13:03:47.197266  7292 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:47.229200  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:47.192941  7744 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:47.222756  7674 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:47.256556  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.219830  7727 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:47.277194  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.281504  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:47.285637  1683 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:47.236516  7739 raft_consensus.cc:3055] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:47.219106  7547 consensus_queue.cc:1059] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 13:03:47.261247  7676 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:47.260392  7749 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.667871  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:47.627640  1638 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:47.368785  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:47.626278  1644 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:47.277935  7568 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:47.675338  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:47.450112   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:47.613170  1560 consensus_meta.cc:287] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.697s	user 0.010s	sys 0.001s
I20260709 13:03:47.411079  7503 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:47.400266  7687 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.684s	user 0.012s	sys 0.000s
W20260709 13:03:47.619580  7550 consensus_meta.cc:287] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.672s	user 0.012s	sys 0.001s
W20260709 13:03:47.577666  7638 consensus_meta.cc:287] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.705s	user 0.010s	sys 0.002s
W20260709 13:03:47.737605  1533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.014s	sys 0.018s
W20260709 13:03:47.628028  7762 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:03:47.742904  1636 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:47.827768  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:47.866518  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:47.743960  1639 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:47.809826  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:47.744488  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 375918us
I20260709 13:03:47.872195  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:47.447779  7748 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:03:47.793951  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2634) took 1146 ms. Trace:
I20260709 13:03:47.877354  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:47.785795  1534 leader_election.cc:336] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.583s (SENT)
W20260709 13:03:47.810658  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 722964us
I20260709 13:03:47.795652  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:47.795905  1714 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:47.794348  1560 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
W20260709 13:03:47.798813  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 618019us
W20260709 13:03:47.881238  1533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.015s	sys 0.027s
I20260709 13:03:47.881042  7729 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:47.708376  7545 consensus_meta.cc:287] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.663s	user 0.011s	sys 0.000s
I20260709 13:03:47.882012  7486 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:47.656711  7500 consensus_meta.cc:287] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.644s	user 0.006s	sys 0.005s
I20260709 13:03:47.296164  7696 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.521s
W20260709 13:03:47.636690  7726 consensus_meta.cc:287] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.262s	user 0.016s	sys 0.000s
I20260709 13:03:47.743860  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:47.744616  7743 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:47.792420  7739 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:47.742571  2299 raft_consensus.cc:2376] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12. Re-sending same reply.
I20260709 13:03:47.802719  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:47.914300  2300 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:47.802991  7576 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:47.403818  7717 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:47.798537  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 55474.1us
W20260709 13:03:47.645557  7604 consensus_meta.cc:287] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.667s	user 0.004s	sys 0.007s
I20260709 13:03:47.440131  7404 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.357576  7751 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:47.277421  7737 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.930995  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:47.931766  1641 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:47.932209  7737 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:47.932972  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:47.913173  1639 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:47.791239  7744 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:47.643672  7758 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:03:47.809252  1709 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:47.882781  7486 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.874205  1568 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:47.675387  7750 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:47.498569  7753 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:03:47.484752  7707 consensus_meta.cc:287] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.613s	user 0.009s	sys 0.000s
I20260709 13:03:47.878608  1534 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:47.623739  7599 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:47.799147  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 513635us
I20260709 13:03:47.458338  7497 consensus_queue.cc:260] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.538024  7736 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.877411  1711 rpcz_store.cc:276] 0709 13:03:46.647964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.649185 (+  1221us) service_pool.cc:224] Handling call
0709 13:03:47.793890 (+1144705us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:47.412424  7710 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:47.576122  7715 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.801401  1567 raft_consensus.cc:2410] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:03:47.300892  7711 raft_consensus.cc:3055] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:47.890107  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:47.965235  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 75207.8us
I20260709 13:03:47.966075  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:47.797767  7702 consensus_meta.cc:287] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.911s	user 0.005s	sys 0.009s
I20260709 13:03:47.803107  6990 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:47.897451  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:47.898162  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:47.518867  7399 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:47.905815  7681 consensus_meta.cc:287] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.713s	user 0.012s	sys 0.000s
I20260709 13:03:47.990268  7702 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.991070  1711 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:47.991218  7486 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:47.297030  7403 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.992094  7755 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:47.992496  7702 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:47.909994  2296 raft_consensus.cc:2393] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:47.994218  2299 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:03:47.802680  7734 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:47.817198  7637 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:47.816210  7687 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:47.559815  7139 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:47.990304  7399 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:47.999145  7734 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.013953  7139 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:48.014057  7734 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:47.950886  7404 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:48.014897  7361 raft_consensus.cc:3055] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:48.015954  7361 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:48.016724  7404 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:47.376192  7520 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.017377  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2636) took 1133 ms. Trace:
I20260709 13:03:48.017014  7361 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.017791  1560 rpcz_store.cc:276] 0709 13:03:46.884312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.884476 (+   164us) service_pool.cc:224] Handling call
0709 13:03:48.017194 (+1132718us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b540041ffd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:48.017319 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13059456}
I20260709 13:03:48.018554  7361 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:47.882198  7729 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.019233  2301 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:47.947878  7292 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.019410  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:47.837186  7763 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:47.468394  7168 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:47.922535  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 45270.8us
I20260709 13:03:47.690097  7735 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:47.909035  7385 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.901324  7545 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:47.937724  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2631) took 1740 ms. Trace:
I20260709 13:03:47.938117  7769 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.807672  7727 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.907863  7500 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:47.801194  7674 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:47.796217  7676 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.942526  1563 raft_consensus.cc:2368] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14.
I20260709 13:03:47.943145  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:47.225934  7394 raft_consensus.cc:697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:47.920775  7733 consensus_meta.cc:287] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.702s	user 0.006s	sys 0.007s
I20260709 13:03:47.945688  2303 raft_consensus.cc:2393] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 13: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:47.826570  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:48.114391  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:47.932870  7751 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.947043  1636 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:47.875128  7550 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:47.883867  7638 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.953608  1681 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:48.119813  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 166343us
I20260709 13:03:47.983366  7757 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:47.998903  7739 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:47.892027  1638 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:47.987727  1565 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:47.984719  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:47.988302  1561 raft_consensus.cc:2376] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13. Re-sending same reply.
W20260709 13:03:48.127863  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 143260us
I20260709 13:03:48.129211  2268 leader_election.cc:400] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12.
I20260709 13:03:48.129535  2268 leader_election.cc:403] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:03:47.988930  7765 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:47.990051  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2569) took 1089 ms. Trace:
I20260709 13:03:47.992358  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:47.996376  1643 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:47.301543  7518 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:47.922865  7576 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.134308  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:48.007236  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:48.014860  1640 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:47.816287  1644 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:47.993443  7403 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:48.018700  7520 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:47.803172  7764 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:48.019498  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:47.884289  7615 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:47.943814  7604 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.004825  1568 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:03:48.022884  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:48.164213  1635 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: [13.13-13.13]
W20260709 13:03:48.024950  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 106634us
I20260709 13:03:48.041505  1560 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
W20260709 13:03:48.164652  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 141869us
I20260709 13:03:47.966578  7768 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:48.042373  1532 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:03:48.167125  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 124904us
I20260709 13:03:48.167919  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:48.068616  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.987565  7711 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:47.985056  7760 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:48.030655  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1710) took 1095 ms. Trace:
I20260709 13:03:48.108994  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2638) took 1068 ms. Trace:
I20260709 13:03:48.021361  7292 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:48.105383  1564 rpcz_store.cc:276] 0709 13:03:46.197770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.199559 (+  1789us) service_pool.cc:224] Handling call
0709 13:03:46.199953 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:46.645627 (+445674us) spinlock_profiling.cc:243] Waited 49.9 ms on lock 0x7b18000ac960. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb1455 00007fd7a1c99060 00007fd7a1c99bfb 00007fd7a1ca464b
0709 13:03:46.645793 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:46.645826 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:46.646382 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 13:03:47.033725 (+387343us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:47.406726 (+373001us) log.cc:844] Serialized 14 byte log entry
0709 13:03:47.410872 (+  4146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.417797 (+  6925us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:47.937366 (+519569us) raft_consensus.cc:1685] finished
0709 13:03:47.937506 (+   140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:47.937646 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41120,"prepare.queue_time_us":454058,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":225911,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":45215,"spinlock_wait_cycles":114743168,"thread_start_us":877689,"threads_started":3,"wal-append.queue_time_us":631603}
I20260709 13:03:48.115204  1566 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:48.220919  2301 rpcz_store.cc:276] 0709 13:03:46.935492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.943181 (+  7689us) service_pool.cc:224] Handling call
0709 13:03:46.963418 (+ 20237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.020442 (+1057024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.030397 (+  9955us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400367ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:48.030593 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2427945856}
I20260709 13:03:47.916419  7743 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:48.052768  7404 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:48.118320  2304 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:47.964684  7707 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:47.957111  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 171386us
I20260709 13:03:48.120443  1718 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:03:47.965377  2300 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:47.985528  7715 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.111428  7769 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.238545  7707 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:48.239249  7769 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:48.125037  7717 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:48.125423  1569 raft_consensus.cc:2368] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 19. Current term is 20.
I20260709 13:03:47.956620  7750 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.242136  7750 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:47.959647  7753 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:47.986505  7736 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.243083  1637 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:48.131748  1641 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 16: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:48.132961  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:48.244167  1636 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 15: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:48.244650  7736 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:48.129065  6990 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.245045  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:48.245672  1636 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:48.245416  6990 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.134943  7599 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12.
I20260709 13:03:48.132076  1714 rpcz_store.cc:276] 0709 13:03:46.881350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.881491 (+   141us) service_pool.cc:224] Handling call
0709 13:03:47.794454 (+912963us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:47.884016 (+ 89562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.970203 (+ 86187us) spinlock_profiling.cc:243] Waited 897 ms on lock 0x7b5400474850. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:47.970441 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2061975808}
I20260709 13:03:48.135829  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:48.247558  1565 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:48.136359  7755 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:48.139137  7394 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.130959  7710 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:48.144214  7497 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:48.144959  7635 consensus_meta.cc:287] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.861s	user 0.011s	sys 0.000s
I20260709 13:03:48.145531  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1706) took 1211 ms. Trace:
I20260709 13:03:48.163244  7734 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:48.146868  2305 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 16: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:48.118806  7733 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:48.149478  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:47.944049  7726 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:03:48.150823  1711 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:48.152117  1712 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:48.152063  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:48.125376  7749 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:48.122705  7696 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:48.151863  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 1220) took 1179 ms. Trace:
I20260709 13:03:48.064674  7385 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:48.157286  1716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:03:48.160189  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35222 (request call id 1068) took 1187 ms. Trace:
I20260709 13:03:48.162711  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 3045) took 1302 ms. Trace:
I20260709 13:03:48.269985  7545 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:48.280452  1643 rpcz_store.cc:276] 0709 13:03:46.852030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.852176 (+   146us) service_pool.cc:224] Handling call
0709 13:03:48.153910 (+1301734us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400195fd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:48.154011 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2564715264}
I20260709 13:03:48.282007  1563 rpcz_store.cc:276] 0709 13:03:46.972825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.973117 (+   292us) service_pool.cc:224] Handling call
0709 13:03:48.154414 (+1181297us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b54001fed54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:48.160130 (+  5716us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2096357760}
I20260709 13:03:48.129478  6959 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.283571  1563 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:03:48.283319  6959 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.147864  7714 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.169523  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:48.168681  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:48.285851  1560 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:48.285960  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2579) took 1081 ms. Trace:
I20260709 13:03:48.285634  6959 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:48.172042  7503 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:48.286415  1718 rpcz_store.cc:276] 0709 13:03:47.204461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.204615 (+   154us) service_pool.cc:224] Handling call
0709 13:03:48.285818 (+1081203us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b54003dbdd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:48.285913 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":739279104}
I20260709 13:03:48.172844  7768 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.170549  1567 raft_consensus.cc:2393] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:48.176196  7576 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:48.156896  7637 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:48.180457  7751 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:48.289233  7751 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:48.181854  7674 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:48.183249  7550 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:48.184494  7638 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:48.290091  7768 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:48.185776  7676 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:48.169538  7139 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:48.186733  7168 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:48.170305  7729 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:48.196753  1681 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:48.291643  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 94987.8us
I20260709 13:03:48.292924  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:48.172873  7735 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:48.211349  7737 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.312906  1643 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:48.025658  7763 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:48.165181  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:48.224334  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.221077  1710 rpcz_store.cc:276] 0709 13:03:47.040984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.052363 (+ 11379us) service_pool.cc:224] Handling call
0709 13:03:47.052737 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:47.061197 (+  8460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:47.061229 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:47.061748 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:03:47.450641 (+388893us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:47.450697 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:47.450790 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:48.108690 (+657900us) raft_consensus.cc:1685] finished
0709 13:03:48.108720 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:48.108913 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1740,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":124,"thread_start_us":388417,"threads_started":1,"wal-append.queue_time_us":418334}
I20260709 13:03:48.227104  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:48.314678  1564 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:48.231340  1566 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:03:48.227409  7547 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:48.262472  1565 raft_consensus.cc:1240] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:03:48.314800  1532 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:48.145542  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:48.281929  1713 rpcz_store.cc:276] 0709 13:03:46.972527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.044593 (+ 72066us) service_pool.cc:224] Handling call
0709 13:03:48.151655 (+1107062us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400337cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:48.151802 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2392355712}
I20260709 13:03:48.285605  7714 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.151163  7739 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:48.169713  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:48.232151  7404 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:48.188222  7711 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.204787  7604 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:48.295372  7674 raft_consensus.cc:2804] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:48.295866  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2639) took 1101 ms. Trace:
I20260709 13:03:48.296273  7696 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:48.296674  2298 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:03:48.291728  2304 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1705) took 1357 ms (client timeout 1703 ms). Trace:
I20260709 13:03:48.299378  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:48.279513  7753 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:48.286290  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 117726us
I20260709 13:03:48.300735  2303 rpcz_store.cc:276] 0709 13:03:46.934220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.943028 (+  8808us) service_pool.cc:224] Handling call
0709 13:03:48.145269 (+1202241us) spinlock_profiling.cc:243] Waited 938 ms on lock 0x7b5400710f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:48.145440 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2157792128}
I20260709 13:03:48.352221  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:48.307309  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:48.308741  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 156744us
I20260709 13:03:48.353814  2301 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:48.301298  1681 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:48.316145  1682 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:48.317133  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:48.318408  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 366999us
I20260709 13:03:48.152762  7757 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:48.320293  1638 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:48.327070  1639 raft_consensus.cc:2410] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:03:48.327518  1568 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:48.317065  7637 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:48.332036  1560 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:48.321400  7717 raft_consensus.cc:2764] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:03:48.371119  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:48.300407  7755 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:48.301347  1716 raft_consensus.cc:606] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Received request to transfer leadership to TS 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:48.338455  1567 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:48.334242  1569 rpcz_store.cc:276] 0709 13:03:47.194741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.195281 (+   540us) service_pool.cc:224] Handling call
0709 13:03:48.295620 (+1100339us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54002981d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:48.295795 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":739592448}
I20260709 13:03:48.373510  1716 raft_consensus.cc:614] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:03:48.338171  2304 rpcz_store.cc:269] 0709 13:03:46.933899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:46.936316 (+  2417us) service_pool.cc:224] Handling call
0709 13:03:48.291515 (+1355199us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002a4254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:48.291644 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2701140224}
I20260709 13:03:48.344012  7139 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:48.355559  1714 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:48.358356  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:48.381458  1565 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:48.331851  7774 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.362282  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:48.330281  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 165178us
I20260709 13:03:48.385617  1564 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:48.317418  7292 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:03:48.338044  7599 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:48.317253  7361 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:03:48.331630  7547 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:48.390251  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:48.420085  1567 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 15: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:48.391564  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 76707.9us
I20260709 13:03:48.393332  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:48.420892  1607 leader_election.cc:400] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 19. Current term is 20.
I20260709 13:03:48.421211  1607 leader_election.cc:403] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:48.421773  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:48.422044  1565 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
W20260709 13:03:48.404773  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 103567us
I20260709 13:03:48.423031  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:48.423280  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:48.405237  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.369771  7615 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:48.403717  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:48.320379  7737 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:48.319530  7635 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:48.396155  2269 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:48.429178  2268 leader_election.cc:400] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
I20260709 13:03:48.418579  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:48.402886  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:48.419255  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.430648  2303 raft_consensus.cc:2368] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:48.430721  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:48.424077  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:48.418344  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:48.318142  7518 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.425064  1644 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:03:48.338394  7770 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:48.407936  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:48.423430  1681 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:48.418294  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:48.407320  1562 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:03:48.382552  7711 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:48.445032  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:48.445319  1638 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: [13.12-14.13]
I20260709 13:03:48.445801  2305 raft_consensus.cc:2368] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:48.369524  7674 raft_consensus.cc:697] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:48.429651  2268 leader_election.cc:403] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:03:48.438936  1533 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:48.439306  1711 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:48.446872  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:48.450081  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:48.582228  2303 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:03:48.450872  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:48.449277  1564 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:03:48.582942  1717 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:48.583566  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:48.584811  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:48.585083  7763 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:48.585975  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:48.586140  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:48.406342  7599 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.454298  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:48.587430  1636 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:48.455279  1681 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:48.457448  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:48.590649  1717 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:48.590829  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:48.460222  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:48.569906  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:48.459971  7615 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:48.419145  7735 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:03:48.570372  7500 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13.
W20260709 13:03:48.570863  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 82937.6us
I20260709 13:03:48.571527  7737 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:48.495488  7781 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:03:48.515190  7547 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:48.572016  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 110437us
I20260709 13:03:48.573174  7385 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:48.574141  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:48.425549  7744 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:48.425777  7676 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:48.494588  7755 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.575467  7404 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:48.471310  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:48.621506  1567 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:03:48.459971  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:48.429724  7717 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:48.576249  1563 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:48.497918  7637 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.576689  1709 raft_consensus.cc:2410] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:03:48.577085  7497 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:48.520617  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:48.428390  7727 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:48.485668  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:48.629799  2301 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:48.461163  7753 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:03:48.634841  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:48.635725  1561 raft_consensus.cc:2393] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:48.627184  7744 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:48.638443  1566 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:03:48.637804  7744 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:48.578044  7674 consensus_queue.cc:237] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:48.407727  7774 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.578903  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:48.579828  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:48.457171  7568 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:48.402837  7399 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:03:48.424755  6990 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:48.494881  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:48.460542  7361 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:03:48.452224  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:48.675058  2299 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:48.425803  7726 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:48.586762  1716 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:48.591519  1635 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:48.592088  1710 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:48.593921  1534 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:03:48.594058  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 138855us
I20260709 13:03:48.570667  7635 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:48.679488  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 85656.1us
I20260709 13:03:48.680773  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:48.595027  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:48.597307  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:48.597656  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 158820us
I20260709 13:03:48.578032  7734 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:48.682157  1564 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:48.599272  7486 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:48.682621  1533 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:48.683740  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 88836.6us
I20260709 13:03:48.587092  7599 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:48.604879  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:48.605420  7292 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:48.579248  7770 raft_consensus.cc:697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:48.685711  7292 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.686160  7292 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:48.603981  1565 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:48.687286  7710 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.687813  7710 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:48.606451  7702 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:48.606515  7751 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:48.605834  7615 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:48.608237  7403 consensus_queue.cc:1059] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:48.608879  7729 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:48.609993  7714 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:48.612776  1714 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:03:48.612591  7764 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:48.610821  7757 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:48.592931  7772 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:48.615619  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:48.696898  7772 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.615804  7518 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:48.686405  7770 consensus_queue.cc:237] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.616809  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:48.617241  1609 leader_election.cc:400] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:48.618122  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:48.618136  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 180575us
I20260709 13:03:48.676796  7568 raft_consensus.cc:697] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:48.707767  7772 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:48.707508  7568 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.435987  7681 raft_consensus.cc:3055] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:48.708570  7681 raft_consensus.cc:740] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:48.638835  1566 pending_rounds.cc:85] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 12
I20260709 13:03:48.709686  1566 pending_rounds.cc:107] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:03:48.710124  1566 raft_consensus.cc:2889] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:48.645327  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:48.653318  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:48.618054  7749 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.626123  7736 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:48.665810  2305 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:03:48.628355  7696 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.630956  7497 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.670217  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:48.627027  7676 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:48.743857  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:48.744616  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 74499us
I20260709 13:03:48.691804  2269 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:48.745329  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:48.745555  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 53839.9us
I20260709 13:03:48.677193  7545 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.695665  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:03:48.619473  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 45404.6us
I20260709 13:03:48.698958  1609 leader_election.cc:403] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:03:48.701393  1642 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:48.705472  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:48.627440  1636 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:03:48.683382  7735 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:48.629714  7717 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:48.755759  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 60189us
I20260709 13:03:48.683305  7604 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:48.719216  1639 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:48.729980  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.760427  1683 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:48.755671  7518 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:48.696085  7702 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.691726  7734 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:48.722745  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:48.690550  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:48.691592  7785 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:48.748674  2268 leader_election.cc:400] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 11 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 11. Current term is 12.
W20260709 13:03:48.752593  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 135508us
I20260709 13:03:48.683264  7711 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:48.762529  7486 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:48.746188  7753 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 19. Current term is 20.
I20260709 13:03:48.752532  7726 consensus_queue.cc:1059] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:03:48.769048  7518 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:48.778842  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:48.731501  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:48.710194  7751 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:48.764776  7743 consensus_meta.cc:287] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.004s
I20260709 13:03:48.755692  7739 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:48.791538  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 101044us
I20260709 13:03:48.792948  1609 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:48.793937  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 45481.2us
I20260709 13:03:48.796131  1639 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:48.760366  7767 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:48.810129  2268 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:48.740106  7749 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.709482  7681 consensus_queue.cc:260] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.801296  1716 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:03:48.756387  7545 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.800253  7486 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:48.781203  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 96065us
I20260709 13:03:48.879964  2268 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:48.803403  7518 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:48.808090  7735 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:48.800117  7711 raft_consensus.cc:697] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:48.797789  7734 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:48.815621  7361 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:48.822242  2299 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:48.743870  7696 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.766135  7604 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:48.834889  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 56099us
I20260709 13:03:48.779558  7787 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:48.813828  1609 leader_election.cc:400] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:48.939781  1609 leader_election.cc:403] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:03:48.939146  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 44308.8us
W20260709 13:03:48.882386  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 53517.7us
I20260709 13:03:48.880681  7292 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.892354  7774 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:48.812157  7743 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:48.775350  7599 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.800719  7710 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:48.896294  1641 raft_consensus.cc:2393] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 19: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:48.813270  7739 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:48.789285  7497 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:48.819099  7764 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:48.900226  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 139875us
I20260709 13:03:48.881765  7736 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:48.881376  7726 raft_consensus.cc:2764] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:03:48.893998  7760 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:48.898823  7361 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:48.948853  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 135187us
I20260709 13:03:48.943759  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:48.955260  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3141) took 1090 ms. Trace:
I20260709 13:03:48.959496  7486 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:48.964365  7711 consensus_queue.cc:237] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.119297  2299 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:48.965170  7800 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:03:49.226289  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:48.965639  7734 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:48.971484  7676 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:48.973423  7751 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:48.974552  7789 log.cc:927] Time spent T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
W20260709 13:03:48.979960  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:48.942628  7787 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:49.006623  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:49.236088  7787 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:49.236840  7518 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.239369  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:49.240281  2296 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 18: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:03:49.242281  1710 consensus_meta.cc:287] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.565s	user 0.006s	sys 0.003s
I20260709 13:03:49.093402  1607 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:48.974587  7702 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:49.095319  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 151645us
I20260709 13:03:49.105934  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:49.275419  1718 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:03:49.105481  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:49.276613  1565 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:03:49.277064  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:49.105063  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:49.095353  1713 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:49.277776  1711 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:48.997864  7739 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:49.279186  2268 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:49.095072  1562 rpcz_store.cc:276] 0709 13:03:47.865365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.865606 (+   241us) service_pool.cc:224] Handling call
0709 13:03:47.967509 (+101903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.443047 (+475538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.649991 (+206944us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b54002aab54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:48.954978 (+304987us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54002aab50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:48.955169 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244055168}
I20260709 13:03:49.227144  1635 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:49.108528  7361 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:49.280699  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:49.282176  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:49.284224  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:49.285622  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:49.286412  1714 raft_consensus.cc:2368] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
W20260709 13:03:49.273968  7776 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.013s	sys 0.027s
I20260709 13:03:49.287621  7803 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.290s
W20260709 13:03:49.288312  7776 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.013s	sys 0.027s
I20260709 13:03:49.288807  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:49.289469  1564 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:49.290508  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:49.291824  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:49.267822  1710 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
W20260709 13:03:49.267534  1642 consensus_meta.cc:287] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.515s	user 0.010s	sys 0.001s
I20260709 13:03:49.294624  1642 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:49.275677  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:49.279229  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:49.101768  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1513) took 2125 ms. Trace:
I20260709 13:03:49.268266  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:49.295289  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 202647us
I20260709 13:03:49.296752  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:49.295077  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:49.350329  1637 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.538s	user 0.009s	sys 0.001s
I20260709 13:03:49.295892  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:49.226797  7638 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:49.351593  1637 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:03:49.309257  1609 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:49.298074  2298 rpcz_store.cc:276] 0709 13:03:46.976367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:47.052043 (+ 75676us) service_pool.cc:224] Handling call
0709 13:03:48.296324 (+1244281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.338052 (+ 41728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.101551 (+763499us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54002a4250. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:49.101698 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2861095168}
I20260709 13:03:49.357417  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:49.352336  1560 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6711207c14044518a57e7c8914ac0091"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:03:49.358521  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:49.374395  1560 raft_consensus.cc:606] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:49.338188  1561 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:49.299914  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:49.375074  1560 raft_consensus.cc:614] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:49.227149  7735 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.268374  7518 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:49.297132  7753 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:49.296095  7794 raft_consensus.cc:2764] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:03:49.296751  7292 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:49.383788  7292 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.386588  7292 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:49.300028  7801 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:03:49.304235  7785 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:49.355551  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 76473us
I20260709 13:03:49.357827  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:49.391321  1563 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:49.301514  7743 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:49.297948  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:49.275311  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 181985us
I20260709 13:03:49.390337  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:49.394358  1710 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
W20260709 13:03:49.304131  7766 log.cc:927] Time spent T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:03:49.291635  7676 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:49.302946  7599 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:49.302207  7717 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:49.405823  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 96655.7us
I20260709 13:03:49.379398  7500 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:49.443554  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:49.445921  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:49.400733  7638 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:49.412554  1609 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:03:49.477089  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 83449.7us
I20260709 13:03:49.482240  1642 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:49.462931  1638 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.047s	sys 0.041s
W20260709 13:03:49.491194  1638 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.047s	sys 0.041s
I20260709 13:03:49.492408  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:49.491091  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:49.403386  7676 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:49.493130  2298 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:49.463583  7702 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:49.395795  7735 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:49.499531  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 87053.4us
I20260709 13:03:49.501713  2300 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:03:49.503439  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:49.504881  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:49.506736  2304 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:03:49.507485  2296 raft_consensus.cc:2410] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:03:49.507351  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:49.508229  2297 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:03:49.508981  2296 raft_consensus.cc:1240] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:49.510305  1609 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:49.510756  2296 raft_consensus.cc:1240] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 13:03:49.519497  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:49.520659  2299 raft_consensus.cc:2393] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:49.547096  1642 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:49.577759  2268 leader_election.cc:400] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14.
I20260709 13:03:49.578258  2268 leader_election.cc:403] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:03:49.497303  7676 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:49.506433  7751 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:49.506222  7749 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:49.587630  2268 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:49.588131  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:49.585857  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:49.611249  1638 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:49.612581  7764 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.615540  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:49.616695  1642 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:49.581496  7778 raft_consensus.cc:2804] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:49.622385  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:49.623090  1560 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:49.588984  7500 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:49.608021  7751 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:49.608489  7702 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:49.636344  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:49.615334  7604 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:49.619771  7520 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:49.632050  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:49.645778  1533 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.503s (SENT) [suppressed 4 similar messages]
I20260709 13:03:49.649441  1564 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:49.650413  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:49.659876  1560 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:49.662011  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:49.662854  1642 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:49.629814  7292 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:49.671957  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:49.614956  7764 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:49.635030  7778 raft_consensus.cc:697] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:49.635843  7751 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:49.675437  1560 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:49.674834  2268 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:49.696969  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:49.638831  7726 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.648078  7755 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:49.639811  7599 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:49.697886  1561 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:49.719311  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:49.723294  1564 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:49.729274  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:49.730358  1560 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 15: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:49.736716  1561 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
W20260709 13:03:49.728390  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 259910us
I20260709 13:03:49.619213  7638 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.673280  7520 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:49.713264  7799 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.357s	user 0.001s	sys 0.000s
I20260709 13:03:49.749583  1564 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:49.721632  1642 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:49.740417  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:49.750952  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:49.665387  7702 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.799387  7292 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.741042  7599 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.800791  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:49.801339  7809 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
W20260709 13:03:49.802399  7814 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 13:03:49.803215  7806 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:03:49.804257  7735 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:49.819334  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:49.827761  7520 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.953513  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:49.830734  7813 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.954792  7813 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.957232  7813 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:49.837103  7764 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:49.958107  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:49.958738  7764 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:49.959417  7764 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:49.838940  7815 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:03:49.852463  7497 consensus_meta.cc:287] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.627s	user 0.008s	sys 0.001s
I20260709 13:03:49.963855  7497 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:49.966432  7497 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:49.727931  7726 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.969389  7726 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:49.859476  7818 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.970247  7818 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.973084  7818 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:49.747582  7568 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:49.973723  7568 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.976065  7568 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:49.748971  7749 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:49.976826  7749 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.738137  7755 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.981829  7755 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:49.872655  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1399) took 1216 ms. Trace:
I20260709 13:03:49.982538  1644 rpcz_store.cc:276] 0709 13:03:48.656867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.796184 (+139317us) service_pool.cc:224] Handling call
0709 13:03:48.796428 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.882502 (+ 86074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.882537 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:48.882576 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:48.882604 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.872348 (+989744us) spinlock_profiling.cc:243] Waited 85.9 ms on lock 0x7b54000d07d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:49.872400 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:49.872559 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197511936}
W20260709 13:03:49.873405  7787 consensus_meta.cc:287] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.000s
I20260709 13:03:49.984673  7787 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:49.877082  7767 consensus_meta.cc:287] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.050s	user 0.004s	sys 0.006s
I20260709 13:03:49.987192  7767 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:49.987975  7767 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:49.988520  7767 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.858604  7778 consensus_queue.cc:237] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.878095  7715 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:49.882444  1534 leader_election.cc:336] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 2.265s (SENT)
I20260709 13:03:49.995402  1534 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:49.996479  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 114077us
I20260709 13:03:49.997028  1643 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:03:49.884187  7803 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:49.998965  7733 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:49.999567  7733 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:50.003791  7729 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:50.006067  7749 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.007716  7787 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:49.917651  7676 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:49.918816  7739 consensus_meta.cc:287] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.000s
I20260709 13:03:50.011061  7739 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.013568  7739 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:49.925273  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:50.014312  1564 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:03:50.014323  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:49.727289  7604 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:50.015501  7604 raft_consensus.cc:697] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:50.016515  7604 consensus_queue.cc:237] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.711022  7635 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:49.927017  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:50.018105  1560 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:50.018016  7635 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:49.928144  2298 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:49.940835  1642 raft_consensus.cc:2393] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 15: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:50.020239  7635 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.021235  1642 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 13:03:50.022373  1642 raft_consensus.cc:3060] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:49.958873  1563 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
W20260709 13:03:49.796923  7810 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 13:03:50.023480  7812 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:49.967069  1716 raft_consensus.cc:2368] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:49.967198  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:49.974005  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:50.024523  1561 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:50.024814  1568 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:49.883831  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1072) took 1523 ms. Trace:
I20260709 13:03:50.025498  1566 rpcz_store.cc:276] 0709 13:03:48.360056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.360228 (+   172us) service_pool.cc:224] Handling call
0709 13:03:48.360515 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:48.710771 (+350256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.883423 (+1172652us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54002986d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:49.883762 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":638353792}
W20260709 13:03:49.857563  7361 consensus_meta.cc:287] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.576s	user 0.010s	sys 0.000s
I20260709 13:03:50.026583  7361 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.028731  7361 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.030565  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:50.030514  7733 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:50.031343  1566 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:50.032280  7764 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:50.034482  7676 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.029886  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:49.990818  7767 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.042925  7785 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:50.044005  7785 consensus_queue.cc:1059] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:50.044514  7733 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:50.045147  7785 raft_consensus.cc:3055] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:50.045643  7785 raft_consensus.cc:740] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:50.046715  7785 consensus_queue.cc:260] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.047636  1561 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:50.063196  1566 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
W20260709 13:03:50.066254  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 420527us
W20260709 13:03:50.066568  7615 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.075s	sys 0.114s
W20260709 13:03:50.067562  7615 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.075s	sys 0.114s
I20260709 13:03:50.071938  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.072691  1642 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
W20260709 13:03:50.073336  7782 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 1.504s	user 0.002s	sys 0.011s
I20260709 13:03:50.075783  7743 raft_consensus.cc:3055] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:50.076186  7743 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:50.076669  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.077108  7743 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:50.080346  7811 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.407s	user 0.001s	sys 0.000s
I20260709 13:03:49.939365  7753 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:50.085069  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 131627us
I20260709 13:03:50.085450  7753 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.088230  7753 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.089960  7774 consensus_queue.cc:1059] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:50.090273  7599 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:49.942862  7292 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:50.091146  7803 raft_consensus.cc:697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:50.092078  7803 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:50.094825  7500 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14.
I20260709 13:03:50.095152  7403 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:50.095731  7403 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:50.097409  1713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2656) took 1502 ms (client timeout 1832 ms). Trace:
W20260709 13:03:50.097940  1713 rpcz_store.cc:269] 0709 13:03:48.595432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.615356 (+ 19924us) service_pool.cc:224] Handling call
0709 13:03:50.097159 (+1481803us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54001c0f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:50.097337 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749346176}
I20260709 13:03:50.099527  7786 raft_consensus.cc:3055] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:50.100206  7786 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:50.101163  7702 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.103216  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3060) took 1110 ms. Trace:
I20260709 13:03:50.101209  7786 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:50.105556  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:50.106122  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.106338  1635 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:50.109251  7772 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.109786  7772 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.099635  7743 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:50.066988  7785 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:50.103577  1636 rpcz_store.cc:276] 0709 13:03:48.992965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.995429 (+  2464us) service_pool.cc:224] Handling call
0709 13:03:48.995877 (+   448us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:48.996398 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.996443 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:49.002210 (+  5767us) log.cc:844] Serialized 52 byte log entry
0709 13:03:49.406783 (+404573us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:49.694419 (+287636us) log.cc:844] Serialized 14 byte log entry
0709 13:03:49.695106 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:49.695208 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:50.102978 (+407770us) raft_consensus.cc:1685] finished
0709 13:03:50.103002 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.103156 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":456545,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":2443,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":18498,"thread_start_us":386283,"threads_started":2,"wal-append.queue_time_us":483963}
I20260709 13:03:50.094714  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:50.115115  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:50.098610  7403 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.115200  1566 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:03:50.115695  1636 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:50.116395  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2638) took 2188 ms. Trace:
I20260709 13:03:50.118786  1635 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:50.117075  1640 rpcz_store.cc:276] 0709 13:03:47.927550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.209356 (+281806us) service_pool.cc:224] Handling call
0709 13:03:48.209722 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.210216 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:03:48.210601 (+   385us) log.cc:844] Serialized 14 byte log entry
0709 13:03:48.335664 (+125063us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:48.336103 (+   439us) log.cc:844] Serialized 26 byte log entry
0709 13:03:48.336219 (+   116us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:48.336250 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.336313 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:50.072713 (+1736400us) spinlock_profiling.cc:243] Waited 7.66 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:50.105304 (+ 32591us) raft_consensus.cc:1685] finished
0709 13:03:50.116193 (+ 10889us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.116339 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":973,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":193,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":5345,"spinlock_wait_cycles":17611008,"thread_start_us":124554,"threads_started":1,"wal-append.queue_time_us":239771}
I20260709 13:03:50.130785  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.111915  1533 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:50.132319  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.133086  1635 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:03:50.092967  7292 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:50.133602  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1227) took 1739 ms. Trace:
I20260709 13:03:50.143244  1717 rpcz_store.cc:276] 0709 13:03:48.394544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.595919 (+201375us) service_pool.cc:224] Handling call
0709 13:03:48.596267 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:48.676490 (+ 80223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:48.677836 (+  1346us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:48.680232 (+  2396us) log.cc:844] Serialized 26 byte log entry
0709 13:03:48.774591 (+ 94359us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:48.774651 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:48.774750 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:50.133384 (+1358634us) raft_consensus.cc:1685] finished
0709 13:03:50.133409 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.133534 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42911,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":105,"thread_start_us":137775,"threads_started":2,"wal-append.queue_time_us":254452}
I20260709 13:03:50.140676  2301 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:50.158285  1636 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:03:50.138969  7786 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:50.144088  7403 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:50.188103  1640 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:03:50.182658  1682 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:50.189324  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:50.192224  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.194662  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.196154  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:50.205082  1640 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:03:50.206504  1636 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:50.176215  7813 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:50.240083  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:50.212836  1637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2645) took 1790 ms (client timeout 2206 ms). Trace:
I20260709 13:03:50.213128  1642 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:03:50.241350  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:50.192251  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:50.215190  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.243377  2296 raft_consensus.cc:2410] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:03:50.217244  7385 raft_consensus.cc:3055] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:50.222409  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.224576  7751 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.227316  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.240782  2300 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
W20260709 13:03:50.229277  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 1389) took 1778 ms (client timeout 1518 ms). Trace:
W20260709 13:03:50.241284  1637 rpcz_store.cc:269] 0709 13:03:48.422439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.427958 (+  5519us) service_pool.cc:224] Handling call
0709 13:03:50.212649 (+1784691us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b540032c8d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:50.212780 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514550144}
W20260709 13:03:50.246281  1643 rpcz_store.cc:269] 0709 13:03:48.451375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.455005 (+  3630us) service_pool.cc:224] Handling call
0709 13:03:48.731271 (+276266us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b58001f4798. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:03:50.229112 (+1497841us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400195fd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:50.229212 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2736791296}
I20260709 13:03:50.231670  2298 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:50.233676  1640 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:50.236963  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1722) took 1231 ms. Trace:
W20260709 13:03:50.213727  1641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2644) took 1791 ms (client timeout 1503 ms). Trace:
I20260709 13:03:50.246282  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:50.248891  1641 rpcz_store.cc:269] 0709 13:03:48.422176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.428056 (+  5880us) service_pool.cc:224] Handling call
0709 13:03:50.213473 (+1785417us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b5400124b54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:50.213649 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":628720000}
I20260709 13:03:50.242544  1566 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 15: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:50.214397  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 1398) took 1583 ms. Trace:
I20260709 13:03:50.251876  1639 rpcz_store.cc:276] 0709 13:03:48.630768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:48.644981 (+ 14213us) service_pool.cc:224] Handling call
0709 13:03:50.214243 (+1569262us) spinlock_profiling.cc:243] Waited 8.09 ms on lock 0x7b54000d16d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:50.214345 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18608512}
I20260709 13:03:50.191222  7403 raft_consensus.cc:697] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:50.254340  7403 consensus_queue.cc:237] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:50.248409  2303 rpcz_store.cc:276] 0709 13:03:49.005879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.006315 (+   436us) service_pool.cc:224] Handling call
0709 13:03:49.011648 (+  5333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:49.124736 (+113088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:49.124782 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:49.226693 (+101911us) log.cc:844] Serialized 26 byte log entry
0709 13:03:49.668319 (+441626us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:49.668359 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.236538 (+568179us) spinlock_profiling.cc:243] Waited 84.8 ms on lock 0x7b54004cf754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:50.236601 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:50.236650 (+    49us) raft_consensus.cc:1685] finished
0709 13:03:50.236663 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:50.236899 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27792,"raft.queue_time_us":28410,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":20582,"spinlock_wait_cycles":194996992,"thread_start_us":412400,"threads_started":1,"wal-append.queue_time_us":523396}
I20260709 13:03:50.190783  7702 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.188344  7714 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:50.259771  2300 raft_consensus.cc:2393] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:03:50.260185  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 67998.2us
I20260709 13:03:50.240613  7794 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:50.268587  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.245204  7385 raft_consensus.cc:740] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:50.269996  7696 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.271299  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.283826  2299 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:03:50.272189  7815 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:50.274791  1644 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:50.278244  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.277184  1682 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:50.281282  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.282845  2302 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 16: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:03:50.285386  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 241707us
I20260709 13:03:50.293239  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:50.295637  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:50.308943  1534 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:50.308828  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:50.315775  1684 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.103s	sys 0.119s
I20260709 13:03:50.311764  1609 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:50.316380  1684 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.103s	sys 0.119s
I20260709 13:03:50.316931  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:50.278754  7751 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.322189  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.322382  1609 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:50.326452  2304 raft_consensus.cc:2368] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18.
I20260709 13:03:50.286957  7815 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:50.327605  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.328899  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:50.329670  1717 raft_consensus.cc:2393] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:50.329788  1609 leader_election.cc:400] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18.
I20260709 13:03:50.330104  1609 leader_election.cc:403] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:50.331627  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:50.331852  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:50.289448  7794 raft_consensus.cc:697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:50.334100  1713 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:50.255887  7813 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.309062  7757 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:50.337160  1718 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:03:50.352069  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:50.308645  7711 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:50.306047  7772 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.309561  7696 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:50.282442  7702 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.332316  7815 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:50.357646  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:50.401583  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:50.360447  1709 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.968s	user 0.013s	sys 0.009s
I20260709 13:03:50.365610  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:50.402953  1710 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:50.366439  1566 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:03:50.368670  1683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.102s	sys 0.187s
W20260709 13:03:50.404443  1683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.102s	sys 0.187s
W20260709 13:03:50.404826  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 647413us
W20260709 13:03:50.405098  1709 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.013s	sys 0.009s
I20260709 13:03:50.405745  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:50.368891  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 416020us
I20260709 13:03:50.320020  7500 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:50.383193  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:50.282312  7714 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:50.330351  7824 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.330634  7753 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:50.333652  7615 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:50.414907  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:50.308861  7715 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 13:03:50.420373  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:50.334687  7825 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:50.334007  7767 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18.
I20260709 13:03:50.458285  1566 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:03:50.424863  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.355566  7818 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:50.425922  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.427816  1683 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:50.460041  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:50.428622  7500 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.422999  7696 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:50.429442  7787 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:50.432394  7714 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:50.433377  1643 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:03:50.433881  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.434674  7755 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.445292  7615 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.436038  7394 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.436445  7769 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.439200  7794 consensus_queue.cc:237] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:50.442070  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:50.484417  1607 leader_election.cc:400] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:50.365319  7497 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:50.379858  7757 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:50.444501  1715 raft_consensus.cc:2410] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:03:50.449098  1710 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:50.458400  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:50.459733  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:50.460443  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:50.458493  7836 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:50.468916  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:50.519387  1563 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:03:50.461015  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:50.467505  1683 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:50.435501  2269 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:50.443668  7676 raft_consensus.cc:2764] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:03:50.426724  7753 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:50.531550  7520 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:50.531774  7753 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:50.531101  1532 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:50.475383  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 370798us
I20260709 13:03:50.336861  7385 consensus_queue.cc:260] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:50.535936  7832 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:03:50.462729  7599 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
W20260709 13:03:50.480983  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 336478us
W20260709 13:03:50.503887  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 191509us
I20260709 13:03:50.471189  7786 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:50.443185  7824 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:50.479295  7615 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:50.522006  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 86604.4us
W20260709 13:03:50.521523  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 60633.2us
I20260709 13:03:50.546691  7576 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:50.547262  7696 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.548184  7576 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:50.549996  7576 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:50.522006  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 54627.3us
I20260709 13:03:50.512558  7769 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.469666  7818 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.521665  7778 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:03:50.556928  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.525182  7830 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:50.465303  7772 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:03:50.521467  7757 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:50.561942  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:50.557804  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.563258  1710 raft_consensus.cc:2410] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:03:50.531078  7736 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:03:50.535771  7845 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:50.465499  7825 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.543920  7851 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:50.550416  7711 consensus_queue.cc:1175] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" responder_term: 14 status { last_received { term: 12 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: [13.13-13.13]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:50.514060  7767 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:50.478528  7755 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.601612  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.604743  7702 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.605311  7702 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.556406  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.513401  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:50.609479  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 96187.6us
I20260709 13:03:50.535287  7737 raft_consensus.cc:3055] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:50.611197  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:50.611488  7737 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:50.612963  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:50.613641  2297 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:03:50.605791  7702 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.487047  7394 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.617439  7394 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:50.589157  7736 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.619095  7736 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:50.578747  2269 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:50.580724  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:50.561424  7830 raft_consensus.cc:697] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:50.582420  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.583194  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:50.621819  1563 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:50.621984  7769 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:50.583899  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:50.584271  7714 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.627027  7394 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:50.627190  7714 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.630177  7714 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.550470  7576 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.578626  1533 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:50.630700  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2594) took 1348 ms. Trace:
W20260709 13:03:50.631207  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 52668us
I20260709 13:03:50.631136  1716 rpcz_store.cc:276] 0709 13:03:49.281904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.295622 (+ 13718us) service_pool.cc:224] Handling call
0709 13:03:50.630509 (+1334887us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54001487d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:50.630640 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407535360}
I20260709 13:03:50.588203  7803 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:50.632300  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:50.632817  7576 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:50.584846  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:50.633642  7778 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.633809  2298 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:50.634039  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1527) took 1133 ms. Trace:
I20260709 13:03:50.634208  7778 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.637247  7778 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.577857  1532 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:50.578267  7385 raft_consensus.cc:3060] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:50.620766  7824 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:50.621532  7830 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.583549  7818 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:50.623894  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 45219.7us
I20260709 13:03:50.624751  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.617537  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:50.625790  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.660823  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:50.661476  1635 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:03:50.658829  2303 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:50.630288  7702 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.632508  7803 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.634514  2301 rpcz_store.cc:276] 0709 13:03:49.501016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.621200 (+120184us) service_pool.cc:224] Handling call
0709 13:03:50.633882 (+1012682us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b5400642154. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:50.633986 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062506880}
I20260709 13:03:50.594072  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:50.596513  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.664597  2302 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:50.613700  7813 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:50.641248  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 63505.4us
I20260709 13:03:50.668924  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:50.523479  7497 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.649757  2298 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:03:50.580992  7681 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:50.654271  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:50.653151  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:50.653129  7547 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:50.688484  1716 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:50.662571  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:50.596966  7825 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.623566  7767 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:50.623742  7764 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:50.670085  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.670527  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:50.652864  7851 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:50.684721  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:50.697603  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.715502  1635 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:50.667093  7837 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:50.700001  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.704742  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.702560  2269 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:50.667500  7778 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:50.716298  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.699947  1681 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:50.718279  1532 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:50.721428  1641 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:50.723675  2298 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:50.712087  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:50.726625  2269 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:50.730700  1607 leader_election.cc:400] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:50.715773  7813 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:50.713424  7851 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:50.689666  7772 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:03:50.693277  7812 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:03:50.724730  7599 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:50.701776  7361 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:03:50.891093  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:50.891546  2300 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:50.739229  7849 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:50.712512  7717 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:03:50.660704  7737 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.790014  7696 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:50.705323  7836 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:03:50.722407  7824 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:50.716938  7751 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:50.730053  7764 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:50.698627  7702 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:50.697235  7818 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:03:50.814303  1607 leader_election.cc:403] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:03:50.931440  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:50.931921  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 205143us
W20260709 13:03:50.815774  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 113540us
I20260709 13:03:50.936132  1533 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:50.817654  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 83470us
I20260709 13:03:50.824151  2297 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:03:50.938040  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.827169  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:50.939558  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 172682us
I20260709 13:03:50.940716  2269 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:50.733088  7520 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 13:03:50.712986  7848 consensus_queue.cc:1059] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.098s
I20260709 13:03:50.825706  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:50.861064  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.881189  7497 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:50.830410  7821 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:50.883291  7743 consensus_meta.cc:287] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.765s	user 0.021s	sys 0.010s
I20260709 13:03:50.885968  2302 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:50.961400  2303 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: [16.13-16.13]
W20260709 13:03:50.778460  7774 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.029s	sys 0.031s
I20260709 13:03:50.738955  7547 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:50.905819  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:50.823951  7813 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:50.850242  7785 consensus_meta.cc:287] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.732s	user 0.013s	sys 0.000s
I20260709 13:03:50.907368  7825 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.920037  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:50.830372  7769 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:50.834059  7714 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:50.832108  7851 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:50.951282  1642 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
W20260709 13:03:50.932504  7816 log.cc:927] Time spent T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:03:50.956501  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.989576  2297 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:50.990562  2268 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:50.957468  2269 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:50.968305  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:50.931430  7702 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:50.975925  7853 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:03:50.980038  1713 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:50.981546  2301 raft_consensus.cc:2376] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14. Re-sending same reply.
I20260709 13:03:50.952901  7394 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:03:50.969019  7774 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.031s	sys 0.034s
W20260709 13:03:50.987074  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 268851us
I20260709 13:03:50.949755  7778 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:50.968809  7760 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:50.941715  7824 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:50.980679  7813 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:51.016593  7726 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:51.010579  7866 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:03:50.968528  7821 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:51.037420  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 101377us
I20260709 13:03:51.013736  7702 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:51.036922  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 102169us
I20260709 13:03:51.060809  1635 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:03:51.062999  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 72558.6us
I20260709 13:03:51.063941  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:51.066033  1635 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:51.017616  7518 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:51.073314  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 122534us
I20260709 13:03:51.024655  7394 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:51.091487  7867 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:51.092093  7867 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.092484  7867 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:51.096099  7868 log.cc:927] Time spent T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:03:51.096781  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:50.991291  7825 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:51.033336  7717 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:51.101783  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:51.101925  1681 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:51.107995  1710 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:51.010854  7714 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.057267  7760 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:51.070627  7751 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.147492  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2654) took 1521 ms. Trace:
I20260709 13:03:51.058851  7726 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.125190  1681 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:03:51.210307  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 108643us
W20260709 13:03:51.212453  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 154228us
I20260709 13:03:51.073061  7830 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
W20260709 13:03:51.229306  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 271979us
I20260709 13:03:51.110662  7764 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.211004  1638 rpcz_store.cc:276] 0709 13:03:49.626394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.885690 (+259296us) service_pool.cc:224] Handling call
0709 13:03:49.885993 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:50.320966 (+434973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.321014 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:50.366030 (+ 45016us) log.cc:844] Serialized 26 byte log entry
0709 13:03:50.643374 (+277344us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:50.643427 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:50.643518 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.147231 (+503713us) raft_consensus.cc:1685] finished
0709 13:03:51.147253 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.147422 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":911,"raft.queue_time_us":38602,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":2800,"thread_start_us":603711,"threads_started":2,"wal-append.queue_time_us":288295}
I20260709 13:03:51.237582  1710 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:51.238749  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:51.241438  1640 raft_consensus.cc:2376] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13. Re-sending same reply.
I20260709 13:03:51.126027  7576 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:51.244416  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:51.246465  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:51.247231  1560 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:03:51.249217  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:51.249461  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:51.250408  1565 raft_consensus.cc:2410] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:03:51.251461  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:51.252241  1565 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:03:51.253293  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:51.087472  7518 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.126253  7520 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:51.088979  7769 raft_consensus.cc:697] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:51.262036  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:51.221551  7836 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:51.228394  7837 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:51.265766  1567 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:03:51.233276  7545 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:51.267020  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:51.278663  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:51.254868  7576 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:51.279745  7848 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:51.280423  7576 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:51.281836  7848 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:51.239456  7638 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
W20260709 13:03:51.283219  7877 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:03:51.283147  7869 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:51.284291  7856 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.212s	user 0.000s	sys 0.000s
I20260709 13:03:51.282378  7848 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:51.284893  7869 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.270503  7520 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.289701  7743 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:51.290328  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:51.291055  7854 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:51.285153  2301 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:03:51.255510  7812 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.221662  7751 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.147599  7739 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:51.284184  7638 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:51.338476  7743 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:51.411650  7757 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.853s	user 0.005s	sys 0.009s
I20260709 13:03:51.268075  7803 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:51.326328  7385 consensus_meta.cc:287] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.653s	user 0.002s	sys 0.009s
W20260709 13:03:51.424244  7871 log.cc:927] Time spent T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 13:03:51.432634  1566 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 13:03:51.424522  7702 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:51.425221  7821 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:51.426643  7751 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:51.416744  7739 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.416901  7638 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:51.428217  1635 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:51.256302  7764 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.424643  7848 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:51.431758  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 209095us
I20260709 13:03:51.435216  1566 pending_rounds.cc:85] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 12
I20260709 13:03:51.450337  1566 pending_rounds.cc:107] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 13:03:51.450657  1566 raft_consensus.cc:2889] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:51.450492  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:51.295578  7836 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.421047  7743 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:51.443053  1683 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:51.448249  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:51.465982  1641 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:51.267501  7394 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:51.453020  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:51.474056  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:51.451970  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:51.416553  7812 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:51.444203  7638 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:51.449795  7739 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:51.493181  1636 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:51.492045  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:51.431172  7757 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.449496  7751 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:51.499751  7889 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:51.503309  7889 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:51.503789  7757 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:51.504525  7885 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:51.455940  7744 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:51.502666  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 59697.3us
I20260709 13:03:51.469640  7385 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:51.519253  1683 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:03:51.509099  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 383985us
I20260709 13:03:51.464777  7874 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:51.469923  7837 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:51.522861  1681 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:51.496779  7824 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:51.519754  7736 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.900s	user 0.012s	sys 0.000s
I20260709 13:03:51.520474  7825 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:51.502127  7518 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:51.523253  7753 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.988s	user 0.008s	sys 0.006s
I20260709 13:03:51.584669  7753 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:51.585700  7615 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.042s	user 0.004s	sys 0.008s
I20260709 13:03:51.586159  7615 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.586906  7753 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:51.587759  7753 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:51.588057  7764 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:51.469201  7769 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.555663  7849 raft_consensus.cc:3055] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:51.588918  7849 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:51.589751  1643 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:51.577512  7757 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:03:51.561036  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2664) took 1225 ms. Trace:
W20260709 13:03:51.592796  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 73628.2us
I20260709 13:03:51.592607  1711 rpcz_store.cc:276] 0709 13:03:50.335423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.335554 (+   131us) service_pool.cc:224] Handling call
0709 13:03:50.335871 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.598587 (+262716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.598615 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:50.598655 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:50.598681 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.560701 (+962020us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54003390d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:51.560765 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.560958 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603808000}
I20260709 13:03:51.594270  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:51.594820  1711 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:51.558085  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:51.595574  7825 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:51.595520  7576 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.597105  7821 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:51.597896  7576 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:51.503791  7889 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:51.599197  7863 raft_consensus.cc:3055] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:51.599635  7863 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:51.603188  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:51.603853  1567 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
W20260709 13:03:51.604765  7859 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.518s	user 0.002s	sys 0.000s
I20260709 13:03:51.523375  7744 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:51.605671  7815 consensus_meta.cc:287] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.178s	user 0.010s	sys 0.002s
I20260709 13:03:51.606106  7815 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:51.607270  7744 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:51.608371  7500 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:51.608738  7815 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:51.609781  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 1238) took 1081 ms. Trace:
I20260709 13:03:51.610363  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:51.611272  1718 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:03:51.612300  1718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2668) took 1230 ms (client timeout 1601 ms). Trace:
W20260709 13:03:51.612679  1718 rpcz_store.cc:269] 0709 13:03:50.382389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.382899 (+   510us) service_pool.cc:224] Handling call
0709 13:03:51.612120 (+1229221us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54000483d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:51.612246 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2769311488}
I20260709 13:03:51.613948  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:51.615276  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:51.575219  7875 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:03:51.582527  7885 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:51.523583  7714 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:51.559217  7874 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:51.576021  7837 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.732916  1639 raft_consensus.cc:2368] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 19. Current term is 20.
I20260709 13:03:51.588459  7753 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:51.596153  7825 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.616590  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:51.608418  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:51.618386  1718 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 11.
I20260709 13:03:51.742679  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:51.582305  7520 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:51.610140  1710 rpcz_store.cc:276] 0709 13:03:50.528749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.714645 (+185896us) service_pool.cc:224] Handling call
0709 13:03:51.609625 (+894980us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54001c1454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:51.609724 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354262400}
I20260709 13:03:51.744290  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:51.618804  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3083) took 1029 ms. Trace:
I20260709 13:03:51.745504  1644 rpcz_store.cc:276] 0709 13:03:50.589630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.589742 (+   112us) service_pool.cc:224] Handling call
0709 13:03:50.590064 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.600664 (+ 10600us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.600709 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:50.600766 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:50.600811 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.618598 (+1017787us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b5400126454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:51.618643 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.618752 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23771904}
I20260709 13:03:51.619769  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2670) took 1149 ms. Trace:
W20260709 13:03:51.552753  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 54035.1us
W20260709 13:03:51.631538  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 139582us
I20260709 13:03:51.746832  1711 rpcz_store.cc:276] 0709 13:03:50.470039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.594007 (+1123968us) service_pool.cc:224] Handling call
0709 13:03:51.619711 (+ 25704us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:51.748999  7757 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:51.752568  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:51.503144  7751 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:51.616948  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:51.721705  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:51.754231  1560 raft_consensus.cc:2376] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19. Re-sending same reply.
I20260709 13:03:51.723431  1681 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:51.741922  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:51.758680  1607 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:51.759877  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:51.760546  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 143695us
I20260709 13:03:51.761221  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:51.761444  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:51.763491  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:51.764526  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:51.764876  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:51.765367  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:51.765985  1711 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:51.766794  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:51.623311  7735 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.526424  7385 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:51.768352  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:51.771204  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:51.772410  7794 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.028s	sys 0.040s
I20260709 13:03:51.780927  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:51.781867  1636 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 18: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:51.784649  7884 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:51.788156  2268 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:51.789124  7879 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:51.789569  7879 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:51.789924  7879 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:51.792177  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1537) took 1061 ms. Trace:
I20260709 13:03:51.792596  2303 rpcz_store.cc:276] 0709 13:03:50.730935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.809589 (+ 78654us) service_pool.cc:224] Handling call
0709 13:03:50.961160 (+151571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:50.961896 (+   736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.791954 (+830058us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54002a4250. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:51.792115 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347761920}
I20260709 13:03:51.793864  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:51.794499  2303 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:51.795799  7755 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:51.796849  2305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1726) took 2773 ms. Trace:
I20260709 13:03:51.797277  2305 rpcz_store.cc:276] 0709 13:03:49.023251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.092471 (+ 69220us) service_pool.cc:224] Handling call
0709 13:03:49.092814 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.602762 (+1509948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.602791 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:50.602830 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:50.602857 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.796597 (+1193740us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b5400168354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:51.796645 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.796764 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3472464128}
I20260709 13:03:51.800752  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:51.802817  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1084) took 2149 ms. Trace:
I20260709 13:03:51.803283  1562 rpcz_store.cc:276] 0709 13:03:49.653262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.653388 (+   126us) service_pool.cc:224] Handling call
0709 13:03:49.653764 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.654157 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:49.945515 (+291358us) log.cc:844] Serialized 14 byte log entry
0709 13:03:51.802280 (+1856765us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b18004e6f40. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb1455 00007fd7a1c99060 00007fd7a1cab716 00007fd7a1cbc287
0709 13:03:51.802396 (+   116us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:51.802451 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:51.802505 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.802604 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.802731 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":520775,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":2826,"raft.queue_time_us":147756,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":9723,"spinlock_wait_cycles":2392320,"thread_start_us":672408,"threads_started":3,"wal-append.queue_time_us":502736,"wal-append.run_cpu_time_us":1195,"wal-append.run_wall_time_us":1002674}
I20260709 13:03:51.798669  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:51.854810  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 66921us
I20260709 13:03:51.859365  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:51.959316  1638 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:51.959429  7897 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:51.605628  7743 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.960343  7764 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:03:51.961200  7786 consensus_meta.cc:287] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.419s	user 0.001s	sys 0.014s
I20260709 13:03:51.961669  7786 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:51.961988  7743 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:51.742046  7885 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:51.962821  7874 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:51.907124  7887 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.963272  7885 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.964507  7726 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:51.757467  7825 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.955323  7735 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:51.740123  7753 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:51.967115  7735 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:51.577054  7736 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.968907  7702 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:51.970407  7736 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:51.970723  7385 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:51.973799  1717 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:51.734545  7500 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.860922  1711 raft_consensus.cc:2393] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:51.953775  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:51.916148  7834 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.146s
I20260709 13:03:51.738894  7821 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:51.954648  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:51.954684  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:51.957041  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:51.996776  2301 raft_consensus.cc:2393] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:51.860575  1718 raft_consensus.cc:2368] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18.
I20260709 13:03:51.899781  7894 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:51.964010  7887 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.630787  7812 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:51.967188  7753 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:51.974146  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:51.979333  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 217942us
I20260709 13:03:51.789924  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:51.770835  7497 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:51.986187  7867 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.893s	user 0.013s	sys 0.000s
I20260709 13:03:51.985836  7863 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:51.995095  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1730) took 1440 ms. Trace:
I20260709 13:03:51.954623  1682 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:52.001652  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.002182  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.015197  2304 rpcz_store.cc:276] 0709 13:03:50.554599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.555427 (+   828us) service_pool.cc:224] Handling call
0709 13:03:50.555864 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:50.602896 (+ 47032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:50.602943 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:50.603582 (+   639us) log.cc:844] Serialized 26 byte log entry
0709 13:03:50.722193 (+118611us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:50.722257 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.805581 (+1083324us) spinlock_profiling.cc:243] Waited 46.7 ms on lock 0x7b5400544cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:51.957669 (+152088us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:51.957784 (+   115us) raft_consensus.cc:1685] finished
0709 13:03:51.957807 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:51.995003 (+ 37196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1218,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"spinlock_wait_cycles":107424640,"thread_start_us":118180,"threads_started":1,"wal-append.queue_time_us":225141,"wal-append.run_cpu_time_us":15752,"wal-append.run_wall_time_us":1125080}
I20260709 13:03:52.009783  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2606) took 1608 ms. Trace:
I20260709 13:03:52.020313  2297 raft_consensus.cc:1275] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:03:52.020474  1718 rpcz_store.cc:276] 0709 13:03:50.401138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.763234 (+1362096us) service_pool.cc:224] Handling call
0709 13:03:52.009691 (+246457us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.023619  7836 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.024190  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2605) took 1639 ms. Trace:
I20260709 13:03:52.011328  1607 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:52.024565  1713 rpcz_store.cc:276] 0709 13:03:50.384691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.401217 (+ 16526us) service_pool.cc:224] Handling call
0709 13:03:52.024035 (+1622818us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54003daed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:52.024134 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1328263936}
I20260709 13:03:52.025125  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:52.025437  7751 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:52.025592  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.026835  7837 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.029942  7887 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.031060  1713 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:38292
I20260709 13:03:52.031642  7834 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:52.031781  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:52.032228  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 77744.9us
I20260709 13:03:52.032197  7834 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:52.033249  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2663) took 1336 ms (client timeout 1503 ms). Trace:
W20260709 13:03:52.033641  1639 rpcz_store.cc:269] 0709 13:03:50.696880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.699693 (+  2813us) service_pool.cc:224] Handling call
0709 13:03:52.033051 (+1333358us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400124b54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:52.033195 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2326963072}
W20260709 13:03:52.034283  7739 consensus_meta.cc:287] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.000s
I20260709 13:03:52.017261  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:51.981263  7744 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.040170  1562 raft_consensus.cc:2393] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 15: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:52.040488  7744 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:52.040997  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:52.041940  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.042136  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.002362  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 1248) took 1023 ms. Trace:
I20260709 13:03:52.043094  1711 rpcz_store.cc:276] 0709 13:03:50.978909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.766595 (+787686us) service_pool.cc:224] Handling call
0709 13:03:52.002295 (+235700us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.039180  1607 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:52.044209  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.044968  1711 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:52.017793  7599 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:52.018004  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.018155  7872 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:52.010252  7696 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:51.973579  7815 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:03:52.018903  7863 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:52.019109  7813 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.986s	user 0.011s	sys 0.000s
I20260709 13:03:52.018978  7497 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:51.979894  7714 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:52.034493  7834 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.032433  1561 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:03:52.031677  1713 raft_consensus.cc:606] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:52.085582  1713 raft_consensus.cc:614] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:52.037488  7879 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:52.012868  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 223034us
I20260709 13:03:52.013639  7821 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.047369  1644 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:52.057296  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:52.003304  7500 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.057605  1569 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:03:52.058017  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.065923  1562 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:03:52.094095  1565 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:51.993211  7486 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:52.079635  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.056999  7778 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:52.055450  7726 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.057255  7599 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:52.082999  7497 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:52.062898  7794 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.115644  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:52.116626  2303 raft_consensus.cc:2368] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 19. Current term is 20.
I20260709 13:03:52.089462  7893 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:52.090061  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:52.062083  7739 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.074632  7791 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:52.101485  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 46480.4us
I20260709 13:03:52.125752  1710 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:03:52.106349  2268 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:52.081411  7696 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:52.058780  7867 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.119216  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.084650  7714 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:52.122478  1635 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:52.097906  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:38292 (request call id 41) took 1478 ms. Trace:
I20260709 13:03:52.108362  7599 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:52.107993  7715 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.032s	sys 0.088s
I20260709 13:03:52.139061  1607 leader_election.cc:400] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18.
I20260709 13:03:52.142114  1713 rpcz_store.cc:276] 0709 13:03:50.609466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.030908 (+1421442us) service_pool.cc:224] Handling call
0709 13:03:52.085990 (+ 55082us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.090960  7818 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:52.138852  1644 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:52.135607  7786 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.108282  7755 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:03:52.117812  7834 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:52.114094  7894 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:52.166320  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:52.146667  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:52.146780  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.147542  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:52.148254  1637 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.033s	sys 0.070s
W20260709 13:03:52.169164  1637 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.033s	sys 0.070s
I20260709 13:03:52.153385  1563 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 16: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:52.084549  7872 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:52.172318  7872 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:52.082803  7813 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:52.131093  7778 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:52.174122  7778 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.175858  7813 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:52.177205  1718 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:52.177393  7778 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.143152  1534 leader_election.cc:400] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 19. Current term is 20.
I20260709 13:03:52.180167  1534 leader_election.cc:403] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:03:52.116964  7726 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.136904  7837 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:52.144532  7714 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:52.137989  7887 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:52.136472  7791 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:52.196468  7791 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:52.197263  1533 leader_election.cc:336] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Timed out: RequestConsensusVote RPC to 127.0.107.194:40023 timed out after 1.503s (SENT)
I20260709 13:03:52.197131  7849 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.198472  7849 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:52.202605  7879 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:52.203743  7851 consensus_meta.cc:287] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.185s	user 0.000s	sys 0.013s
I20260709 13:03:52.204248  7851 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:52.206212  7717 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:52.206650  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:52.206868  7717 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.208057  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.208822  1640 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:52.209751  1560 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 19: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:52.187050  7894 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:52.164917  7815 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:52.210907  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.211697  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:52.216192  2298 raft_consensus.cc:2393] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 20: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:52.216470  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.216993  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:52.216709  7696 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.217747  2298 raft_consensus.cc:2393] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 20: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:52.217818  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.108793  7486 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.140576  7518 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:52.190501  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.161336  7834 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:52.193809  7827 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:03:52.217216  1644 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 18: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:52.129395  7794 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.239698  1609 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:52.186393  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:52.218708  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:52.214959  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2610) took 1271 ms. Trace:
I20260709 13:03:52.244544  1710 rpcz_store.cc:276] 0709 13:03:50.943242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.956728 (+1013486us) service_pool.cc:224] Handling call
0709 13:03:52.214770 (+258042us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54003d9fd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:52.214875 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296535808}
I20260709 13:03:52.227144  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:52.246968  1713 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:52.246141  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.250262  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:52.220906  1533 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:52.251386  7800 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:52.251878  7800 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:52.250425  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 77257.8us
I20260709 13:03:52.253280  7800 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:52.255189  1710 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:52.235674  1635 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:03:52.258319  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:52.258894  2297 raft_consensus.cc:2468] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:52.187636  7836 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:52.222764  7813 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:52.213213  7894 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:52.236707  7834 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:52.219391  7837 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:52.279713  7837 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:52.281751  7895 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 13:03:52.212769  7815 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:52.271107  7744 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:52.273301  7739 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.273764  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:52.274323  7717 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:52.259263  7751 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:52.323145  1715 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2659) took 1992 ms (client timeout 2134 ms). Trace:
I20260709 13:03:52.273212  7702 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:03:52.225543  7791 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:52.214006  7869 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:52.369470  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 328293us
I20260709 13:03:52.163468  7786 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.236497  7887 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.263527  7518 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:52.261355  7906 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.292088  7615 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.292691  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1415) took 1027 ms. Trace:
I20260709 13:03:52.294524  1681 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:52.215160  7821 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.294749  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:52.292121  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.395601  1644 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
W20260709 13:03:52.395855  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 101206us
W20260709 13:03:52.260123  7889 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.661s	user 0.008s	sys 0.012s
W20260709 13:03:52.297640  1717 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 1235) took 1769 ms (client timeout 2158 ms). Trace:
W20260709 13:03:52.398821  1717 rpcz_store.cc:269] 0709 13:03:50.527800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.096504 (+568704us) service_pool.cc:224] Handling call
0709 13:03:52.297400 (+1200896us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b54000b0254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:52.297565 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1998949248}
I20260709 13:03:52.299136  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:52.301968  7880 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:52.275890  7867 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.339716  1710 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
W20260709 13:03:52.323699  1715 rpcz_store.cc:269] 0709 13:03:50.330541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.330883 (+   342us) service_pool.cc:224] Handling call
0709 13:03:52.322870 (+1991987us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400239954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:52.323077 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258492928}
I20260709 13:03:52.407954  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.261693  7872 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:52.406769  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2591) took 3117 ms. Trace:
I20260709 13:03:52.263389  7813 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:52.258463  7638 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.750s	user 0.012s	sys 0.001s
W20260709 13:03:52.396277  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 101980us
I20260709 13:03:52.412673  1681 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:52.385334  1532 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:52.385555  1637 rpcz_store.cc:276] 0709 13:03:51.265459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.266065 (+   606us) service_pool.cc:224] Handling call
0709 13:03:51.296383 (+ 30318us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:51.296799 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.296841 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:51.409774 (+112933us) log.cc:844] Serialized 52 byte log entry
0709 13:03:51.510851 (+101077us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:52.169572 (+658721us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.169817 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.169903 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.281083 (+111180us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:52.292320 (+ 11237us) raft_consensus.cc:1685] finished
0709 13:03:52.292447 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.292623 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":779091,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":41021,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":16460,"spinlock_wait_cycles":9227648,"thread_start_us":722841,"threads_started":2,"wal-append.queue_time_us":635631}
I20260709 13:03:52.398286  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2663) took 1152 ms. Trace:
I20260709 13:03:52.414184  1566 rpcz_store.cc:276] 0709 13:03:51.245530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.245652 (+   122us) service_pool.cc:224] Handling call
0709 13:03:51.245940 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.451132 (+205192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.398020 (+946888us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400217354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:52.398222 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428514048}
I20260709 13:03:52.414963  1532 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:52.366485  7778 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:52.404663  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.416849  1642 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:52.405941  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.417311  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.410400  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:52.419225  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 120216us
I20260709 13:03:52.384817  7399 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.422976  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.423019  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.420324  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:52.402359  7803 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:52.377972  7791 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.406888  7897 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:52.416049  7813 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:52.403210  6959 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:52.426612  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.435765  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.454854  1637 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:52.427297  1644 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:52.411139  7884 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.431453  7885 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:03:52.436365  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.460341  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.436612  7867 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:52.461563  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:52.438510  7815 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:52.439184  7896 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.351s	user 0.002s	sys 0.000s
I20260709 13:03:52.464829  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.439632  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.465507  1714 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:52.441335  7825 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.412096  7872 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:52.407878  7889 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.420562  7794 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.434783  7897 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.488374  7897 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:52.488850  7889 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:52.490286  7909 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:03:52.421427  7729 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:52.490553  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:52.427142  7778 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.464301  7884 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.430675  7399 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.450234  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1092) took 1203 ms. Trace:
I20260709 13:03:52.422531  7755 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:52.451392  1532 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:52.394510  7753 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:52.433352  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.505897  1715 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:52.507345  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.425480  7615 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:03:52.465974  1714 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 1241) took 1936 ms (client timeout 2064 ms). Trace:
I20260709 13:03:52.470989  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.507467  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:52.511436  1714 rpcz_store.cc:269] 0709 13:03:50.529609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.464482 (+1934873us) service_pool.cc:224] Handling call
0709 13:03:52.465892 (+  1410us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.405084  7906 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:52.476246  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.476269  1569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c129438e9d58493a93890208b25765cf"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:03:52.463551  7726 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.508955  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.478420  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1546) took 1183 ms. Trace:
I20260709 13:03:52.481050  1716 raft_consensus.cc:2368] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16.
I20260709 13:03:52.482208  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.481712  7872 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:52.432638  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:52.493716  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:52.495847  1566 raft_consensus.cc:2410] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
W20260709 13:03:52.497911  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 46704.5us
I20260709 13:03:52.466173  7867 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:52.412895  7880 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.493116  1568 rpcz_store.cc:276] 0709 13:03:51.246995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.259042 (+ 12047us) service_pool.cc:224] Handling call
0709 13:03:51.259368 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:51.294304 (+ 34936us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.294349 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:52.090330 (+795981us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b2c00000658. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:03:52.090743 (+   413us) log.cc:844] Serialized 26 byte log entry
0709 13:03:52.374552 (+283809us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:52.374601 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.374694 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.449926 (+ 75232us) raft_consensus.cc:1685] finished
0709 13:03:52.449947 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.450169 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34546,"raft.queue_time_us":75528,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":5188,"spinlock_wait_cycles":11848448,"thread_start_us":275043,"threads_started":1,"wal-append.queue_time_us":345308}
I20260709 13:03:52.517649  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.518502  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:52.420083  7638 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.512374  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.525765  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:52.526525  1714 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:52.513471  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.528455  1717 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:03:52.471179  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:52.513998  1569 raft_consensus.cc:606] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:52.530071  1569 raft_consensus.cc:614] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:52.478153  7851 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:52.515854  2296 rpcz_store.cc:276] 0709 13:03:51.294669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.294868 (+   199us) service_pool.cc:224] Handling call
0709 13:03:51.334755 (+ 39887us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:51.428499 (+ 93744us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.428578 (+    79us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:51.429235 (+   657us) log.cc:844] Serialized 26 byte log entry
0709 13:03:51.775134 (+345899us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:51.775174 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:51.775241 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.308151 (+532910us) spinlock_profiling.cc:243] Waited 3.58 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:52.478064 (+169913us) raft_consensus.cc:1685] finished
0709 13:03:52.478086 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.478335 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86815,"raft.queue_time_us":94270,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":3955,"spinlock_wait_cycles":8240000,"thread_start_us":319973,"threads_started":1,"wal-append.queue_time_us":576270}
W20260709 13:03:52.520229  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 87685.9us
I20260709 13:03:52.531592  1681 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:52.519204  1568 raft_consensus.cc:2368] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:52.533406  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:52.523159  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.512398  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:52.537850  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:52.540530  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:52.541312  7753 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:52.541935  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.468835  6959 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.519522  7615 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:52.543291  1569 raft_consensus.cc:2410] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 16 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:03:52.544166  7836 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 19. Current term is 20.
I20260709 13:03:52.544088  7615 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:52.545639  7399 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.492997  7778 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.547235  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.547932  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:52.548235  1637 raft_consensus.cc:2410] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:03:52.599624  7786 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.512308  7729 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:52.601204  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:52.624496  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.602334  1562 raft_consensus.cc:2368] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
W20260709 13:03:52.602702  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 131680us
W20260709 13:03:52.614609  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:52.626622  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:52.627399  1562 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:52.629654  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.630411  1568 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:03:52.630445  2298 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:52.631582  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.632937  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:52.627239  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 196083us
I20260709 13:03:52.521899  7894 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:52.513507  7919 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:52.660786  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:52.519590  7867 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:03:52.661710  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:52.620002  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 52970.7us
I20260709 13:03:52.673637  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:52.549729  7794 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:52.684536  1637 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:52.601509  7764 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:52.687882  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.660827  7894 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:52.622831  7726 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:52.732671  7726 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:52.674475  1715 raft_consensus.cc:2376] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18. Re-sending same reply.
I20260709 13:03:52.713320  7800 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:52.722395  1714 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:52.723906  7894 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:52.724324  7884 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.647681  7599 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.629640  7399 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:52.661407  7889 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.747619  1565 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:03:52.742609  7744 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:52.779300  7924 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:03:52.742599  7919 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:52.783970  7880 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:52.785473  7638 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:52.788146  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:52.632117  7787 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.110s
I20260709 13:03:52.791133  7764 consensus_queue.cc:1048] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:52.794210  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1421) took 1016 ms. Trace:
I20260709 13:03:52.863909  1636 rpcz_store.cc:276] 0709 13:03:51.777700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.782494 (+  4794us) service_pool.cc:224] Handling call
0709 13:03:51.782817 (+   323us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:52.123244 (+340427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.123283 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:52.144829 (+ 21546us) log.cc:844] Serialized 52 byte log entry
0709 13:03:52.419985 (+275156us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:52.431764 (+ 11779us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.431964 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.432038 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.793968 (+361930us) raft_consensus.cc:1685] finished
0709 13:03:52.793992 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.794146 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":172925,"prepare.queue_time_us":11599,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":177922,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":20801,"thread_start_us":363860,"threads_started":2,"wal-append.queue_time_us":471504}
I20260709 13:03:52.795222  7825 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.797030  7791 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:52.868683  2301 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:52.800756  7836 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:52.803296  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:52.807070  7726 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:52.810194  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3184) took 1200 ms. Trace:
I20260709 13:03:52.818392  7520 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:52.847433  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:52.857225  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 69165.7us
I20260709 13:03:52.883435  1715 raft_consensus.cc:2410] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:03:52.883572  1607 leader_election.cc:400] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:52.884084  1607 leader_election.cc:403] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:03:52.879611  1561 rpcz_store.cc:276] 0709 13:03:51.609488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.090966 (+481478us) service_pool.cc:224] Handling call
0709 13:03:52.091263 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:52.091625 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.091654 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:52.115997 (+ 24343us) log.cc:844] Serialized 26 byte log entry
0709 13:03:52.421864 (+305867us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:52.421922 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.421996 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:52.809873 (+387877us) raft_consensus.cc:1685] finished
0709 13:03:52.809906 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.810110 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7933,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":14031,"thread_start_us":271117,"threads_started":1,"wal-append.queue_time_us":671273}
I20260709 13:03:52.885746  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:52.886140  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.867506  1568 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:52.851136  7778 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:52.852455  7399 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:52.873817  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:52.907105  1636 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
W20260709 13:03:52.796084  7863 consensus_meta.cc:287] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.714s	user 0.011s	sys 0.000s
I20260709 13:03:52.856350  7744 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:52.868098  7753 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:52.887943  7764 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 12. Current term is 13.
I20260709 13:03:52.897915  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1242) took 2368 ms. Trace:
I20260709 13:03:52.946482  7599 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:52.884102  7836 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.854756  7729 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:52.855875  7889 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.927016  1636 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:52.887276  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:52.934752  1714 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:52.912009  7399 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:52.947703  1715 rpcz_store.cc:276] 0709 13:03:50.529915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.896887 (+2366972us) service_pool.cc:224] Handling call
0709 13:03:52.897308 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.897495 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.897532 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:52.897596 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:52.897641 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.897711 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.897845 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.887742  7702 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:52.912870  7778 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.868175  7919 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:52.900465  7834 consensus_meta.cc:287] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.000s
W20260709 13:03:52.954366  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 301204us
I20260709 13:03:52.862046  1562 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:03:52.971287  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.972764  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:52.974053  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:52.956034  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1239) took 2427 ms. Trace:
I20260709 13:03:52.975345  1714 rpcz_store.cc:276] 0709 13:03:50.529030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.853516 (+2324486us) service_pool.cc:224] Handling call
0709 13:03:52.934465 (+ 80949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.955809 (+ 21344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.955985 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.892983  7520 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:52.960273  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:52.962503  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 75303us
I20260709 13:03:52.996971  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:52.974655  1562 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:52.957486  7604 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.008s	sys 0.018s
I20260709 13:03:52.999869  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:53.007733  1561 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:53.008975  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1244) took 2478 ms. Trace:
I20260709 13:03:53.009528  1714 rpcz_store.cc:276] 0709 13:03:50.530500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.008135 (+2477635us) service_pool.cc:224] Handling call
0709 13:03:53.008426 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.008592 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.008614 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.008650 (+    36us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:53.008681 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.008756 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.008902 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.013711  1714 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:53.014586  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1245) took 2119 ms. Trace:
I20260709 13:03:53.015031  1714 rpcz_store.cc:276] 0709 13:03:50.895539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.010733 (+2115194us) service_pool.cc:224] Handling call
0709 13:03:53.013476 (+  2743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.014302 (+   826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.014512 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:52.954617  7764 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:53.021303  1607 leader_election.cc:400] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:03:53.024242  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:03:52.998319  7714 consensus_meta.cc:287] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.758s	user 0.010s	sys 0.000s
I20260709 13:03:52.961959  7931 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:53.022658  7497 consensus_meta.cc:287] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.881s	user 0.012s	sys 0.000s
W20260709 13:03:53.041661  7928 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:03:53.041919  7778 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.042280  7889 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.036998  7714 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.043764  7919 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:53.029948  7927 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:53.047276  7897 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.002s
I20260709 13:03:53.061542  1562 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
W20260709 13:03:53.048555  7813 consensus_meta.cc:287] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.004s
I20260709 13:03:53.012568  7702 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:53.022500  7753 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:53.037599  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:53.060139  1638 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:53.001775  7834 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.047747  7715 raft_consensus.cc:3055] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:53.040252  7800 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:53.056212  7545 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:53.036119  7764 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.027361  7867 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.088694  7737 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:53.089895  7894 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:53.040218  7931 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.087662  1638 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:53.099654  1639 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:53.100136  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:53.098037  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2667) took 2745 ms. Trace:
I20260709 13:03:53.069139  7897 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.075205  7813 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.101291  7726 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:53.131199  7888 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.131445  7744 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:53.131884  7888 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.133260  7726 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:53.133610  1715 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:53.091912  7545 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:53.134390  7545 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:53.137303  7894 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:53.055877  7497 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.139308  1569 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:03:53.124249  1560 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:53.140180  1710 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:03:53.140309  1714 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:53.077273  7889 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:53.140916  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2608) took 2207 ms. Trace:
I20260709 13:03:53.143929  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:53.147356  1710 rpcz_store.cc:276] 0709 13:03:50.933798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.139623 (+2205825us) service_pool.cc:224] Handling call
0709 13:03:53.139966 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.140587 (+   621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.140845 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.133313  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3107) took 1042 ms. Trace:
I20260709 13:03:53.142042  1712 rpcz_store.cc:276] 0709 13:03:50.352675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.352932 (+   257us) service_pool.cc:224] Handling call
0709 13:03:50.382670 (+ 29738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:50.422890 (+ 40220us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:50.671543 (+248653us) log.cc:844] Serialized 14 byte log entry
0709 13:03:50.832078 (+160535us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:50.832191 (+   113us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:50.832273 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.097792 (+2265519us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b54001360d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:53.097849 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.097977 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43262,"prepare.queue_time_us":132755,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":37610,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":36862,"spinlock_wait_cycles":91866368,"thread_start_us":344686,"threads_started":2,"wal-append.queue_time_us":156101,"wal-append.run_cpu_time_us":1118,"wal-append.run_wall_time_us":1001138}
I20260709 13:03:53.149262  1638 rpcz_store.cc:276] 0709 13:03:52.091417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.071452 (+980035us) service_pool.cc:224] Handling call
0709 13:03:53.087463 (+ 16011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.133062 (+ 45599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.133253 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.077770  7715 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:53.154129  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 146818us
I20260709 13:03:52.408881  1709 rpcz_store.cc:276] 0709 13:03:49.281012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:49.305897 (+ 24885us) service_pool.cc:224] Handling call
0709 13:03:49.391941 (+ 86044us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:49.392178 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:03:50.420434 (+1028256us) log.cc:844] Serialized 14 byte log entry
0709 13:03:51.732816 (+1312382us) spinlock_profiling.cc:243] Waited 5.1 ms on lock 0x7b1800345c40. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a
0709 13:03:52.101979 (+369163us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.374549 (+272570us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b480009ffd8. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a6bcaaa1 00007fd7a6bc1b37 00007fd7a6bc19ca 00007fd7a6cb926f 00007fd7a6cb8f37 00007fd7a6ccd55c 00007fd7a6ccd508 00007fd7a6ccd488 00007fd7a6ccd443 00007fd7a6ccc6ec 00007fd7a5bf6a96
0709 13:03:52.397502 (+ 22953us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:52.397914 (+   412us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:52.397969 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:52.398051 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:52.398197 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5806,"prepare.queue_time_us":1599457,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":100,"raft.queue_time_us":9559,"raft.run_cpu_time_us":3414,"raft.run_wall_time_us":631847,"spinlock_wait_cycles":25239168,"thread_start_us":1762486,"threads_started":4,"wal-append.queue_time_us":173206,"wal-append.run_cpu_time_us":1015,"wal-append.run_wall_time_us":1001049}
I20260709 13:03:53.167128  1711 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:53.167304  2268 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:53.168332  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:53.121531  7520 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:53.065397  7927 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.171460  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:53.121824  7834 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.169773  1643 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 22: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:53.133291  7813 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.123978  7867 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.164173  7880 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.172155  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:53.175616  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:53.163908  7599 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:53.175379  7925 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:53.172976  7937 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:53.188848  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:53.159475  7399 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:53.179483  7520 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:03:53.171404  7889 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:53.172531  7714 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.194010  7599 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.192128  7497 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.195818  1712 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:53.187191  7872 consensus_meta.cc:287] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.000s
I20260709 13:03:53.220638  7919 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:03:53.227855  7919 consensus_queue.cc:1243] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:53.228397  7919 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:53.228514  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:53.228811  7919 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:53.229190  1568 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:53.185554  7836 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.226197  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:53.204787  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:53.210247  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2671) took 2698 ms. Trace:
I20260709 13:03:53.234539  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:53.206790  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:53.244607  2298 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:53.229696  7894 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:53.232728  1718 rpcz_store.cc:276] 0709 13:03:50.512262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.037568 (+1525306us) service_pool.cc:224] Handling call
0709 13:03:52.037867 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.177861 (+139994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.209990 (+1032129us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400268ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:53.210173 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319835520}
W20260709 13:03:53.212214  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:53.250957  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.252475  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:53.256421  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:53.259892  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:53.263957  7599 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:53.264140  7944 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:53.204393  7399 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:53.203789  7818 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:53.265295  7818 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:53.265604  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 744541us
I20260709 13:03:53.202811  7931 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:53.267546  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3093) took 1231 ms. Trace:
I20260709 13:03:53.268018  1640 rpcz_store.cc:276] 0709 13:03:52.036159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.218896 (+182737us) service_pool.cc:224] Handling call
0709 13:03:52.219210 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.284736 (+ 65526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:52.284767 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:52.284808 (+    41us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:52.284838 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.267275 (+982437us) spinlock_profiling.cc:243] Waited 65.3 ms on lock 0x7b54002d8f54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:53.267335 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.267470 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150265088}
I20260709 13:03:53.263368  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:53.269483  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.270680  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:53.271436  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:53.273295  7893 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:53.273464  7755 consensus_meta.cc:287] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.770s	user 0.011s	sys 0.000s
I20260709 13:03:53.273823  7893 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:53.274032  7755 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:53.221908  7764 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.273499  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:53.201288  7889 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.276314  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:53.277282  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:53.271507  7864 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:53.280737  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.280874  7864 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:53.281445  7864 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:53.281575  7764 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:53.281878  7486 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.282174  7764 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.199947  7942 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:03:53.225752  7872 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:53.210493  7927 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:53.285650  7872 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:53.186544  7880 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.287159  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:53.272006  7867 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:53.278661  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:53.291790  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.292439  1640 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:53.277350  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:53.294762  1567 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:03:53.295172  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.293180  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:53.295815  1640 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:53.296420  1712 raft_consensus.cc:2468] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:53.297147  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.297699  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:53.297758  2268 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:53.298578  1718 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:53.302083  1568 raft_consensus.cc:2410] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:03:53.302381  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:53.302971  1568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:03:53.303614  1568 raft_consensus.cc:606] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:53.303987  1568 raft_consensus.cc:614] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:03:53.303920  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:53.305153  1718 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:53.255638  7937 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:53.307158  2269 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:53.307828  1640 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:53.309226  2296 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:53.309279  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:53.311148  1640 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:53.290376  7812 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.291242  7867 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:53.294193  7936 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:03:53.312917  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:53.229737  7919 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.291615  7927 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:53.319056  2269 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:53.320640  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:53.254678  7894 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:53.215618  7715 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.301349  7925 raft_consensus.cc:697] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:53.304638  7399 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:03:53.344476  7399 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:53.344874  7399 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:53.345606  7399 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:53.349294  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3108) took 1200 ms. Trace:
I20260709 13:03:53.349763  1638 rpcz_store.cc:276] 0709 13:03:52.149246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.150440 (+1001194us) service_pool.cc:224] Handling call
0709 13:03:53.165589 (+ 15149us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.170358 (+  4769us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:53.170389 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:53.170871 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 13:03:53.284313 (+113442us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b480015da58. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a6bcaaa1 00007fd7a6bc1b37 00007fd7a6bc19ca 00007fd7a6cb926f 00007fd7a6cb8f37 00007fd7a6ccd55c 00007fd7a6ccd508 00007fd7a6ccd488 00007fd7a6ccd443 00007fd7a6ccc6ec 00007fd7a5bf6a96
0709 13:03:53.294267 (+  9954us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:53.294329 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.294399 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:53.348926 (+ 54527us) raft_consensus.cc:1685] finished
0709 13:03:53.348951 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.349214 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4277,"raft.queue_time_us":5584,"raft.run_cpu_time_us":2063,"raft.run_wall_time_us":108718,"spinlock_wait_cycles":2504448,"thread_start_us":121789,"threads_started":1,"wal-append.queue_time_us":176210}
W20260709 13:03:53.352885  7935 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.264s	user 0.002s	sys 0.000s
I20260709 13:03:53.356437  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:53.359699  1713 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2611) took 1298 ms (client timeout 1561 ms). Trace:
W20260709 13:03:53.360247  1713 rpcz_store.cc:269] 0709 13:03:52.060914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.190142 (+129228us) service_pool.cc:224] Handling call
0709 13:03:53.359473 (+1169331us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b54003db3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:53.359617 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60195200}
I20260709 13:03:53.362385  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2673) took 2343 ms. Trace:
I20260709 13:03:53.323935  7867 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:53.357993  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:53.316553  7576 raft_consensus.cc:2804] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:53.326201  7769 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:53.363305  7486 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.327275  7764 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.327582  7812 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.416414  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1236) took 2888 ms. Trace:
I20260709 13:03:53.364542  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1545) took 2103 ms. Trace:
I20260709 13:03:53.385347  1640 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:53.364933  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:53.417512  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.381261  1710 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:03:53.379253  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:53.427754  1683 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:53.364367  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:53.375118  1717 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:53.375077  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1250) took 2395 ms. Trace:
I20260709 13:03:53.437493  1635 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:53.438112  1712 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:53.362887  1713 rpcz_store.cc:276] 0709 13:03:51.019448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.361409 (+2341961us) service_pool.cc:224] Handling call
0709 13:03:53.361774 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.361994 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.362032 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.362087 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:53.362127 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.362197 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.362320 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.436861  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2681) took 1053 ms. Trace:
I20260709 13:03:53.436857  2305 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:03:53.435266  1711 rpcz_store.cc:276] 0709 13:03:50.528132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.045834 (+1517702us) service_pool.cc:224] Handling call
0709 13:03:52.046168 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.169666 (+1123498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.416094 (+246428us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540033a4d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:53.416338 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2577602304}
I20260709 13:03:53.438454  1709 rpcz_store.cc:276] 0709 13:03:50.979483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.156239 (+2176756us) service_pool.cc:224] Handling call
0709 13:03:53.171785 (+ 15546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.171969 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:03:53.292274 (+120305us) log.cc:844] Serialized 14 byte log entry
0709 13:03:53.374732 (+ 82458us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.374785 (+    53us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:53.374830 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.374896 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.375024 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3464,"prepare.queue_time_us":132928,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":96,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":86,"thread_start_us":192918,"threads_started":2,"wal-append.queue_time_us":127491}
I20260709 13:03:53.313903  7937 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:53.442090  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:53.385468  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1550) took 1287 ms. Trace:
I20260709 13:03:53.443382  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:53.443791  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:53.456027  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 76871.2us
I20260709 13:03:53.456370  1534 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:53.441651  1640 rpcz_store.cc:276] 0709 13:03:52.383313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.337622 (+954309us) service_pool.cc:224] Handling call
0709 13:03:53.385153 (+ 47531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.436600 (+ 51447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.436805 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.456059  2301 rpcz_store.cc:276] 0709 13:03:52.098630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.098809 (+   179us) service_pool.cc:224] Handling call
0709 13:03:52.119470 (+ 20661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:52.869267 (+749797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.385232 (+515965us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b5400686ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:53.385405 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1722576640}
I20260709 13:03:53.446108  7726 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:53.446108  7764 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:53.447062  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:53.379710  7896 log.cc:927] Time spent T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:03:53.460282  1635 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:53.410557  7925 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.463902  7937 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:53.464092  7818 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.386898  7919 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:53.410578  7813 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:53.467643  7813 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:53.469964  7737 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:53.470404  7737 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.382675  7715 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:53.384061  7399 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:53.419387  6959 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.445061  1683 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:53.459116  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:53.351121  7888 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.473650  1718 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:53.481298  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:53.481436  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:53.452239  7894 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:53.380344  7836 consensus_queue.cc:1175] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" responder_term: 15 status { last_received { term: 15 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 15 index: 11 } }
I20260709 13:03:53.435691  2302 rpcz_store.cc:276] 0709 13:03:51.260950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.261052 (+   102us) service_pool.cc:224] Handling call
0709 13:03:51.263738 (+  2686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:51.295573 (+ 31835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:51.295619 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:51.312656 (+ 17037us) log.cc:844] Serialized 26 byte log entry
0709 13:03:52.135889 (+823233us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b18002efba0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:03:52.289910 (+154021us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:52.366943 (+ 77033us) log.cc:844] Serialized 14 byte log entry
0709 13:03:52.367169 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.364141 (+996972us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b54006077d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:53.364215 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:53.364277 (+    62us) raft_consensus.cc:1685] finished
0709 13:03:53.364297 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.364466 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83288,"prepare.queue_time_us":79446,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":2898,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":35137,"spinlock_wait_cycles":74871296,"thread_start_us":153809,"threads_started":1,"wal-append.queue_time_us":981714,"wal-append.run_cpu_time_us":2324,"wal-append.run_wall_time_us":1031606}
I20260709 13:03:53.402407  7906 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.401902  7897 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.474439  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:53.483708  1718 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:53.495428  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:53.496255  1718 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:53.497292  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:53.497985  1718 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:03:53.480429  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.485446  1683 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:53.500058  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:53.500890  1718 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:53.475422  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:53.487000  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2665) took 2544 ms. Trace:
I20260709 13:03:53.502339  1641 rpcz_store.cc:276] 0709 13:03:50.935775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:50.935942 (+   167us) service_pool.cc:224] Handling call
0709 13:03:50.936428 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:51.469696 (+533268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.479445 (+2009749us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54002d8054. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:53.479650 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1217465728}
I20260709 13:03:53.486980  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:53.504354  1713 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 21: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:53.505815  1713 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:03:53.506522  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2675) took 2478 ms. Trace:
I20260709 13:03:53.459829  7726 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.383639  7751 consensus_queue.cc:1059] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.097s
I20260709 13:03:53.508152  1682 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:53.470762  7737 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:53.509991  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.511922  1711 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:53.400578  7864 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.474083  7764 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.466645  7545 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:53.439198  7486 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.511538  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:53.530670  1682 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:53.510495  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:53.514832  1717 raft_consensus.cc:1240] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:03:53.532325  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2674) took 2508 ms. Trace:
I20260709 13:03:53.532729  1717 rpcz_store.cc:276] 0709 13:03:51.024436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.502059 (+2477623us) service_pool.cc:224] Handling call
0709 13:03:53.514520 (+ 12461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.532051 (+ 17531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.532265 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.530835  1635 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:53.506935  1713 rpcz_store.cc:276] 0709 13:03:51.027980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.505030 (+2477050us) service_pool.cc:224] Handling call
0709 13:03:53.505402 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.506240 (+   838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.506457 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.537009  1713 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:53.518093  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1243) took 2987 ms. Trace:
I20260709 13:03:53.519080  1711 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:03:53.510583  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:53.494621  6959 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:53.512127  1640 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:53.537650  1715 rpcz_store.cc:276] 0709 13:03:50.530198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.004294 (+2474096us) service_pool.cc:224] Handling call
0709 13:03:53.004675 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.134055 (+129380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.517747 (+383692us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54000b0ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:53.518010 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296102656}
I20260709 13:03:53.538760  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2676) took 2089 ms. Trace:
I20260709 13:03:53.548681  1711 rpcz_store.cc:276] 0709 13:03:51.449057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.515714 (+2066657us) service_pool.cc:224] Handling call
0709 13:03:53.516098 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.538404 (+ 22306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.538684 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.539074  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2682) took 1471 ms. Trace:
I20260709 13:03:53.549427  1715 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:53.512470  7726 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:53.551013  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2683) took 1477 ms. Trace:
I20260709 13:03:53.549912  1713 rpcz_store.cc:276] 0709 13:03:52.067267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.536349 (+1469082us) service_pool.cc:224] Handling call
0709 13:03:53.536762 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.538764 (+  2002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.539004 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.510376  7872 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:53.551477  1715 rpcz_store.cc:276] 0709 13:03:52.073302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.548768 (+1475466us) service_pool.cc:224] Handling call
0709 13:03:53.549184 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.550725 (+  1541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.550956 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.553251  1715 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:03:53.535434  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:53.519225  7497 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:53.525592  7714 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:53.559418  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.560138  1712 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:53.560395  1635 raft_consensus.cc:2410] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:03:53.560756  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2678) took 2091 ms. Trace:
I20260709 13:03:53.561137  1712 rpcz_store.cc:276] 0709 13:03:51.469372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.531860 (+2062488us) service_pool.cc:224] Handling call
0709 13:03:53.559975 (+ 28115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.560525 (+   550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.560707 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.557946  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2680) took 1515 ms. Trace:
I20260709 13:03:53.563812  1711 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:53.563071  1717 rpcz_store.cc:276] 0709 13:03:52.042370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.533611 (+1491241us) service_pool.cc:224] Handling call
0709 13:03:53.557421 (+ 23810us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.557582 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.557623 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.557670 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:53.557711 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.557774 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.557887 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:53.532385  7743 raft_consensus.cc:2764] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:03:53.557946  1718 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 22.
I20260709 13:03:53.524164  7764 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:53.571159  1713 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:53.550628  7825 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:53.559476  7714 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.569793  6959 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:53.592664  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 57325.8us
I20260709 13:03:53.581984  7849 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.579898  7751 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:53.591171  7714 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:53.607334  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1427) took 1567 ms. Trace:
I20260709 13:03:53.607733  1636 rpcz_store.cc:276] 0709 13:03:52.039763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.965776 (+926013us) service_pool.cc:224] Handling call
0709 13:03:52.966263 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.019375 (+ 53112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.019420 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.019477 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:53.019521 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.607165 (+587644us) spinlock_profiling.cc:243] Waited 52.8 ms on lock 0x7b54001c7354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:53.607196 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.607291 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121534848}
I20260709 13:03:53.594827  6959 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.611058  6959 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:53.611646  1636 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:53.615132  1569 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:03:53.624323  7825 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:53.626299  7743 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:03:53.627568  7744 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:53.627987  7744 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:53.628850  7744 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:53.629568  7812 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.631001  1682 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:53.632282  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:53.634148  7497 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:53.634599  7497 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:53.634879  7825 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:53.635109  7497 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:53.635481  7825 raft_consensus.cc:697] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:03:53.636412  1716 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 1252) took 1848 ms (client timeout 2078 ms). Trace:
W20260709 13:03:53.637027  1716 rpcz_store.cc:269] 0709 13:03:51.787845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.800467 (+ 12622us) service_pool.cc:224] Handling call
0709 13:03:53.636207 (+1835740us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b54003377d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:53.636353 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1562799360}
I20260709 13:03:53.634644  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3097) took 1597 ms. Trace:
I20260709 13:03:53.641203  1637 rpcz_store.cc:276] 0709 13:03:52.037240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.913060 (+875820us) service_pool.cc:224] Handling call
0709 13:03:52.949783 (+ 36723us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.030939 (+ 81156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.030969 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:53.053952 (+ 22983us) log.cc:844] Serialized 26 byte log entry
0709 13:03:53.187439 (+133487us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:53.187503 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.634242 (+446739us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b5400246b54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:53.634289 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:53.634334 (+    45us) raft_consensus.cc:1685] finished
0709 13:03:53.634348 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.634586 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5854,"raft.queue_time_us":8073,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":6532,"spinlock_wait_cycles":3669248,"thread_start_us":191043,"threads_started":2,"wal-append.queue_time_us":148631}
W20260709 13:03:53.647565  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:53.660964  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.671250  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:53.628851  7887 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.605722  7849 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:53.607055  7751 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.687512  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2677) took 1089 ms. Trace:
I20260709 13:03:53.661952  7872 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:53.610231  7893 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.689507  1568 rpcz_store.cc:276] 0709 13:03:52.597980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.329483 (+731503us) service_pool.cc:224] Handling call
0709 13:03:53.329977 (+   494us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:53.330488 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.330529 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:53.380285 (+ 49756us) log.cc:844] Serialized 52 byte log entry
0709 13:03:53.555227 (+174942us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:53.677645 (+122418us) log.cc:844] Serialized 14 byte log entry
0709 13:03:53.687186 (+  9541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.687260 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:53.687298 (+    38us) raft_consensus.cc:1685] finished
0709 13:03:53.687308 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.687455 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1064,"raft.queue_time_us":9449,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":5595,"thread_start_us":232813,"threads_started":2,"wal-append.queue_time_us":187598}
I20260709 13:03:53.712925  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:53.636448  7825 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.725955  1636 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:53.731025  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 98804.1us
I20260709 13:03:53.731932  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:53.736006  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.714448  7864 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.687724  6959 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:53.740547  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:53.747139  1716 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:03:53.752398  7888 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.753057  7888 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.756613  7888 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.740144  7966 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:53.764582  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.765465  1637 raft_consensus.cc:2410] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:03:53.766402  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:53.766955  7961 log.cc:927] Time spent T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:03:53.763684  7966 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:53.768023  7966 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:53.758147  7545 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:53.780606  7744 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:53.782073  7737 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.785167  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2685) took 1374 ms. Trace:
W20260709 13:03:53.787618  7957 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:03:53.787901  7849 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:53.803900  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.804663  1637 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:53.805226  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1108) took 1009 ms. Trace:
I20260709 13:03:53.805610  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.806262  1637 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
W20260709 13:03:53.807494  2297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1552) took 1702 ms (client timeout 1836 ms). Trace:
W20260709 13:03:53.808050  2297 rpcz_store.cc:269] 0709 13:03:52.105514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.124085 (+ 18571us) service_pool.cc:224] Handling call
0709 13:03:53.807275 (+1683190us) spinlock_profiling.cc:243] Waited 90.6 ms on lock 0x7b5400650254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:53.807419 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208367488}
I20260709 13:03:53.810068  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1257) took 1518 ms. Trace:
I20260709 13:03:53.799021  1638 rpcz_store.cc:276] 0709 13:03:52.411193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.352679 (+941486us) service_pool.cc:224] Handling call
0709 13:03:53.352957 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.471837 (+118880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.471882 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.471940 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:53.471984 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.784802 (+312818us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540032c3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:53.784873 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.785076 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":272825728}
I20260709 13:03:53.714895  7872 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:53.762135  7867 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.805642  1565 rpcz_store.cc:276] 0709 13:03:52.797095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.366612 (+569517us) service_pool.cc:224] Handling call
0709 13:03:53.367041 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.367305 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.367341 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:53.367918 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 13:03:53.513704 (+145786us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:53.513738 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.513797 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:53.804995 (+291198us) raft_consensus.cc:1685] finished
0709 13:03:53.805016 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.805166 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":145464,"threads_started":1,"wal-append.queue_time_us":239534}
I20260709 13:03:53.820479  1565 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:53.821965  1565 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:53.810566  1711 rpcz_store.cc:276] 0709 13:03:52.291585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.555448 (+1263863us) service_pool.cc:224] Handling call
0709 13:03:53.555770 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.564179 (+  8409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.809788 (+245609us) spinlock_profiling.cc:243] Waited 7.73 ms on lock 0x7b54001c1454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:53.810013 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17772928}
I20260709 13:03:53.791455  6959 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.811968  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.835767   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:53.760835  7486 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.839111  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:53.840756  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:53.841517  1638 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:53.839906  1637 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:53.845284  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:53.811720  7864 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:53.846316  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:53.846976  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2686) took 1430 ms. Trace:
I20260709 13:03:53.847703  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.848305  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:53.857751  1565 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:03:53.858810  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:53.862360  1561 raft_consensus.cc:2393] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 16: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:53.849736  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:53.864089  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:53.834800  7888 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:53.867360  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:53.872140  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:53.876271  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:53.877143  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:53.875557  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:53.832602  7497 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.866614  7894 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:53.884433  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:53.884565  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.886044  1569 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:53.868826  7937 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:03:53.874567  7893 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.883555  7906 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:53.889438  1638 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:53.878183  7545 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:53.897032  7893 consensus_queue.cc:1059] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:53.865569  7864 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.885288  7894 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:53.888388  7937 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:53.912997  1565 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
W20260709 13:03:53.915709  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 144017us
I20260709 13:03:53.901953  7906 raft_consensus.cc:697] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:53.870167  7888 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.907966  7500 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:53.910210  7894 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:53.925576  1533 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:53.913713  7937 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:03:53.935683  7970 log.cc:927] Time spent T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:03:53.921720  7751 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:53.948997  7956 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.361s	user 0.003s	sys 0.000s
I20260709 13:03:53.951812  7813 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:53.953292  7955 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.532s	user 0.001s	sys 0.001s
I20260709 13:03:53.960381  1569 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 22.
I20260709 13:03:53.965179  7714 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:53.969676  7906 consensus_queue.cc:237] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:53.970779  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 45321.1us
I20260709 13:03:54.017663  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:53.972690  7849 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:53.973284  7887 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:53.975598  7737 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.976511  7497 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.977573  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3196) took 1328 ms. Trace:
I20260709 13:03:54.025557  1568 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:53.979451  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:53.979794  7864 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.980185  7867 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.980659  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2667) took 1520 ms. Trace:
I20260709 13:03:53.856415  1640 rpcz_store.cc:276] 0709 13:03:52.416346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.556465 (+1140119us) service_pool.cc:224] Handling call
0709 13:03:53.556718 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.556977 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:53.710599 (+153622us) log.cc:844] Serialized 14 byte log entry
0709 13:03:53.846637 (+136038us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:53.846688 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:53.846735 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.846803 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.846903 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":202753,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":15569,"thread_start_us":246847,"threads_started":2}
I20260709 13:03:54.026612  1566 rpcz_store.cc:276] 0709 13:03:52.460016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.460115 (+    99us) service_pool.cc:224] Handling call
0709 13:03:53.980454 (+1520339us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b54001ff754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:53.980603 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79731584}
I20260709 13:03:53.988521  7927 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:54.036156  7927 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:54.036592  7927 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:54.037299  7764 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:53.989023  7576 raft_consensus.cc:697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:54.039204  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.039443  1641 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:03:54.039806  2305 raft_consensus.cc:3060] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:54.040092  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1246) took 3068 ms. Trace:
I20260709 13:03:54.039682  7576 consensus_queue.cc:237] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.040462  1710 rpcz_store.cc:276] 0709 13:03:50.971473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.148580 (+2177107us) service_pool.cc:224] Handling call
0709 13:03:53.373911 (+225331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.437398 (+ 63487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.039902 (+602504us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b54003db3d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:54.040035 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":517240064}
I20260709 13:03:53.944861  7888 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:53.991595  7726 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:53.992449  7974 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:03:54.045032  7726 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:54.036602  1564 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:03:54.045688  7894 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:54.046445  1715 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:54.047587  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1094) took 2800 ms. Trace:
I20260709 13:03:53.992334  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:53.988601  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 669763us
I20260709 13:03:54.047994  1564 rpcz_store.cc:276] 0709 13:03:51.247565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.247687 (+   122us) service_pool.cc:224] Handling call
0709 13:03:54.024788 (+2777101us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:54.046041 (+ 21253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.046788 (+   747us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b54002986d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:54.047414 (+   626us) spinlock_profiling.cc:243] Waited 2.76 s on lock 0x7b54002986d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:54.047527 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6372717184}
I20260709 13:03:53.996367  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.049103  2269 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:53.997051  2300 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:03:54.015726  2302 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:54.051122  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.052222  2302 raft_consensus.cc:2410] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:03:54.021668  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.023681  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:54.052836  2303 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:54.024727  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:54.028486  1563 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2664) took 2778 ms (client timeout 2800 ms). Trace:
W20260709 13:03:54.054821  1563 rpcz_store.cc:269] 0709 13:03:51.245832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:51.248950 (+  3118us) service_pool.cc:224] Handling call
0709 13:03:54.024323 (+2775373us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b54002986d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:54.024466 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2077433984}
I20260709 13:03:54.024583  7887 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.024349  7399 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:54.032064  7897 raft_consensus.cc:3055] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:54.032480  7879 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:54.034842  7880 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:54.035306  7966 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.054487  1561 raft_consensus.cc:2410] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:03:54.054463  1681 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:54.063185  2305 raft_consensus.cc:2468] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:54.067802  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.068683  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.069281  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.080039  1533 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:54.070183  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.081254  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:54.070199  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 181375us
I20260709 13:03:54.024551  1567 rpcz_store.cc:276] 0709 13:03:52.649524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.339092 (+689568us) service_pool.cc:224] Handling call
0709 13:03:53.341602 (+  2510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.341935 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.341966 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:53.342456 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:03:53.480305 (+137849us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:53.586826 (+106521us) log.cc:844] Serialized 14 byte log entry
0709 13:03:53.587069 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:53.587300 (+   231us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:53.948449 (+361149us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:53.956660 (+  8211us) raft_consensus.cc:1685] finished
0709 13:03:53.977343 (+ 20683us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:53.977501 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":187557,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":5368,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":5597,"spinlock_wait_cycles":7454080,"thread_start_us":211111,"threads_started":2,"wal-append.queue_time_us":211931}
I20260709 13:03:54.081542  1681 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:54.075300  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:54.032400  7715 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.002s
I20260709 13:03:54.088186  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:54.091928  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:54.093917  7849 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:54.094350  7726 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:54.095498  1533 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:54.096323  7849 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.097476  1533 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:54.101581  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:54.103978  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:54.069063  7714 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:54.068307  7812 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:54.078076  7879 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.107486  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:54.106750  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.069267  7897 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:03:54.111729  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 155808us
I20260709 13:03:54.112679  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:54.118518  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 69497.4us
I20260709 13:03:54.075927  7545 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:54.121546  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.122853  2303 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:54.084498  7864 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:03:54.086450  7497 raft_consensus.cc:2804] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:54.126861  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:54.106158  7399 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:03:54.110559  7751 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:54.106029  7827 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.136993  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.146432  2304 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:03:54.086333  7867 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:03:54.133664  2297 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:03:54.085019  7737 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:03:54.123299  7715 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:54.154881  2302 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:54.166666  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.120805  7545 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:54.162647  2298 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:54.162281  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.163623  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.106269  7744 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:03:54.125463  7923 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:54.141709  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:54.135761  7925 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:54.177984  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.145684  7518 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:54.143949  7497 raft_consensus.cc:697] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:54.181434  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.120112  7937 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.196386  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.172060  7715 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.184564  7867 consensus_queue.cc:1243] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:54.155458  7760 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:03:54.163928  7827 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.182262  7919 consensus_queue.cc:1059] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:03:54.203393  7894 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:54.199945  7897 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.218026  7897 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:54.219446  7966 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:54.223685  7937 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:54.224149  7951 raft_consensus.cc:3055] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:54.224741  7951 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:54.225898  7951 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:54.227034  7954 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.817s	user 0.004s	sys 0.000s
I20260709 13:03:54.227622  7951 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:54.229400  7943 raft_consensus.cc:3055] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:54.268585   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:03:54.269138  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 171546us
I20260709 13:03:54.279072  1533 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:54.281311  1533 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:54.297435  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:54.299499  1567 raft_consensus.cc:2368] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 20. Current term is 21.
I20260709 13:03:54.301268  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:54.302124  1567 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:54.303486  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:54.304329  1567 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:54.304872  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:54.306375  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:54.311108  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:54.311997  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1106) took 1852 ms. Trace:
I20260709 13:03:54.313035  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:54.308427  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:54.314226  1563 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:54.316057  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3208) took 1285 ms. Trace:
I20260709 13:03:54.316461  1562 rpcz_store.cc:276] 0709 13:03:53.030502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.673630 (+643128us) service_pool.cc:224] Handling call
0709 13:03:53.687326 (+ 13696us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.716801 (+ 29475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:53.716848 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:53.718334 (+  1486us) log.cc:844] Serialized 26 byte log entry
0709 13:03:53.764515 (+ 46181us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:53.764576 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.315290 (+550714us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b54002a9754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:54.315380 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:54.315455 (+    75us) raft_consensus.cc:1685] finished
0709 13:03:54.315468 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.315871 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":17623,"spinlock_wait_cycles":66812288,"thread_start_us":44055,"threads_started":1,"wal-append.queue_time_us":48763}
I20260709 13:03:54.318814  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:54.320025  1562 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 16: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:54.320703  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:54.321291  1564 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:54.325311  1564 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:54.321849  1562 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:54.323283  1566 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:03:54.312471  1560 rpcz_store.cc:276] 0709 13:03:52.459313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.462235 (+  2922us) service_pool.cc:224] Handling call
0709 13:03:52.462505 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.140234 (+677729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.311363 (+1171129us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b54000a7dd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:54.311772 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1515279232}
I20260709 13:03:54.327982  1567 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:03:54.306689  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:54.337891  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 225121us
I20260709 13:03:54.326769  1561 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:03:54.343045  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.339857  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:54.343106  1561 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:54.349200  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:54.352123  1561 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:54.360772  1561 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:54.365526  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1440) took 1060 ms. Trace:
W20260709 13:03:54.367260  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 84770.1us
W20260709 13:03:54.371883  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 230165us
I20260709 13:03:54.372646  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1259) took 1704 ms. Trace:
I20260709 13:03:54.353134  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:03:54.374318  1643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 3112) took 2146 ms (client timeout 2474 ms). Trace:
I20260709 13:03:54.374528  1644 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:54.379904  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2687) took 1099 ms. Trace:
W20260709 13:03:54.391413  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 208609us
W20260709 13:03:54.391638  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 3105) took 2322 ms (client timeout 1957 ms). Trace:
I20260709 13:03:54.179682  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.214856  7818 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:54.422864  2297 raft_consensus.cc:2393] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 16: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:54.195374  2298 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:54.374132  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2607) took 3440 ms. Trace:
W20260709 13:03:54.394943  1639 rpcz_store.cc:269] 0709 13:03:52.069612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.123939 (+ 54327us) service_pool.cc:224] Handling call
0709 13:03:54.391221 (+2267282us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b540032cdd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:54.391432 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2141267712}
I20260709 13:03:54.394761  1715 rpcz_store.cc:276] 0709 13:03:53.280349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.555982 (+275633us) service_pool.cc:224] Handling call
0709 13:03:53.557350 (+  1368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.046869 (+489519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.379291 (+332422us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b54001473d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:54.379673 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1124483456}
I20260709 13:03:54.399812  7497 consensus_queue.cc:237] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.401750  7893 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:54.402521  7894 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.463956  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:54.465157  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:54.466060  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:54.468053  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:54.469053  1635 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:54.394379  1643 rpcz_store.cc:269] 0709 13:03:52.227730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.522777 (+295047us) service_pool.cc:224] Handling call
0709 13:03:54.373702 (+1850925us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b5400123c54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:54.374025 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1456185088}
I20260709 13:03:54.404307  1563 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:54.407496  7737 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:54.471117  7737 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.471172  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:54.471673  7737 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:54.408551  7978 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:54.474714  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:54.475723  1711 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:03:54.409360  7520 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:54.478096  7520 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:54.478554  7520 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:54.479174  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.479867  7520 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.482030  7520 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:54.483947  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1441) took 1177 ms. Trace:
I20260709 13:03:54.484668  1644 rpcz_store.cc:276] 0709 13:03:53.306452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.904284 (+597832us) service_pool.cc:224] Handling call
0709 13:03:54.373983 (+469699us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:54.411637 (+ 37654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.483640 (+ 72003us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b5400123c50. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:54.483843 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1057017088}
I20260709 13:03:54.417814  7869 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:54.418327  7827 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:54.197337  7518 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:54.419155  2296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1556) took 1725 ms (client timeout 2147 ms). Trace:
I20260709 13:03:54.420274  2303 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:03:54.202883  7715 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:54.428691  7812 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:54.432039  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:54.432514  7864 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:54.432615  7906 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:54.434201  7576 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:54.435585  7638 consensus_queue.cc:1059] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:54.507459  7638 raft_consensus.cc:3055] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:54.507859  7638 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:54.508926  7638 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.436331  1641 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:54.510017  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:54.438803  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.511744  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:54.439842  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.512748  2304 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:54.512508  7518 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.441149  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.513402  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:54.445572  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:54.449813  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:54.515206  1636 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:54.515903  1709 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:54.516307  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:54.442458  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:54.519438  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 58597us
I20260709 13:03:54.520128  7995 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.520903  7995 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.414449  7879 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:54.451444  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.232301  7943 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:54.515214  7880 consensus_queue.cc:1059] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:03:54.524433  7880 raft_consensus.cc:3055] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:54.524843  7880 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:54.524770  7943 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.527011  7943 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:54.511759  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:54.527765  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 96215.4us
I20260709 13:03:54.530107  7486 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:54.393669  1637 rpcz_store.cc:276] 0709 13:03:53.305214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.888136 (+582922us) service_pool.cc:224] Handling call
0709 13:03:53.888423 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:53.918783 (+ 30360us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:03:53.918825 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:53.923569 (+  4744us) log.cc:844] Serialized 26 byte log entry
0709 13:03:54.071896 (+148327us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:54.071945 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.364733 (+292788us) spinlock_profiling.cc:243] Waited 7.09 ms on lock 0x7b54001955d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:54.364792 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:54.364868 (+    76us) raft_consensus.cc:1685] finished
0709 13:03:54.364881 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.365312 (+   431us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3540,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":4080,"spinlock_wait_cycles":16297728,"thread_start_us":170602,"threads_started":2,"wal-append.queue_time_us":251177}
I20260709 13:03:54.455082  7760 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:54.531494  7760 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:54.531352  7867 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:03:54.531949  7760 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:54.533298  7760 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.533896  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.428017  7751 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:54.535185  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:54.535422  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.456089  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:54.394081  1713 rpcz_store.cc:276] 0709 13:03:52.668187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.552096 (+883909us) service_pool.cc:224] Handling call
0709 13:03:53.570810 (+ 18714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.571802 (+   992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.371976 (+800174us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b54003d9fd0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:54.372302 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41938176}
I20260709 13:03:54.422367  7887 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:54.457038  1714 rpcz_store.cc:276] 0709 13:03:50.933323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.023058 (+2089735us) service_pool.cc:224] Handling call
0709 13:03:53.023417 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:53.140736 (+117319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.373475 (+1232739us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540023a854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:54.373892 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268539776}
I20260709 13:03:54.463366  7893 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:54.485626  7872 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:54.487092  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:54.531965  1637 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:54.539463  1609 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:54.489468  7737 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:54.404551  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 88355.3us
W20260709 13:03:54.497040  2296 rpcz_store.cc:269] 0709 13:03:52.667281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:53.155017 (+487736us) service_pool.cc:224] Handling call
0709 13:03:54.392415 (+1237398us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54004af454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:54.392625 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319517568}
I20260709 13:03:54.505393  7869 raft_consensus.cc:3055] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:54.540534  2299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 1406) took 2078 ms. Trace:
I20260709 13:03:54.490222  7726 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:03:54.536805  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.542510  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:54.552225  1640 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:54.536437  7760 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:54.501797  7864 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:54.526126  7880 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.502624  7906 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:54.551301  2299 rpcz_store.cc:276] 0709 13:03:52.462324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:52.494992 (+ 32668us) service_pool.cc:224] Handling call
0709 13:03:53.811256 (+1316264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:54.068083 (+256827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:54.068112 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:54.068561 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 13:03:54.430329 (+361768us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:54.430420 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:54.430545 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:54.430621 (+    76us) raft_consensus.cc:1685] finished
0709 13:03:54.430634 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:54.540181 (+109547us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400650250. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:54.540358 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":290920,"raft.queue_time_us":257341,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":6875,"spinlock_wait_cycles":3026688768,"thread_start_us":301824,"threads_started":1,"wal-append.queue_time_us":146605}
I20260709 13:03:54.558454  1644 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:54.562217  7879 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:54.562712  7879 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.563076  7879 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:54.563621  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.567875  1635 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:03:54.552860  7869 raft_consensus.cc:740] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:54.606674  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 110477us
I20260709 13:03:54.542709  7812 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:54.557082  1533 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:54.530870  7486 consensus_queue.cc:1059] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:03:54.570348  1642 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:03:54.613559  7880 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:54.615072  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.621440  7714 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:54.622185  7864 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:54.623505  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.623594  7864 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:54.624923  7926 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:54.625397  7926 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:54.626142  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.627665  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 171762us
I20260709 13:03:54.628574  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.567597  7906 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:54.615477  7751 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.633570  7995 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:54.635320  7943 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.636061  7943 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.638720  7943 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:54.639213  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:54.639635  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 138369us
I20260709 13:03:54.640628  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.542477  7836 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:03:54.622509  7893 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:54.621488  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:54.649449  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 90529us
W20260709 13:03:54.626821  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:54.653410  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.656174  1636 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:54.641625  1682 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:54.659570  1637 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:54.661394  7399 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:54.662122  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.662566  7943 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:54.657750  7717 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:54.666797  1561 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:54.679308  1642 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
I20260709 13:03:54.666998  7545 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.663287  7943 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:54.682276  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 59179.1us
I20260709 13:03:54.683766  2305 raft_consensus.cc:2410] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:03:54.683615  7943 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:54.684058  1682 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:54.667555  7812 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:54.685397  1682 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:54.686200  7726 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:54.686964  7726 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:54.687340  7926 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.687973  7726 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:54.688599  7926 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:54.689332  6959 raft_consensus.cc:3055] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:54.689862  6959 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:54.690480  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:54.691241  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 152004us
I20260709 13:03:54.675573  7717 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.692173  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.692482  1609 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:54.693061  2300 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:03:54.693926  7717 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:54.694342  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.662853  2296 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:54.686573  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.738132  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.728783  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:03:54.726212  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.694603  7893 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.713716  1563 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:03:54.756666  2300 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:54.739411  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.723621  1532 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:03:54.807571  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 84146.8us
I20260709 13:03:54.702065  7836 consensus_queue.cc:1059] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:03:54.706223  7737 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:54.694878  6959 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:54.796537  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:54.797561  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 112269us
W20260709 13:03:54.797055  1534 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 2.147s (SENT) [suppressed 19 similar messages]
I20260709 13:03:54.695128  7726 raft_consensus.cc:697] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:54.704905  7764 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:54.725335  7995 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.087s
I20260709 13:03:54.812932  7825 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:54.833385  7825 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:54.834712  7887 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:54.834604  7825 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.836089  6959 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:54.813658  2305 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:54.838250  7726 consensus_queue.cc:237] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:54.839313  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.694636  7545 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:54.817044  7927 consensus_meta.cc:287] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.780s	user 0.014s	sys 0.004s
I20260709 13:03:54.666040  7576 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:54.823518  1637 raft_consensus.cc:2410] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:03:54.819546  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:54.830130  7399 consensus_queue.cc:1059] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:54.732813  7906 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:54.832660  7995 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:54.835251  7887 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:54.840636  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 43634.1us
W20260709 13:03:54.815477  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 119529us
I20260709 13:03:54.849822  1532 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:54.852862  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.862079  7995 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:54.858325  1636 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:03:54.872978  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.886101  7995 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.767256  7869 consensus_queue.cc:260] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.887791  7995 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:54.888360  7869 raft_consensus.cc:3060] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:54.864516  7887 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:54.858887  7927 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:54.892401  7737 raft_consensus.cc:3055] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:54.892863  7737 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:54.893579  7927 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:54.893898  7737 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:54.895453  2301 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:54.896876  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:54.864583  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:54.899354  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:54.899387  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:54.918527  2301 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:54.858819  7867 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:54.927708  2296 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
I20260709 13:03:54.937585  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:54.938661  1567 raft_consensus.cc:2410] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:03:54.942341  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:54.945137  1712 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:54.946810  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:54.947697  1564 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:54.948776  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:54.950204  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.941787  7906 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:54.952092  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:54.952961  1562 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:54.954761  2299 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:03:54.955493  1560 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:54.961078  1636 raft_consensus.cc:1240] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:54.930809  7399 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:54.963006  7399 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:03:54.964543  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 91785.2us
I20260709 13:03:54.969111  2297 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:54.961002  2299 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:54.972975  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:54.975184  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 622317us
W20260709 13:03:54.981164  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 152526us
I20260709 13:03:54.987411  2302 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:54.983243  7812 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:55.015710  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 48977.9us
I20260709 13:03:55.015929  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:55.021812  7906 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:55.024765  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:55.025799  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:55.027345  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.027509  1641 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:55.029506  2269 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:55.030138  7989 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:55.029529  7755 consensus_queue.cc:1059] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:55.027407  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:55.031005  1681 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:55.038664  7994 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
W20260709 13:03:55.042562  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:54.972249  7867 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:55.018739  1609 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:55.101167  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.048620  2301 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:03:55.053793  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:55.055876  7879 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.057353  7737 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:03:55.067215  7715 consensus_meta.cc:287] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.550s	user 0.013s	sys 0.004s
I20260709 13:03:55.074193  7996 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.075006  7893 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:55.080816  8007 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.021966  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:55.092532  1635 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:55.098453  1709 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: []
I20260709 13:03:55.099270  7755 raft_consensus.cc:3055] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:55.113550  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:55.118211  1640 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:03:55.120009  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:55.122478  2300 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:03:55.124492  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 102647us
I20260709 13:03:55.124625  2304 raft_consensus.cc:2393] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 20: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:03:55.128662  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 74964.8us
I20260709 13:03:55.115531  7715 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.129594  7995 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.130053  1569 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:55.130295  7995 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.131223  7715 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:55.123509  7812 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:55.131878  7715 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:55.132404  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 132249us
I20260709 13:03:55.132539  7715 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.132920  7995 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:55.133786  7744 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:55.135041  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:55.135456  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 165141us
I20260709 13:03:55.136293  1681 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:55.137681  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:55.126811  7714 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:55.138758  7714 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:55.139073  7864 raft_consensus.cc:3055] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:55.139298  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:55.139549  7864 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:55.140375  1718 raft_consensus.cc:2393] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:55.140729  7864 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.119170  7893 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:55.141608  6959 consensus_queue.cc:1059] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:55.142030  7864 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:55.142658  7497 raft_consensus.cc:3055] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:55.143069  7497 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:55.143661  7893 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:55.144073  7497 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.145834  1681 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:55.148574  7867 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:55.122032  7755 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:55.150377  7755 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.151947  7755 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:55.122273  7989 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:55.156100  7638 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:55.156330  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:55.157233  1709 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 20: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:55.157312  7989 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:55.131907  7812 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.158407  7812 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:55.158835  7979 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.009s	sys 0.010s
W20260709 13:03:55.159291  7979 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.010s	sys 0.010s
W20260709 13:03:55.146757  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 117350us
I20260709 13:03:55.159745  8007 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.161468  7518 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:55.162060  8007 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:55.163460  7518 consensus_queue.cc:1059] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:55.164590  7764 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:55.164988  7943 raft_consensus.cc:3055] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:03:55.165736  7943 raft_consensus.cc:740] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:55.165778  7604 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:55.165206  7764 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 22 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:03:55.165132  1683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.002s	sys 0.027s
W20260709 13:03:55.169168  1683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.003s	sys 0.027s
W20260709 13:03:55.171394  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:55.169660  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 558263us
I20260709 13:03:55.121798  7545 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:55.118821  7996 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.186106  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:55.134792  7715 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:55.189845  1711 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:55.184962  7887 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.186180  7943 consensus_queue.cc:260] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.185310  7714 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:03:55.193161  7966 consensus_queue.cc:1059] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:03:55.164606  7717 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.016s
W20260709 13:03:55.197464  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.196580  8007 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 13:03:55.202216  1642 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:55.207278  1607 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:55.222631  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 53290.1us
I20260709 13:03:55.231325  1636 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 16: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:03:55.243974  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.240933  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:55.246308  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:55.265743  1640 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:55.234329  7518 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:55.255548  7869 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:55.284369  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.236843  7717 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:03:55.257021  7943 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:03:55.276822  7518 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:03:55.291965  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 104516us
I20260709 13:03:55.293177  6959 raft_consensus.cc:3055] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:55.277298  7545 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:55.299564  6959 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:55.289073  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:55.276971  7744 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:55.300513  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 54336.2us
W20260709 13:03:55.313161  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.326438  6959 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.292964  7869 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:55.266032  7872 consensus_queue.cc:1059] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:03:55.329547  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:55.336012  1712 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:55.330051  1607 leader_election.cc:400] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 20. Current term is 21.
I20260709 13:03:55.337201  1607 leader_election.cc:403] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:55.331360  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 79789.3us
I20260709 13:03:55.291954  7926 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.041s
W20260709 13:03:55.334787  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 352574us
I20260709 13:03:55.336023  7869 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:55.369652  6959 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:03:55.373728  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 48798.4us
W20260709 13:03:55.374763  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.355504  7879 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:55.388375  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 109049us
I20260709 13:03:55.386193  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:55.282552  7764 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.373026  7996 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:55.376418  7715 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.379942  7887 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:55.407392  1561 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:03:55.410804  8004 log.cc:927] Time spent T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
W20260709 13:03:55.411011  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 81313.5us
I20260709 13:03:55.411700  7576 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:55.411922  1607 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:55.412263  7576 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:55.412734  7576 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:55.416404  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.416311  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:55.420500  1560 raft_consensus.cc:2410] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 13:03:55.429360  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.408589  7897 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 20. Current term is 21.
I20260709 13:03:55.387161  7714 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
W20260709 13:03:55.474501  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 98218.8us
I20260709 13:03:55.476686  1683 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:55.489543  7545 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:55.490770  2268 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:55.492177  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:55.415208  7879 consensus_queue.cc:1059] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:03:55.493688  7879 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:55.494109  7717 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:55.494388  7879 raft_consensus.cc:3055] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:55.494767  7879 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:55.494853  7717 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:55.496063  7880 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:55.495754  7879 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.496464  7880 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:55.497165  7879 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:55.497339  7880 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:55.498888  8010 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 13:03:55.503109  8001 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:55.506027  8018 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.506776  8018 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:55.427780  7825 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:03:55.423019  7638 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:03:55.425990  7715 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.516263  8018 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:55.517889  7715 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:55.521549  7812 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.533967  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:55.535491  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:55.535867  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 124048us
I20260709 13:03:55.536793  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:55.537760  7638 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:55.538244  1607 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:55.539500  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.539753  7996 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:55.446555  7979 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:55.531153  7576 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.561372  1712 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:03:55.565979  1561 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:03:55.538431  7638 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:55.567449  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 158196us
I20260709 13:03:55.572299  7518 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.573658  7518 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:55.581262  1560 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:55.583402  6959 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:55.586704  2303 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:03:55.588137  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:55.590533  1564 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:03:55.592653  1532 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:55.593840  7887 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:55.595145  7906 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:03:55.547025  7979 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:03:55.616680  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 140167us
I20260709 13:03:55.594549  7887 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:55.580184  7893 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:55.648156  1713 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:55.647933  6959 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:55.597157  7906 consensus_queue.cc:1243] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:03:55.620827  7979 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:55.652697  6959 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:03:55.653007  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.618925  7604 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:55.625214  7879 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:03:55.660856  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 167605us
I20260709 13:03:55.658979  7966 raft_consensus.cc:3055] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:55.649595  7818 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:55.654178  7812 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:55.668224  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:55.678771  8026 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:55.679438  8026 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:55.679852  8026 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:55.687803  1683 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:55.646355  7726 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:03:55.665251  7966 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:55.694059  7638 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:55.700284  7576 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:55.701567  1462 catalog_manager.cc:5697] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 8 to 15, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 15 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:55.697032  7869 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:55.729357  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:55.708345  7497 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:55.709506  7744 raft_consensus.cc:3055] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:03:55.715034  7887 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.716224  7867 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:55.722110  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 157451us
I20260709 13:03:55.665217  7926 consensus_queue.cc:1059] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:03:55.347724   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:55.734537  7812 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.735618  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:55.666981  7864 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
W20260709 13:03:55.741904  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 73780us
I20260709 13:03:55.688828  7879 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.674669  7604 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.752182  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:55.735445  7726 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:55.759362  2268 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:55.757761  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:55.763379  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 102938us
W20260709 13:03:55.763772  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.765120  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:55.737856  7744 raft_consensus.cc:740] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:55.742002  7893 raft_consensus.cc:697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:03:55.770156  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.793453  7744 consensus_queue.cc:260] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:55.795302  7744 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:55.796542  7893 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.799466  7638 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:55.800096  7638 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:55.800606  7638 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:55.807674  7867 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:55.809155  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:55.811172  2303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 1420) took 1332 ms (client timeout 1702 ms). Trace:
W20260709 13:03:55.811566  2303 rpcz_store.cc:269] 0709 13:03:54.478619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.536507 (+ 57888us) service_pool.cc:224] Handling call
0709 13:03:55.810995 (+1274488us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b54002a44d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:55.811116 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1885861632}
I20260709 13:03:55.812299  7869 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:55.764346  7906 consensus_queue.cc:1059] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:03:55.816023  7906 raft_consensus.cc:3055] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:55.816535  7906 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:03:55.817294  8020 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.151s	user 0.002s	sys 0.000s
I20260709 13:03:55.817708  7906 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:55.819070  1714 raft_consensus.cc:2393] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:55.890882  1644 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:03:55.772109  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 166836us
I20260709 13:03:55.766243  1717 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:03:55.778379  1718 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:55.779332  7966 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.780524  1713 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:55.762368  7726 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:55.784785  7879 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:55.786998  7604 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:55.909551  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:55.758985  7812 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:55.916787  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:55.917972  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:55.918195  1683 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:55.932816  1642 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:55.932484  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:55.940119  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:55.918865  6959 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:03:55.949061  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:55.949297  1567 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:55.952924  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:55.953910  1635 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:55.951519  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.953167  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:55.907614  7726 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:55.958055  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:55.910703  7926 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.966013  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:55.964536  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 46447.6us
I20260709 13:03:55.962002  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:55.917151  7879 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.916754  7737 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.905478  8025 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:55.975961  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 210939us
I20260709 13:03:55.971408  1644 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 16: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:55.974520  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:55.949271  7906 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:55.978597  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:55.979271  1640 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:55.979878  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:55.939524  8038 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:55.977425  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:55.984551  1638 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
W20260709 13:03:55.985757  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:55.987196  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:55.987411  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:55.987814  1567 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:55.989403  1641 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:03:55.961068  7715 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:55.957171  8035 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:55.997862  1712 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:55.997466  1713 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:55.991612  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:55.967314  8018 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:56.000473  1635 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:03:56.001983  1562 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:56.003401  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:56.006785  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:56.013489  1712 raft_consensus.cc:1275] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:56.013669  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:56.017668  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:55.964071  7714 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:03:56.025729  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:56.030081  1635 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:56.034838  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:56.035238  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:56.035693  2297 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:56.035750  2303 raft_consensus.cc:2376] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19. Re-sending same reply.
W20260709 13:03:56.039700  8031 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:03:56.040046  8026 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.040750  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:56.041342  2300 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:55.975030  7926 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.044808  7926 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:56.046289  8027 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:03:56.034655  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:56.047600  2304 raft_consensus.cc:2410] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:03:56.048347  8032 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:56.048697  7989 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:56.049242  7989 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:56.048904  8032 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.050161  7989 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:56.034456  1567 pending_rounds.cc:85] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 8
I20260709 13:03:56.051149  1567 pending_rounds.cc:107] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:03:56.051484  1567 raft_consensus.cc:2889] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:56.052343  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:56.052986  2304 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:56.055303  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:56.056481  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:56.027272  7825 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:03:56.059162  2300 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:03:56.059441  7825 raft_consensus.cc:697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:56.060495  7825 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.064014  1718 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 21: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:56.067660  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:03:56.068994  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:56.069779  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.068874  2304 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:03:55.995103  7737 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:56.028100  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:56.074524  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 97199.1us
I20260709 13:03:55.980309  7879 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.042601  8026 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:56.077872  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:03:56.124742  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 166365us
I20260709 13:03:56.125520  1533 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:56.126549  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:56.127163  2303 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:56.023831  7867 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:56.079892  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:55.982615  8025 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:56.117442  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.117504  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.120302  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:56.122295  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:56.119649  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 43841.8us
I20260709 13:03:56.011652  7604 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:03:56.156198  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:56.066818  7926 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.135212  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:56.172330  1641 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 16: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:56.157089  1711 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
W20260709 13:03:56.145797  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 68093.4us
I20260709 13:03:56.177464  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:56.157552  1643 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:03:56.078181  1713 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:03:56.179770  1715 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19. Ops: [16.12-18.13]
I20260709 13:03:56.179631  7926 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.180243  8042 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:56.180800  8042 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:56.181711  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:56.164862  1714 raft_consensus.cc:2393] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 16: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:03:56.186829  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.165133  1533 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:56.167364  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:56.168509  2300 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:56.168617  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:56.199606  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:56.064769  7715 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:56.012622  8035 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.215615  2303 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:56.182722  7926 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:56.204444  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:56.147455  8026 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:56.226482  2303 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:56.194115  7764 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:56.190269  8038 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:56.194839  8042 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:56.238054  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 111707us
I20260709 13:03:56.240717  8040 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:56.241680  8046 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:56.242166  8046 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:56.242543  8046 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:56.247021  8036 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:03:56.249248  8041 consensus_queue.cc:1048] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.056s
W20260709 13:03:56.250388  7979 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.587s	user 0.006s	sys 0.002s
I20260709 13:03:56.252533  8035 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:56.254598  7879 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:56.255856  7576 raft_consensus.cc:3055] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:03:56.256285  7576 raft_consensus.cc:740] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:56.257318  7576 consensus_queue.cc:260] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.258579  7576 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:56.259395  7867 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:56.259996  7867 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:56.260666  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 56319.8us
I20260709 13:03:56.260949  7867 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.261518  1534 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:56.261656  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:56.250407  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.263850  1712 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:56.242571  7880 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:56.190654  8023 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:03:56.202095  7545 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
W20260709 13:03:56.276283  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.232208  8026 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:03:56.321395  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 53532.2us
I20260709 13:03:56.212736  7726 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.275938  7812 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:56.218824  7893 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:03:56.265231  8035 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:56.276836  8043 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:56.276757  7864 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:56.325457  7869 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.321053  7764 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:56.352288  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 168980us
W20260709 13:03:56.367229  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.250901  7979 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:56.370522  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 172891us
W20260709 13:03:56.373848  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.369861  8032 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:56.377800  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:56.377888  1533 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:56.384521  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:56.386749  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:56.394090  1637 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:56.394852  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:56.395480  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 392291us
I20260709 13:03:56.398751  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:56.399704  1641 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
I20260709 13:03:56.328586  8038 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:56.404691  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 205213us
I20260709 13:03:56.346966  8035 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.408317  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.411805  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:56.387974  7545 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:56.390575  8043 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:03:56.401381  7497 consensus_meta.cc:287] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.659s	user 0.011s	sys 0.000s
I20260709 13:03:56.429148  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.370726  7764 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:56.385525  7864 consensus_queue.cc:1059] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:03:56.423305  8023 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:56.424815  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 304600us
I20260709 13:03:56.437386  1683 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:56.437079  8023 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.439383  1683 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:56.440574  1683 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:56.395526  7737 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:56.419445  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.441905  1714 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:56.422262  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:56.433175  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 170585us
I20260709 13:03:56.443557  1712 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
I20260709 13:03:56.447942  1709 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 22.
I20260709 13:03:56.390405  7869 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:56.394254  8032 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:03:56.457441  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:56.390228  8042 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:03:56.452447  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:56.433748  7518 consensus_meta.cc:287] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.860s	user 0.014s	sys 0.000s
I20260709 13:03:56.458189  1638 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:56.458951  1639 raft_consensus.cc:2468] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:56.463945  1534 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:56.466468  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:56.449313  7996 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.906s	user 0.009s	sys 0.003s
I20260709 13:03:56.467669  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:56.472359  1635 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:56.470885  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:56.458016  7744 consensus_meta.cc:287] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.662s	user 0.010s	sys 0.002s
I20260709 13:03:56.476730  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:56.442797  8023 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:56.490339  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:56.464738  7545 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:56.494025  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:56.494403  8035 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:56.494525  7968 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:03:56.495597  8052 log.cc:927] Time spent T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:03:56.495851  7979 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:56.496155  7864 raft_consensus.cc:3055] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:56.497298  8053 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:56.467165  7497 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:56.467502  8036 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:56.498607  1568 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:56.531431  8036 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:03:56.532510  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:56.533303  1568 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 22.
I20260709 13:03:56.534636  1568 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:03:56.449753  8043 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:56.494470  2304 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:03:56.494757  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:56.500869  1640 consensus_meta.cc:287] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.503s	user 0.012s	sys 0.000s
I20260709 13:03:56.536208  1640 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:03:56.473987  7518 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.476442  7996 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:56.509454  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 122909us
I20260709 13:03:56.534114  8036 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:56.539284  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:56.540418  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:56.510056  1561 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:56.513744  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:56.508137  7715 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:56.510363  7825 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:56.515990  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:56.517815  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:56.546150  2301 raft_consensus.cc:2376] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21. Re-sending same reply.
I20260709 13:03:56.518114  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.516212  7737 raft_consensus.cc:2764] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:03:56.550576  8062 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:56.551219  8062 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:56.550639  7518 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.551626  8062 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:56.551788  7968 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:56.552522  7638 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.751s	user 0.010s	sys 0.003s
I20260709 13:03:56.552385  7968 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:56.553064  7518 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:56.552958  7638 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.553876  8054 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:56.554044  2302 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:56.554349  8054 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.524348  8056 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.554785  8054 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:56.554733  7968 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:56.555049  8056 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.557490  8038 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:56.545130  8053 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.559267  8032 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:56.559717  8032 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:56.560096  8032 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:56.525257  8046 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.513296  7872 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:56.471490  8025 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:56.546057  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:56.509469  7887 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:56.579347  8056 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:56.579438  7887 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:56.580758  7764 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:56.582026  7887 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:56.582688  7497 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:56.583429  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.583278  7497 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:56.550168  1568 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:56.549597  8043 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.550315  1643 raft_consensus.cc:2410] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:03:56.519893  7726 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:56.591320  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:56.591981  1561 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:56.575309  1566 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:56.567180  7825 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.578589  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.517603  7864 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:03:56.598788  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.603137  2300 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
W20260709 13:03:56.606230  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.001s	sys 0.009s
W20260709 13:03:56.608249  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.001s	sys 0.009s
W20260709 13:03:56.608587  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 536062us
W20260709 13:03:56.623476  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 182973us
I20260709 13:03:56.623806  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:56.624429  1683 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:56.627154  1560 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:03:56.629323  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:56.631515  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:56.631995  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 155368us
I20260709 13:03:56.632474  1717 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 15: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:56.635179  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:56.635818  1635 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 11.
I20260709 13:03:56.638046  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:56.638505  6959 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:56.639251  1533 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:56.640401  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:56.641212  6959 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:56.641626  6959 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.641942  6959 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:56.643368  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:56.643605  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:56.578604  8053 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:56.605772  7825 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:56.638778  1568 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:56.648650  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:56.638813  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:56.650596  1681 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:56.658776  1564 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:03:56.661939  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:56.671761  1640 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:56.666308  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.667274  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:56.669420  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.649083  7726 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:03:56.673609  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 209752us
I20260709 13:03:56.675668  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.652714  8042 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:56.651798  7497 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:56.682134  1709 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:56.668709  7867 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:56.687053  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:56.687170  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.688504  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.689944  6959 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:56.684643  1717 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:56.685751  1561 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:56.672427  7869 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:56.700629  6959 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:56.701864  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:56.702822  1561 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:56.703855  8042 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:56.704397  8068 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:56.704990  8042 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.708317  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.714993  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:56.715355  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.716724  1567 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:56.722249  8035 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:56.723860  8018 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:56.724354  8018 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:56.724733  8018 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:56.677757  8061 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:56.705021  8068 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:56.728163  8068 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:56.728544  8061 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.729099  8061 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:56.715289  8073 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 13:03:56.734668  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:56.735373  1562 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:03:56.735680  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:56.743227  7638 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:56.744482  8046 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:56.745935  8074 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:03:56.746628  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 59658us
I20260709 13:03:56.747545  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:56.749357  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:56.684374  7996 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:56.751956  1564 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:56.755877  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:56.772790  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 99565.1us
I20260709 13:03:56.693300  7867 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.782822  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:56.775741  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:56.777658  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:56.777154  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:56.781252  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:56.793035  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:56.794314  1564 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:03:56.794360  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:56.798456  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:56.801505  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:56.806432  2268 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:56.807541  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:56.808246  1636 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:03:56.811807  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:56.812978  8041 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:56.784381  8025 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:56.813771  8041 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:56.814901  8041 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:56.857318  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 51233.6us
I20260709 13:03:56.691334  7864 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:56.793967  8072 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:03:56.872262  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 76900.3us
I20260709 13:03:56.897089  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:56.784889  7867 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:56.789517  8053 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 13:03:56.784353  7996 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.788225  7497 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.098s
I20260709 13:03:56.817688  7576 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:56.856418  7897 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:56.803313  7764 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:56.791689  7869 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:56.778348  7638 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 13:03:56.871256  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:56.815578  8025 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:56.809007  7880 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:03:56.893596  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:56.802889  7989 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:56.856635  8056 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:56.856858  8038 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:56.928213  2299 raft_consensus.cc:1275] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:03:56.902745  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:56.930023  1638 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 17: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:56.906700  7715 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:56.904554  1564 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:03:56.910192  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.911612  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:56.920115  1640 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:56.870677  7726 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:03:56.931234  1562 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:03:56.899647  7979 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:03:56.909687  8072 raft_consensus.cc:697] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:56.862108  2301 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:03:56.925958  8056 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.922293  7869 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:56.922091  7880 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:56.925422  7897 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:56.927853  8038 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.817523  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:56.954879  8056 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:56.944001  8070 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
W20260709 13:03:56.965090  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 147666us
I20260709 13:03:56.941291  8048 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.943086  8046 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:56.968312  2268 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:56.941974  8071 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:56.985447  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:56.959992  7880 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:03:56.955754  7869 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:56.969913  8038 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:57.003786  1566 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
W20260709 13:03:56.994503  8077 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:03:56.975287  8023 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:56.993551  7744 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:57.013208  1568 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:03:56.999075  7638 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.034310  8066 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.997283  8046 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:57.011200  1640 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
W20260709 13:03:57.039117  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 181097us
W20260709 13:03:57.034327  8076 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:03:57.065445  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:57.020614  7764 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.982604  8048 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.026401  7744 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:57.088476  2301 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:03:57.008116  8058 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.200s
I20260709 13:03:57.041795  8059 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:57.074563  8053 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:57.023387  8062 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:57.099417  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 131216us
I20260709 13:03:57.100342  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:57.026250  8023 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:57.091215  8078 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
W20260709 13:03:57.091466  8079 log.cc:927] Time spent T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
W20260709 13:03:57.112475  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 47176.8us
I20260709 13:03:57.040246  7638 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.082578  8088 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:57.126684  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:57.130228  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:57.096004  8053 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.135351  1563 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 15: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:03:57.116830  8085 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:03:57.137632  2303 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:57.137660  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:57.045023  8066 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.083195  7764 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:57.158746  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:57.165899  7906 consensus_meta.cc:287] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.183s	user 0.011s	sys 0.003s
I20260709 13:03:57.166580  7906 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:57.167618  1642 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:03:57.076860  7897 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.142158  8053 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:57.134840  8064 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:57.182528  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:57.098407  8059 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:57.187091  2303 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 15: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:03:57.182040  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 51906.8us
I20260709 13:03:57.083043  8072 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.170598  7996 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:57.195828  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:57.199018  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:57.200855  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:57.203526  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.205585  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:57.208458  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 127480us
I20260709 13:03:57.210109  8090 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.210960  8091 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:03:57.211369  8083 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
W20260709 13:03:57.212152  8089 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:03:57.189783  8025 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:57.155503  8046 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.213904  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:57.173118  7744 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.190871  8058 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:57.238757  8082 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:03:57.240960  7996 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:57.242528  7996 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:57.243086  7996 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.244405  7714 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:57.245257  8084 log.cc:927] Time spent T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 13:03:57.248108  7764 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:57.248912  8081 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 13:03:57.250519  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3240) took 1097 ms. Trace:
I20260709 13:03:57.253750  1609 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:57.253901  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:57.254563  1712 raft_consensus.cc:2368] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19.
I20260709 13:03:57.244887  7714 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:57.255160  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:57.259541  1713 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
W20260709 13:03:57.216486  7825 consensus_meta.cc:287] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.000s
I20260709 13:03:57.264648  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:57.266788  1532 leader_election.cc:400] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19.
I20260709 13:03:57.267202  1532 leader_election.cc:403] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:57.254877  1609 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:03:57.270246  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.250878  1569 rpcz_store.cc:276] 0709 13:03:56.153558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.153735 (+   177us) service_pool.cc:224] Handling call
0709 13:03:56.171392 (+ 17657us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:56.232004 (+ 60612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:56.232046 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:56.556507 (+324461us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b2c00000658. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:03:56.584699 (+ 28192us) log.cc:844] Serialized 78 byte log entry
0709 13:03:56.808288 (+223589us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:03:56.808350 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:56.808461 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:57.250220 (+441759us) raft_consensus.cc:1685] finished
0709 13:03:57.250239 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:57.250463 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45453,"raft.queue_time_us":79258,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":2738560,"thread_start_us":222350,"threads_started":1,"wal-append.queue_time_us":340011}
I20260709 13:03:57.191323  8064 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.289009  8108 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:57.293267  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:57.293992  8108 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.294106  1569 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:03:57.294602  8108 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.293273  8086 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.295631  8086 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:57.296809  8110 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:57.272814  7764 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:57.210721  8090 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.272630  7825 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.261137  8091 consensus_queue.cc:1059] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 13:03:57.280684  7812 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19.
I20260709 13:03:57.257427  7576 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.229423  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:57.360119  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 105306us
I20260709 13:03:57.362639  1609 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:57.360457  1711 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:03:57.370213  2303 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 19: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:03:57.396800  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.401826  1609 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:57.397616  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:57.413182  2303 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:57.342937  8075 raft_consensus.cc:3055] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:57.289335  7737 consensus_queue.cc:1059] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.111s
I20260709 13:03:57.352619  7764 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:03:57.425487  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:57.324715  8062 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:57.325150  8048 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:57.313915  7825 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:57.305646  8088 raft_consensus.cc:697] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:57.313375  8091 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:57.447118  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:57.400163  8066 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:57.423206  8075 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:57.374491  7576 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.473047  7638 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:57.437750  7996 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:57.474157  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:57.475200  1565 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:57.442497  8091 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:57.442291  8025 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:57.480443  2303 raft_consensus.cc:2468] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:57.471873  8046 raft_consensus.cc:3055] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:57.496520  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:57.460531  8110 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:57.523409  1713 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:57.471405  7764 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:57.497592  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:57.541569  1565 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:57.530648  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:57.521030  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:57.544486  2303 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:57.489385  7996 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:57.552465  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1282) took 1533 ms. Trace:
I20260709 13:03:57.560473  1715 rpcz_store.cc:276] 0709 13:03:56.019204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.019783 (+   579us) service_pool.cc:224] Handling call
0709 13:03:56.179473 (+159690us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:56.180250 (+   777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.552155 (+1371905us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54001c1450. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:57.552381 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264533376}
I20260709 13:03:57.527114  8104 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:57.562498  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:57.516767  8046 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:57.566457  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:57.524421  8108 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:57.577440  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:57.529450  8064 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:57.538061  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:57.589098  8039 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 1.315s	user 0.000s	sys 0.006s
I20260709 13:03:57.567384  7996 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:57.593319  1713 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:57.531607  8102 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:57.605567  7737 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:57.567488  7638 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:57.605612  8056 consensus_meta.cc:287] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.624s	user 0.009s	sys 0.009s
I20260709 13:03:57.551607  7714 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:57.568035  8090 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:57.595211  8108 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:03:57.608392  8054 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.053s	user 0.020s	sys 0.000s
W20260709 13:03:57.622819  8061 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.893s	user 0.016s	sys 0.000s
I20260709 13:03:57.559682  7764 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:57.623319  8041 consensus_meta.cc:287] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.805s	user 0.015s	sys 0.000s
W20260709 13:03:57.612979  8068 consensus_meta.cc:287] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.884s	user 0.016s	sys 0.000s
I20260709 13:03:57.634335  7638 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:57.654554  7867 consensus_meta.cc:287] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.738s	user 0.009s	sys 0.011s
W20260709 13:03:57.627487  8032 consensus_meta.cc:287] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.067s	user 0.015s	sys 0.001s
I20260709 13:03:57.621742  7737 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:57.668951  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:57.690711  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.625880  7966 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:57.662375  7880 consensus_meta.cc:287] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.640s	user 0.011s	sys 0.000s
I20260709 13:03:57.573097  8104 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.641780  8108 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.561229  8088 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.637663  8056 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.569103  8091 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.693253  7737 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:57.658558  8054 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.667702  8061 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:57.704217  8108 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:57.678586  8041 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.682122  8068 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:57.725629  8091 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:57.692847  8032 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:57.730278  7764 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:57.731721  8125 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:03:57.733579  7638 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:57.716495  8018 consensus_meta.cc:287] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.991s	user 0.011s	sys 0.003s
W20260709 13:03:57.763213  8129 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:03:57.663373  8110 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:57.763645  8018 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:57.765884  8018 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:57.737057  8035 raft_consensus.cc:3055] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:03:57.737510  8071 consensus_meta.cc:287] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.726s	user 0.010s	sys 0.007s
I20260709 13:03:57.739681  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:57.689817  7867 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:57.767257  1714 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:03:57.739583  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:57.697256  7880 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.740366  8090 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:57.772473  8090 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.773397  8041 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:57.773661  7880 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:57.752454  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:03:57.739512  8109 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:03:57.670056  8046 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.720737  7576 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:57.770783  1712 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:57.769098  8035 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:57.774890  8090 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:57.777664  8104 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:57.776993  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:57.810163  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:57.811510  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:57.812729  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:03:57.815115  8038 consensus_meta.cc:287] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.792s	user 0.012s	sys 0.009s
I20260709 13:03:57.815654  8038 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.816668  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2706) took 1225 ms. Trace:
I20260709 13:03:57.817066  1568 rpcz_store.cc:276] 0709 13:03:56.591714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.653013 (+ 61299us) service_pool.cc:224] Handling call
0709 13:03:57.010534 (+357521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:57.024014 (+ 13480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.816479 (+792465us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b5400276850. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:57.816615 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":778770688}
I20260709 13:03:57.818321  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:57.819598  1683 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:57.821108  2268 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:57.822073  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:57.821213  8112 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:57.823081  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:57.823936  7518 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:57.804409  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:57.824419  7518 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.824802  7518 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:57.825635  2305 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:03:57.829207  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.830325  8102 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:57.818507  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:57.832439  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:57.833066  1709 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:57.831086  2305 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:03:57.831290  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:57.835877  1683 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:57.778424  1569 raft_consensus.cc:2410] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:03:57.780763  2297 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:03:57.782804  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:57.787878  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1464) took 2254 ms. Trace:
I20260709 13:03:57.838383  1637 rpcz_store.cc:276] 0709 13:03:55.533831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.533974 (+   143us) service_pool.cc:224] Handling call
0709 13:03:55.534319 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.402583 (+868264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.787584 (+1385001us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b54000d1454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:57.787797 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1977005824}
I20260709 13:03:57.835940  2305 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:57.799839  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:57.799876  1642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2711) took 1849 ms (client timeout 1528 ms). Trace:
I20260709 13:03:57.789239  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1140) took 1250 ms. Trace:
I20260709 13:03:57.842661  1560 rpcz_store.cc:276] 0709 13:03:56.538987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.543821 (+  4834us) service_pool.cc:224] Handling call
0709 13:03:56.589256 (+ 45435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:56.629389 (+ 40133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.788998 (+1159609us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b54001cdc54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:57.789177 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86679808}
I20260709 13:03:57.810830  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:57.843830  1566 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 12: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:57.796147  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:57.791239  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:57.780398  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:57.793644  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1590) took 1290 ms. Trace:
I20260709 13:03:57.843487  2305 raft_consensus.cc:1240] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 22. Current term is 23. Ops: []
W20260709 13:03:57.841857  1642 rpcz_store.cc:269] 0709 13:03:55.951014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.963987 (+ 12973us) service_pool.cc:224] Handling call
0709 13:03:57.799719 (+1835732us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400155754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:57.799814 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2730337024}
I20260709 13:03:57.845322  2298 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:03:57.856076  1713 raft_consensus.cc:2393] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 16: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:57.840296  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:57.845379  2300 rpcz_store.cc:276] 0709 13:03:56.503098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.503243 (+   145us) service_pool.cc:224] Handling call
0709 13:03:57.793504 (+1290261us) spinlock_profiling.cc:243] Waited 57.2 ms on lock 0x7b540068d3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:57.793591 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131515776}
I20260709 13:03:57.857136  2298 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:03:57.857503  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:57.857993  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:57.858690  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:57.861774  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:57.863837  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:57.866675  1683 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:57.867043  1712 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:03:57.859544  1609 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:57.790095  1462 catalog_manager.cc:5697] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 13 to 16, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 16 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:57.870535  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:57.871109  8042 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:57.871850  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:57.872563  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2718) took 1488 ms. Trace:
I20260709 13:03:57.872923  1643 rpcz_store.cc:276] 0709 13:03:56.384563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.385031 (+   468us) service_pool.cc:224] Handling call
0709 13:03:57.872386 (+1487355us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5400155254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:57.872511 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352064768}
I20260709 13:03:57.874039  7755 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:03:57.874497  7755 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.784510  7867 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:57.785804  8061 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:57.875240  7897 consensus_queue.cc:1059] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:57.793007  8056 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:57.876242  7897 consensus_queue.cc:1243] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:57.793694  7996 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:57.876924  7897 raft_consensus.cc:3055] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:03:57.800913  8053 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.610s	user 0.014s	sys 0.000s
I20260709 13:03:57.877465  7897 raft_consensus.cc:740] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:57.877651  8053 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.879246  7715 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:03:57.879385  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 88260.7us
I20260709 13:03:57.803821  8032 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:57.797262  7737 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.798064  8054 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:57.881345  8048 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:03:57.881929  7737 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:57.793180  8071 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:57.884217  7755 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:57.882664  7737 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:03:57.886334  2305 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: [15.12-16.13]
W20260709 13:03:57.886723  7869 consensus_meta.cc:287] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.864s	user 0.013s	sys 0.004s
I20260709 13:03:57.887257  7869 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.887830  2305 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:03:57.888273  8098 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:57.889053  2305 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:57.872007  8042 raft_consensus.cc:3055] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:57.889516  7869 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:57.889945  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.802945  7726 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.910807  1717 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 15: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:57.799495  8068 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:57.801589  8067 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 1.132s	user 0.002s	sys 0.000s
I20260709 13:03:57.907438  2301 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:57.906301  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:57.925246  1563 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
W20260709 13:03:57.907361  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 64504.3us
I20260709 13:03:57.874755  1709 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:57.864122  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:57.872931  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:57.907150  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:57.911729  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:57.886240  8108 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.927063  1710 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:57.934037  1532 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:57.926716  7893 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:57.926707  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 62673.7us
I20260709 13:03:57.926308  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:57.929752  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:57.930248  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:57.943318  1569 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:57.943342  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:57.933812  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:57.944684  1709 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:57.939224  1718 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:57.946604  1563 raft_consensus.cc:1240] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:57.889935  8042 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:57.928130  8059 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:57.947755  1563 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:57.983373  1716 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:57.942096  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:57.890367  8104 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:57.907291  7979 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:57.891776  8036 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:57.951094  8018 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.951267  8080 consensus_queue.cc:1059] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.011s
I20260709 13:03:57.953251  8032 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:57.886241  7518 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.955039  8068 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.955649  8071 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:57.956013  1532 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:57.913479  8112 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
W20260709 13:03:58.008958  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 53016us
I20260709 13:03:57.956622  7715 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:57.957291  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:57.958359  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:57.919322  8065 raft_consensus.cc:2804] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:57.959219  8120 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:03:57.963115  8095 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:57.991775  7755 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.980245  8124 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:03:57.986860  1565 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
W20260709 13:03:57.994880  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.937424  8054 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:57.936779  7737 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:57.998061  1710 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 22.
I20260709 13:03:57.878458  7897 consensus_queue.cc:260] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.926824  7726 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:57.805812  8035 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.008157  1709 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:57.937942  8098 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:03:57.996927  8104 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:57.999514  8102 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:58.013895  1715 raft_consensus.cc:2393] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:57.861346  7966 consensus_queue.cc:1048] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 13:03:58.031025  8124 raft_consensus.cc:3055] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:58.031492  8124 raft_consensus.cc:740] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:58.035043  7518 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:58.035463  8035 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:58.013540  8114 consensus_queue.cc:1059] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.028s
I20260709 13:03:58.036317  8102 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.011763  8073 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:03:58.021579  8054 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.039427  8054 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:58.039660  8108 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:58.009575  8099 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 13:03:58.042301  7755 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:58.043110  2298 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:58.043505  1713 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:03:58.043846  1713 pending_rounds.cc:85] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 13
I20260709 13:03:58.044044  1713 pending_rounds.cc:107] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:03:58.018069  8065 raft_consensus.cc:697] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:58.044284  1713 raft_consensus.cc:2889] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:58.044973  7979 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:58.026885  7737 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.020862  8042 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.048470  8042 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:58.027163  8095 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:58.049806  8095 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.051236  7897 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:58.013219  7715 consensus_queue.cc:1243] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:03:58.052727  2302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 1427) took 1891 ms. Trace:
W20260709 13:03:58.052892  8061 consensus_queue.cc:1175] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" responder_term: 18 status { last_received { term: 17 index: 12 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:03:58.012243  7869 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:03:58.054495  7869 raft_consensus.cc:697] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:58.044978  8065 consensus_queue.cc:237] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.000346  8018 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.053151  2302 rpcz_store.cc:276] 0709 13:03:56.161576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.168355 (+  6779us) service_pool.cc:224] Handling call
0709 13:03:58.052571 (+1884216us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54007102d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:58.052670 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837950720}
I20260709 13:03:58.058167  8018 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:58.047699  1462 catalog_manager.cc:5697] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:58.055490  7869 consensus_queue.cc:237] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.023362  1532 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:58.060883  1711 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:03:58.063457  1711 raft_consensus.cc:1217] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Deduplicated request from leader. Original: 13.11->[15.12-20.14]   Dedup: 15.12->[18.13-20.14]
I20260709 13:03:58.063912  1711 pending_rounds.cc:85] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 12
I20260709 13:03:58.064142  1711 pending_rounds.cc:107] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:03:58.064383  1711 raft_consensus.cc:2889] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:58.066104  1560 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:58.022226  1681 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:03:58.067247  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:58.067919  8071 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:58.072353  1569 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:03:58.070345  8073 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:58.011014  8068 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.075662  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:03:58.085239  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 63153.1us
I20260709 13:03:58.078907  8073 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:58.080365  8110 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:58.090150  8073 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.070309  8071 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 22 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:58.091596  8073 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:03:58.094408  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 99372.9us
I20260709 13:03:58.098280  1718 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:58.103570  1561 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:03:58.096741  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:58.095031  7966 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:58.093284  7545 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:58.097033  8110 raft_consensus.cc:697] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:58.131541  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:58.126788  7966 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:58.135690  7744 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:58.136346  7966 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.116861  7737 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:58.144145  1639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2717) took 1760 ms (client timeout 1686 ms). Trace:
W20260709 13:03:58.144596  1639 rpcz_store.cc:269] 0709 13:03:56.384085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.384234 (+   149us) service_pool.cc:224] Handling call
0709 13:03:58.143962 (+1759728us) spinlock_profiling.cc:243] Waited 73.6 ms on lock 0x7b54001969d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:58.144076 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169387392}
W20260709 13:03:58.145310  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.146138  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3158) took 1036 ms. Trace:
I20260709 13:03:58.147086  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53446 (request call id 1432) took 1005 ms. Trace:
I20260709 13:03:58.147153  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.147548  2296 rpcz_store.cc:276] 0709 13:03:57.165072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.833149 (+668077us) service_pool.cc:224] Handling call
0709 13:03:57.833527 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:57.833909 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:57.833937 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:57.834472 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 13:03:58.014352 (+179880us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:58.014392 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.014465 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.146853 (+132388us) raft_consensus.cc:1685] finished
0709 13:03:58.146874 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.147011 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":25984,"thread_start_us":179508,"threads_started":1,"wal-append.queue_time_us":267801}
I20260709 13:03:58.149312  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.149585  1643 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:03:58.150156  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:58.146556  1641 rpcz_store.cc:276] 0709 13:03:57.110156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.240372 (+130216us) service_pool.cc:224] Handling call
0709 13:03:58.144467 (+904095us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.144644 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:58.144683 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:58.144724 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:03:58.144753 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.144809 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:58.145967 (+  1158us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b54001969d0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:58.146082 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2078573824}
I20260709 13:03:58.150378  8035 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:58.149387  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:58.151443  2296 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:58.151496  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:58.152384  8042 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.152103  8071 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.152942  8042 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.154073  1683 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:58.154992  7737 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:58.155560  7737 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.124848  8104 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:58.163546  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.124722  8099 consensus_queue.cc:1059] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
W20260709 13:03:58.171828  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.172031  8110 consensus_queue.cc:237] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.172995  1641 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:03:58.173287  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:58.174508  1643 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:03:58.127396  7545 raft_consensus.cc:697] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:58.175736  8035 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:58.176268  8035 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.176168  7545 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.177752  8058 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:58.178318  8058 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.180815  7893 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:58.181066  8058 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:58.181412  7893 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:58.170964  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.183856  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1591) took 1062 ms. Trace:
I20260709 13:03:58.184099  8068 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:58.184327  2301 rpcz_store.cc:276] 0709 13:03:57.121768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.121907 (+   139us) service_pool.cc:224] Handling call
0709 13:03:58.183579 (+1061672us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b54004998d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:58.183765 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1717709184}
I20260709 13:03:58.190028  8112 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:58.190682  8112 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.191362  8146 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:58.192023  8146 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.192524  8146 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:58.194957  8080 raft_consensus.cc:3055] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:58.195641  8080 raft_consensus.cc:740] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:58.196419  8099 raft_consensus.cc:3055] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:03:58.196758  8001 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:58.196936  8099 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:58.196605  8080 consensus_queue.cc:260] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.197484  8001 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.198081  8080 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:58.197962  8099 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.201341  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:58.201579  8099 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:58.198447  8075 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:58.200240  2304 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 1424) took 2039 ms (client timeout 2099 ms). Trace:
W20260709 13:03:58.203090  2304 rpcz_store.cc:269] 0709 13:03:56.160501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.167014 (+  6513us) service_pool.cc:224] Handling call
0709 13:03:58.200056 (+2033042us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b5400661554. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:58.200179 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":643052160}
I20260709 13:03:58.204517  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:58.206102  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:58.206328  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:58.207167  1643 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:03:58.207691  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:58.203027  8075 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:58.206254  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.199862  1462 catalog_manager.cc:5697] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.208659  1568 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 13: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:58.215010  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:58.215685  2304 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:58.202157  1714 raft_consensus.cc:2393] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 16: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:58.183873  1639 raft_consensus.cc:2410] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:03:58.221078  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:58.195472  8112 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:58.223730  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:58.158638  7737 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:58.204449  8001 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:03:58.232554  8106 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.774s	user 0.003s	sys 0.001s
I20260709 13:03:58.235636  8086 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:58.237635  7864 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:58.222241  7996 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:58.202891  8053 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:58.243716  8035 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:58.244500  8146 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.247673  8023 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:58.243778  1709 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:58.250869  1643 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:58.294261  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.295022  1643 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:58.250933  1714 raft_consensus.cc:2393] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 16: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:03:58.254405  1639 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:03:58.261793  1644 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:58.262187  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:58.265453  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:03:58.285727  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.268950  1462 catalog_manager.cc:5697] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 12 to 14, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 14 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.305459  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:03:58.309121  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.292296  8054 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.313731  8054 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:58.315304  1637 raft_consensus.cc:2368] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18.
I20260709 13:03:58.317195  7893 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.319478  8146 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:58.321489  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:58.322221  2303 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:58.324607  8143 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.219s	user 0.000s	sys 0.003s
I20260709 13:03:58.327132  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:58.328606  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:58.329279  2300 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:58.330277  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:58.331914  2296 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:03:58.308027  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 45949.2us
I20260709 13:03:58.333541  8110 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:58.308351  7897 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:58.336195  8066 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:58.337129  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1789) took 1029 ms. Trace:
I20260709 13:03:58.336846  8066 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:58.339349  1460 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60118 (request call id 232) took 41435 ms (client timeout 15000 ms). Trace:
I20260709 13:03:58.340503  8066 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:58.341270  8025 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.341816  8025 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.342834  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:58.343453  2302 raft_consensus.cc:2376] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19. Re-sending same reply.
I20260709 13:03:58.337464  2298 rpcz_store.cc:276] 0709 13:03:57.360286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.881691 (+521405us) service_pool.cc:224] Handling call
0709 13:03:57.882009 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:58.043500 (+161491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.336887 (+293387us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54006077d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:58.337072 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370087296}
I20260709 13:03:58.351504  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.352185  1642 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:03:58.352254  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.333846  1462 catalog_manager.cc:5697] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:58.353117  1639 raft_consensus.cc:2393] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 16: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:03:58.353109  1609 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:58.355491  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:58.351787  1533 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:58.357347  1609 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:58.358352  2298 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: [15.12-16.13]
I20260709 13:03:58.358259  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:58.304494  8041 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.360078  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.343997  8025 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:58.363336  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.294406  8095 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.361915  1641 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:58.365725  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.366557  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:58.367460  2296 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
I20260709 13:03:58.408406  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:58.369140  1462 catalog_manager.cc:5697] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 10 to 18. New cstate: current_term: 18 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.362390  8110 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:03:58.413020  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 54826.6us
I20260709 13:03:58.414598  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:58.415400  1533 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:58.291917  7996 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:03:58.417611  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:58.362051  8150 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:58.360726  7897 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:58.360239  7989 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:58.360988  7906 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:58.351186  8061 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 13:03:58.373099  2303 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:03:58.414193  8001 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:58.435009  1683 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:58.417397  8071 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:58.359661  8066 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:03:58.414212  8110 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:58.441148  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:58.441407  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:58.442135  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.419672  7737 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:03:58.447055  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.513777  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:58.406280  8023 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:03:58.372479  8041 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.481048  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.411046  8095 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.431777  7968 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:58.450408  1642 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:03:58.407680  8054 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:03:58.480183  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.480978  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.582530  7867 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:58.510368  1682 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:58.444185  7726 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:58.452390  8110 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:58.444553  7714 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:58.433969  7906 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.621495  7906 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:58.488991  8001 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:58.448226  7715 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.123s
W20260709 13:03:58.567764  8150 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 13:03:58.623598  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 375179us
I20260709 13:03:58.530562  7893 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:58.587348  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 109919us
W20260709 13:03:58.576782  8159 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 13:03:58.625622  2269 connection.cc:256] RPC call timeout handler was delayed by -1.0336419629999796s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
I20260709 13:03:58.573855  7869 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:58.625937  8041 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:58.592378  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:58.626397  2269 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:58.626567  8041 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.627023  8123 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.627151  8041 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.627516  8123 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.627945  8090 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.628621  7576 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.605965  1636 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:03:58.629446  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:58.630003  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.582638  8071 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:58.630596  1636 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:03:58.631490  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:58.631224  8071 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.631985  1636 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:58.627969  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:58.472781  1462 catalog_manager.cc:5697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 7 to 16, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 16 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.635757  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:58.636163  1462 catalog_manager.cc:5697] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 13 to 17, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 17 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.637554  7518 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:03:58.638300  7518 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:58.637998  1462 catalog_manager.cc:5697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 10 to 17, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
W20260709 13:03:58.642472  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 132264us
I20260709 13:03:58.629179  7576 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.613302  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.660404  1682 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:58.661083  1635 raft_consensus.cc:2410] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 20 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
W20260709 13:03:58.616882  8161 log.cc:927] Time spent T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:03:58.616571  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:58.640874  2269 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:58.628412  8090 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.640378  7989 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:58.665792  7714 raft_consensus.cc:697] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:03:58.667313  8090 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:58.666874  7714 consensus_queue.cc:237] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.641485  7996 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:58.645609  8080 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:58.642935  8038 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:58.626736  1644 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:58.658349  1636 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:58.646178  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:58.678951  1641 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:58.613932  7867 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.683101  1682 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:03:58.592114  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 150770us
W20260709 13:03:58.612375  8162 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:03:58.672113  1635 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:58.673053  1642 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
W20260709 13:03:58.669551  8164 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:03:58.675272  8094 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:03:58.772786  1462 catalog_manager.cc:5697] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:03:58.738323  2269 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:58.722314  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:58.771842  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:58.786461  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 103549us
I20260709 13:03:58.788463  8059 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:58.789009  8059 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:58.789464  8059 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:58.791109  8120 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:58.791882  8120 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.804541  1644 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:58.809247  7726 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:58.770431  8146 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:58.810091  7726 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.777107  8038 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:58.824889  2269 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:58.829387  1462 catalog_manager.cc:5697] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 14 to 17, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 17 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:58.830485  1462 catalog_manager.cc:5697] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 12 to 15, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 15 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:03:58.832216  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1779) took 2033 ms. Trace:
I20260709 13:03:58.832755  2297 rpcz_store.cc:276] 0709 13:03:56.798994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:56.799219 (+   225us) service_pool.cc:224] Handling call
0709 13:03:56.857354 (+ 58135us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:57.837024 (+979670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.831886 (+994862us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b54003a2354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:58.832137 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2123291904}
I20260709 13:03:58.838501  7726 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:58.339814  1460 rpcz_store.cc:269] 0709 13:03:16.903650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:16.904043 (+   393us) service_pool.cc:224] Handling call
0709 13:03:58.339123 (+41435080us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:48.171821 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:48.172505 (+   684us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:48.172559 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:03:48.172579 (+    20us) tablet.cc:662] Decoding operations
0709 13:03:48.174404 (+  1825us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:48.174496 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:48.174522 (+    26us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:03:48.577424 (+402902us) tablet.cc:701] Row locks acquired
0709 13:03:48.577472 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:03:48.577632 (+   160us) write_op.cc:270] Start()
0709 13:03:48.577779 (+   147us) write_op.cc:276] Timestamp: P: 1783602228577572 usec, L: 0
0709 13:03:48.577817 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:03:48.578546 (+   729us) log.cc:844] Serialized 7683 byte log entry
0709 13:03:48.827232 (+248686us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd79eb970ac 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b
0709 13:03:50.547343 (+1720111us) op_driver.cc:464] REPLICATION: finished
0709 13:03:50.827752 (+280409us) write_op.cc:301] APPLY: starting
0709 13:03:50.827864 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:51.908411 (+1080547us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:51.983965 (+ 75554us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:52.797689 (+813724us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:52.799895 (+  2206us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 13:03:52.800181 (+   286us) write_op.cc:312] APPLY: finished
0709 13:03:52.803164 (+  2983us) log.cc:844] Serialized 172 byte log entry
0709 13:03:53.062944 (+259780us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:53.094095 (+ 31151us) write_op.cc:454] Released schema lock
0709 13:03:53.122628 (+ 28533us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:53.123188 (+   560us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:53.123237 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":342755,"reported_tablets":42,"spinlock_wait_cycles":108288,"child_traces":[["op",{"apply.queue_time_us":280004,"apply.run_cpu_time_us":59048,"apply.run_wall_time_us":2295407,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83585,"num_ops":19,"prepare.queue_time_us":47226,"prepare.run_cpu_time_us":50086,"prepare.run_wall_time_us":1117115,"raft.queue_time_us":380693,"raft.run_cpu_time_us":26287,"raft.run_wall_time_us":237283,"replication_time_us":1938876,"spinlock_wait_cycles":10775424,"thread_start_us":1337550,"threads_started":5,"wal-append.queue_time_us":1219147,"wal-append.run_cpu_time_us":16514,"wal-append.run_wall_time_us":2807623}]]}
I20260709 13:03:58.766861  8053 consensus_queue.cc:1059] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:03:58.822053  8146 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.845984  1644 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:58.847554  1644 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:58.846869  1641 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:58.844704  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:58.848500  1644 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.852957  8104 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:58.855786  1644 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:58.844923  8094 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:58.860803  1462 catalog_manager.cc:5697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 7 to 14. New cstate: current_term: 14 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:58.848910  8146 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:58.846894  8095 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:58.848410  8123 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:58.853440  8046 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:58.857308  8104 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:58.880439  1462 catalog_manager.cc:5697] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 9 to 12, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 12 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:03:58.863626  8041 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:58.891423  1641 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:58.882787  7576 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:58.882504  8104 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:58.884722  7867 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:58.899324  1462 catalog_manager.cc:5697] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 9 to 16, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 16 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:03:58.908121  1644 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:03:58.908743  1644 pending_rounds.cc:85] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 11
I20260709 13:03:58.909010  1644 pending_rounds.cc:107] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 13:03:58.908250  1462 catalog_manager.cc:5697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 10 to 11, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 11 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:58.909291  1644 raft_consensus.cc:2889] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:58.923439  8038 raft_consensus.cc:3055] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:03:58.894183  8123 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.910282  1462 catalog_manager.cc:5697] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 10 to 15, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 15 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:58.910436  8099 consensus_meta.cc:287] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.708s	user 0.007s	sys 0.005s
W20260709 13:03:58.930269  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 76453.4us
I20260709 13:03:58.926668  8038 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:58.943926  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:58.943140  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:58.941004  2269 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:58.943131  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:58.943746  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:58.945372  1714 raft_consensus.cc:3060] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:03:58.945461  1711 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:58.927878  7864 consensus_meta.cc:287] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.638s	user 0.009s	sys 0.002s
I20260709 13:03:58.944840  1563 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:58.943316  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:58.946532  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:58.946966  1569 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:03:58.928893  8072 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:58.944952  1713 raft_consensus.cc:2410] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:03:58.931429  8123 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.951249  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:03:58.951990  1568 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 15: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:03:58.952962  8123 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:58.954978  8072 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:58.957336  8072 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:58.957685  8110 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.960299  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.947207  1718 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:58.928434  7906 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:58.963002  8041 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:58.963639  8041 raft_consensus.cc:697] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:58.963749  7989 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:58.964121  7887 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:58.964398  7989 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.964692  8041 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.927726  8071 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:58.945762  1534 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:58.962265  7906 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:58.971735  1569 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:58.975097  1718 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:03:58.946376  2269 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:58.983332  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:58.984042  1638 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:58.966555  7887 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:03:58.967051  8094 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.010s
W20260709 13:03:58.992113  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 45938.5us
I20260709 13:03:58.991770  1563 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:58.974941  8097 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:58.997066  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:58.977095  7869 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.999245  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:58.968381  8048 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.999552  1569 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:59.005699  1638 raft_consensus.cc:1275] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:03:59.002195  8072 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:59.009006  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:59.009598  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:59.009807  1718 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:03:59.009972  1569 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:03:58.991811  1711 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:03:59.014844  7518 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.005403  8097 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.015403  7518 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.017292  7518 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:59.017357  8097 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.019071  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.019613  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:59.001279  2269 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:59.020637  1718 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:03:59.022810  1713 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:59.019701  1638 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 23.
I20260709 13:03:59.023824  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.006095  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:59.027585  2302 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:03:59.029366  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:03:59.029976  1566 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:03:59.030233  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:59.000435  8173 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:59.036681  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.007032  8094 consensus_queue.cc:1243] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:03:59.032088  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:59.036681  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:59.020417  8119 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 1.400s	user 0.000s	sys 0.020s
I20260709 13:03:59.044773  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.035230  7518 raft_consensus.cc:2804] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:59.044699  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.046568  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:59.007206  7869 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:59.048658  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:59.049398  2300 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:03:59.056514  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:59.049572  2302 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:03:59.051712  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.038029  2305 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 15: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:59.001828  8071 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.047935  2297 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:59.054541  2304 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:59.008186  8048 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:59.054500  1638 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:03:59.060640  1564 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:59.060037  1714 raft_consensus.cc:2468] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:03:59.061638  2299 raft_consensus.cc:2368] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17.
I20260709 13:03:59.061676  2300 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 23: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:59.006985  8059 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.069275  8110 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.072978  7989 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.030732  8038 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.075300  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.075726  8038 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:59.076344  8146 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:59.050016  8173 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:59.079671  8146 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:59.080687  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3156) took 2263 ms. Trace:
I20260709 13:03:59.102296  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.096716  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:59.102878  1635 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:03:59.096024  1533 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:59.084302  7869 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:59.095152  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:59.100610  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 48757.7us
I20260709 13:03:59.103621  2302 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: [15.12-16.13]
I20260709 13:03:59.102129  1640 rpcz_store.cc:276] 0709 13:03:56.816876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.069857 (+252981us) service_pool.cc:224] Handling call
0709 13:03:57.070308 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:57.070647 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:57.070669 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:57.098026 (+ 27357us) log.cc:844] Serialized 26 byte log entry
0709 13:03:57.441122 (+343096us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:57.441182 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:57.441273 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:58.912626 (+1471353us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:03:59.080335 (+167709us) raft_consensus.cc:1685] finished
0709 13:03:59.080359 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.080604 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15652,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":21097,"spinlock_wait_cycles":5202176,"thread_start_us":272353,"threads_started":1,"wal-append.queue_time_us":418416}
I20260709 13:03:59.081558  8059 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.113049  2299 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:59.109273  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:59.116755  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:59.121023  2268 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:59.123600  8168 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.111083  8046 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:59.112273  8165 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:03:59.126945  1533 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:59.133205  8094 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:03:59.125032  7715 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:03:59.133857  2268 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:59.117305  7887 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:03:59.139381  7518 raft_consensus.cc:697] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:59.139741  8184 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:59.144433  8181 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:59.112188  8110 consensus_queue.cc:1059] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
W20260709 13:03:59.167856  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.137704  8046 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:59.160310  2297 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:59.127801  8147 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:03:59.165655  8181 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.179073  8181 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:59.170418  8064 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:59.174904  8046 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:59.172587  8094 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:59.137058  8168 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:59.196736  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 62947.4us
I20260709 13:03:59.199784  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:59.205595  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:59.206503  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:59.210175  2297 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: [15.12-16.13]
I20260709 13:03:59.212949  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:59.192469  8064 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.214756  7518 consensus_queue.cc:237] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.162007  8173 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.228847  2298 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:59.215018  8184 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:03:59.215427  7576 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:59.218801  8064 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:59.268677  8168 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.245949  8184 raft_consensus.cc:697] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:59.271113  8184 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.242566  7576 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.287124  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.291258  8097 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.243395  8041 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:59.293829  8097 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.293334  1642 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:03:59.219835  8104 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.329286  7576 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:59.374447  7576 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.384701  8097 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.379004  7576 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:59.387298  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:59.388109  2305 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:03:59.391921  8124 consensus_queue.cc:260] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:59.393426  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.394151  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:59.394941  2299 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 15: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:03:59.395396  8042 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.395583  2303 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:03:59.396237  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:03:59.396706  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:59.397297  2301 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:03:59.393460  8124 raft_consensus.cc:3060] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:59.378427  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 3267) took 1174 ms. Trace:
I20260709 13:03:59.402067  1564 rpcz_store.cc:276] 0709 13:03:58.203689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:58.204230 (+   541us) service_pool.cc:224] Handling call
0709 13:03:59.378120 (+1173890us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54002c5954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:03:59.378290 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1967075200}
I20260709 13:03:59.403254  7497 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:59.404088  7497 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:59.406167  7497 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:59.410151  8048 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:59.412212  2302 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:59.415196  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.415606  1609 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:59.417351  7545 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:59.417927  7545 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:59.420190  7545 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:59.420846  7545 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:03:59.291967  8094 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.424059  8123 raft_consensus.cc:3055] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:59.424480  8123 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:59.364239  8104 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:59.364398  8149 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 1.029s	user 0.004s	sys 0.000s
I20260709 13:03:59.428241  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:59.429164  1714 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 12.
I20260709 13:03:59.431529  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:59.433308  1714 raft_consensus.cc:1240] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: []
I20260709 13:03:59.436261  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.437201  1640 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:03:59.439999  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1146) took 2241 ms. Trace:
I20260709 13:03:59.443208  7497 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.443957  7497 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:59.440639  1567 rpcz_store.cc:276] 0709 13:03:57.120594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.214884 (+ 94290us) service_pool.cc:224] Handling call
0709 13:03:57.215262 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:57.889226 (+673964us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:03:57.889543 (+   317us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.270629 (+381086us) log.cc:844] Serialized 14 byte log entry
0709 13:03:58.290530 (+ 19901us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:58.308896 (+ 18366us) log.cc:844] Serialized 26 byte log entry
0709 13:03:58.309095 (+   199us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:58.309143 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:58.918074 (+608931us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b1800273340. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:03:59.202801 (+284727us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b1800273340. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb4522 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:03:59.316645 (+113844us) spinlock_profiling.cc:243] Waited 960 us on lock 0x7b1800273340. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb822a6 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419
0709 13:03:59.318651 (+  2006us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54003e7e54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:03:59.318737 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:59.318811 (+    74us) raft_consensus.cc:1685] finished
0709 13:03:59.318832 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:59.361657 (+ 42825us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26561,"prepare.queue_time_us":1263,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":95,"raft.queue_time_us":82417,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":33786,"spinlock_wait_cycles":1613829632,"thread_start_us":338181,"threads_started":1,"wal-append.queue_time_us":335616}
I20260709 13:03:59.452548  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:59.459822  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:03:59.460731  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:59.463024  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.465301  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.359549  8041 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.470641  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:59.474289  7497 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:59.448701  8199 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:59.476668  8062 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:59.477180  8062 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.477558  8062 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:59.478250  1638 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:03:59.479025  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.479411  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:59.479921  1638 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:59.481441  1643 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:59.482785  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.483729  1639 raft_consensus.cc:1275] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:03:59.487586  1639 pending_rounds.cc:85] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 12
I20260709 13:03:59.487681  1641 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 23.
I20260709 13:03:59.488054  1639 pending_rounds.cc:107] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:03:59.488377  1639 raft_consensus.cc:2889] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:59.489943  1718 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:03:59.448385  8099 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.453528  7714 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:59.475684  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:59.494339  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:03:59.456255  7545 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:03:59.472189  8053 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.498837  8118 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:03:59.473097  8041 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:59.500455  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:59.477576  7818 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:59.473732  8147 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:59.499032  8135 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:03:59.499536  8036 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.525981  1718 raft_consensus.cc:1275] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:03:59.507505  7887 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:59.499258  7715 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:59.485148  8181 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.537508  7545 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.538666  8062 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.542017  8046 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.492617  8123 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.512586  8102 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.513671  7714 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.512614  8053 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.515558  7818 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.635517  1638 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:03:59.605396  8146 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.640308  8146 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:59.642863  8146 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:59.643811  8064 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:59.636993  8097 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:03:59.510406  8099 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.648725  7818 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:59.568275  8036 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.604636  8065 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:03:59.656248  8065 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.656775  8065 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:59.663028  8071 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.654827  7906 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:59.657006  8174 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.638s	user 0.007s	sys 0.002s
I20260709 13:03:59.664541  8099 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.655030  8064 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:59.667245  8191 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.406s	user 0.007s	sys 0.000s
I20260709 13:03:59.669474  8036 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:59.671850  8038 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.596s	user 0.004s	sys 0.007s
I20260709 13:03:59.606194  8112 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:59.673476  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 460591us
I20260709 13:03:59.674590  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:59.678138  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:03:59.528981  7887 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.688011  2298 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:03:59.690817  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:59.652913  8123 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:03:59.693297  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.675119  8097 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.701433  8046 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.569399  8135 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.703866  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:59.704671  1711 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 23.
I20260709 13:03:59.705518  1714 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:03:59.706604  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:59.701594  8097 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.604300  7715 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.526540  8147 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.689076  7576 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:03:59.690188  7887 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:59.711506  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:59.724328  1644 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:59.713806  1636 raft_consensus.cc:1217] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Deduplicated request from leader. Original: 13.10->[15.11-16.12]   Dedup: 15.11->[16.12-16.12]
I20260709 13:03:59.713725  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:03:59.725791  1636 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17. Ops: [15.11-16.12]
I20260709 13:03:59.680285  8192 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:03:59.728966  8135 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.729580  8192 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.729943  8200 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:03:59.730185  8192 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.721378  1709 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:03:59.731930  8181 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:59.734301  8147 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.735937  7715 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.736801  8053 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:03:59.722051  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:59.723486  8036 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.738165  8036 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:59.740031  7714 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.740289  8036 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:03:59.743595  1635 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.018s	sys 0.043s
I20260709 13:03:59.749636  1638 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:03:59.680078  8112 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.773800  1644 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:03:59.774775  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:03:59.775643  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.776872  1709 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:03:59.735960  8062 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.784813  8192 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:59.787750  8120 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:03:59.837433  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.843487  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:59.844177  1709 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:03:59.738696  8041 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.778905  7576 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:03:59.855134  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:03:59.856046  1715 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:03:59.858377  1532 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:03:59.845709  8171 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:03:59.871246  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:59.873804  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:59.865775  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:03:59.875263  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:03:59.878741  8168 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:59.880400  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:59.881354  1638 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:59.883154  8112 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.884855  8041 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:59.886521  7576 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:59.887372  1534 leader_election.cc:336] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.815s (SENT)
I20260709 13:03:59.887801  1534 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:03:59.890111  8072 consensus_queue.cc:1059] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:59.890403  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:03:59.876379  8035 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:03:59.898783  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.899802  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:59.875913  7715 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:03:59.905936  1681 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:59.908946  1534 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.837s (SENT) [suppressed 8 similar messages]
I20260709 13:03:59.913057  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:03:59.913919  2298 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:03:59.929045  8071 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:03:59.943356  8112 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:03:59.944024  8112 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:03:59.944517  8112 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:03:59.911437  7715 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.946631  8065 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.947592  7715 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:03:59.951798  8072 consensus_queue.cc:1243] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 17.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:03:59.952415  8072 raft_consensus.cc:3055] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:59.952739  8072 raft_consensus.cc:740] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:03:59.953491  8072 consensus_queue.cc:260] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:03:59.955924  8165 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.956511  8165 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:03:59.954670  8072 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:59.908914  1638 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:03:59.966114  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:03:59.967809  1644 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:59.976678  7497 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.977495  7497 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:03:59.990751  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 69384.5us
I20260709 13:03:59.991833  1532 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:03:59.992511  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 88785.8us
I20260709 13:03:59.992646  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:03:59.993594  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.997906  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:00.000803  1640 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.024s	sys 0.042s
W20260709 13:04:00.002238  1640 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.024s	sys 0.042s
I20260709 13:04:00.006002  1644 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:03:59.958738  8165 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:03:59.998744  1635 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:03:59.911540  8046 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:03:59.999701  1609 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:00.015406  1532 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:00.000392  1533 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:03:59.892556  7818 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.017324  7497 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:03:59.895792  8035 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:00.010548  8171 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:04:00.024662  8046 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.010654  8105 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:04:00.021111  7764 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:00.039335  1718 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:04:00.040189  1637 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.016s	sys 0.013s
W20260709 13:04:00.042191  1637 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.016s	sys 0.013s
I20260709 13:04:00.040373  8042 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:00.019273  7869 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:00.045197  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.016s	sys 0.014s
W20260709 13:04:00.045620  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.016s	sys 0.014s
I20260709 13:04:00.048309  7497 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.023635  8165 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:00.052680  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:00.011178  8048 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:04:00.054841  1714 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
W20260709 13:04:00.020423  8180 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.010s	sys 0.019s
I20260709 13:04:00.058388  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:00.044103  7764 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.059415  1714 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:00.048969  7497 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.063504  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:00.034598  8064 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:04:00.020713  7715 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.037691  7887 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:00.044936  8042 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:00.093398  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:00.058789  8180 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.010s	sys 0.019s
I20260709 13:04:00.054606  8165 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:00.087230  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:00.095352  7497 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:00.096256  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:00.100512  1709 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:04:00.098076  1718 raft_consensus.cc:1240] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:04:00.067041  7764 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:00.090273  8173 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.011s	sys 0.032s
I20260709 13:04:00.103230  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:00.106074  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:00.107018  1711 raft_consensus.cc:2393] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:00.107519   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:00.069599  7818 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:00.102015  8165 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:00.125600  1718 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22. Ops: []
W20260709 13:04:00.105194  8173 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.011s	sys 0.032s
I20260709 13:04:00.135317  8105 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:00.135988  8105 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:00.137084  8105 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.138840  8105 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:00.164065  1713 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.012s	sys 0.023s
W20260709 13:04:00.167404  1713 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.012s	sys 0.023s
I20260709 13:04:00.170473  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:00.171694  1711 raft_consensus.cc:2368] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17.
W20260709 13:04:00.172317  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 457650us
I20260709 13:04:00.174968  1718 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:00.177553  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:00.179405  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:00.180950  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:00.160092  8104 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.183218  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:00.187099  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:00.187947  1643 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:04:00.164435  8035 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:00.190011  7518 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:00.135396  8042 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:00.189327  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:00.181975  8065 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:00.197413  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:00.209933  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:00.197527  8184 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.013s	sys 0.025s
I20260709 13:04:00.189863  8123 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:00.217108  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:00.201272  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:00.234411  1711 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:04:00.227351  2269 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:00.231971  1682 leader_election.cc:400] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18.
I20260709 13:04:00.237368  1682 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:00.237711  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:00.230191  8053 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:04:00.234711  8184 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.015s	sys 0.030s
I20260709 13:04:00.210788  7518 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.246009  8192 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:00.227732  8201 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 13:04:00.190258  8104 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.257815  7497 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:00.237231  8173 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:00.266304  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 251003us
I20260709 13:04:00.267041  7497 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:04:00.254444  1683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.007s	sys 0.019s
W20260709 13:04:00.268062  1683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.007s	sys 0.019s
W20260709 13:04:00.268393  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 774141us
I20260709 13:04:00.252969  7818 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:00.270403  7715 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:00.291038  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 274961us
I20260709 13:04:00.265830  8053 raft_consensus.cc:697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:00.294080  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:00.305768  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.293643  1532 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:00.304725  1635 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:00.267633  8192 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:00.273742  8046 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:00.316475  1532 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:00.326323  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:00.330533  1712 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:00.336618  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:00.337332  1712 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:00.337577  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:00.339346  2304 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:04:00.340078  2304 pending_rounds.cc:85] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 11
I20260709 13:04:00.340378  2304 pending_rounds.cc:107] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:04:00.340719  2304 raft_consensus.cc:2889] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:00.312860  8048 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:04:00.345211  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:00.304400  8110 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.346051  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:00.350946  1711 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:00.347160  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.348536  1709 raft_consensus.cc:2393] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:00.349018  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:00.354779  1718 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:00.315266  7715 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:00.350327  1717 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:04:00.356626  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:00.364656  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:00.365484  1710 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:00.327991  7867 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.381379  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:00.383057  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:00.384037  1639 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:00.345531  8064 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:00.327826  8097 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:00.349759  8046 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.374349  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:04:00.369479  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.405920  1710 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:00.441799  1712 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
W20260709 13:04:00.429239  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 72052.6us
I20260709 13:04:00.448392  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.448698  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.360785  8062 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:00.446795  1711 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
W20260709 13:04:00.452128  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.347684  8072 consensus_queue.cc:1059] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:04:00.372759  8048 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:04:00.411541  8208 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
W20260709 13:04:00.411031  8209 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:04:00.408562  8104 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:00.456344  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.456914  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.460268  1711 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:04:00.410354  8206 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 13:04:00.461016  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.533370  2302 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:00.534713  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.534832  1711 raft_consensus.cc:1275] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:04:00.443463  7545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.010s	sys 0.040s
I20260709 13:04:00.461202  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:00.407265  8205 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 13:04:00.463642  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.521660  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.536427  1711 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:00.535363  2302 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:00.397248  8064 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:00.537686  2305 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:00.413452  8071 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.396334  7867 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:00.415647  8212 log.cc:927] Time spent T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:04:00.463508  8048 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:00.530270  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.408721  8221 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:00.366993  8110 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.370343  7715 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.533283  2299 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:04:00.525766  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.543676  1710 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:04:00.451552  8046 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.536971  7869 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.535146  8210 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.457687  8102 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:00.401003  8097 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:00.542589  8064 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:00.544507  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:00.538780  7545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.010s	sys 0.041s
I20260709 13:04:00.551838  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:00.551828  1534 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.024s	sys 0.036s
I20260709 13:04:00.575331  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2755) took 1327 ms. Trace:
W20260709 13:04:00.575541  1534 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.024s	sys 0.037s
W20260709 13:04:00.576366  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 667444us
I20260709 13:04:00.560602  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:00.577186  2300 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:00.554719  2297 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 20: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:00.527369  8062 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:00.566017  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:00.575771  1637 rpcz_store.cc:276] 0709 13:03:59.247831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.254397 (+  6566us) service_pool.cc:224] Handling call
0709 13:03:59.426622 (+172225us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:59.445486 (+ 18864us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:59.445530 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:59.451791 (+  6261us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.042706 (+590915us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:00.042767 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.042850 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:00.575056 (+532206us) raft_consensus.cc:1685] finished
0709 13:04:00.575081 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.575270 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18543,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":8964,"thread_start_us":549491,"threads_started":1,"wal-append.queue_time_us":724064}
I20260709 13:04:00.579689  2302 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:00.550937  8181 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.567041  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.567890  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:00.568455  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:00.549961  8071 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.560272  7869 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.572757  8046 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:00.533351  8053 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.573662  8222 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.562032  8097 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:00.558754  8221 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:00.557673  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:00.603469  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 45902.9us
I20260709 13:04:00.603729  2297 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:04:00.575615  7518 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:00.608355  8046 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:00.616587  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.616876  8046 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:00.616271  7867 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:00.616271  8110 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:00.618856  8110 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:00.599934  7545 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:00.619463  8062 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:00.619434  8110 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:00.620745  8217 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 13:04:00.600231  8181 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.577648  2305 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 23: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:00.616206  8221 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:00.624439  8181 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:00.624898  8221 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:00.625600  8221 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:00.617878  8046 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.614822  8222 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.616206  7715 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:00.660980  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.534897  8192 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.662860  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.672320  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:00.673269  2301 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:04:00.678330  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:00.578168  8102 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:00.595458  2302 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:04:00.575918  8220 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 13:04:00.691767  7497 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.576603  8210 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:00.640360  8104 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:00.692849  7497 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:00.693763  2300 raft_consensus.cc:2468] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:04:00.639415  8110 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:00.694478  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 428792us
I20260709 13:04:00.618571  8165 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:00.699486  1640 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.026s	sys 0.046s
W20260709 13:04:00.720511  1640 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.026s	sys 0.046s
I20260709 13:04:00.721473  1640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3175) took 1261 ms. Trace:
I20260709 13:04:00.721938  1640 rpcz_store.cc:276] 0709 13:03:59.460489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.460683 (+   194us) service_pool.cc:224] Handling call
0709 13:03:59.460914 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.461062 (+   148us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:03:59.476742 (+ 15680us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.009475 (+532733us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.011953 (+  2478us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.100412 (+ 88459us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b18003c9c40. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb2ec0a 00007fd7a826e219 00007fd79ebb6a40 00007fd79ebb14e4 00007fd79ebb1455 00007fd7a1c99060 00007fd7a1cab716 00007fd7a1cbc287 00007fd7a1cbc208
0709 13:04:00.254376 (+153964us) spinlock_profiling.cc:243] Waited 7.84 ms on lock 0x7b18003c9c40. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:04:00.253182 (+ -1194us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b18003c9c40. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb4522 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:04:00.720982 (+467800us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:00.721068 (+    86us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:00.721129 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.721222 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.721388 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113209,"prepare.queue_time_us":261855,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":86,"raft.queue_time_us":93262,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":13160,"spinlock_wait_cycles":79255296,"thread_start_us":1148273,"threads_started":2,"wal-append.queue_time_us":687005}
I20260709 13:04:00.717146  1639 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:00.656523  7545 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:00.752669  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.611418  8048 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:00.767809  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 393604us
W20260709 13:04:00.766003  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 354001us
I20260709 13:04:00.769472  7497 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:00.770017  1533 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:00.775843  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.781875  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.782820  1569 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:00.784510  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.786427  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.719952  8104 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:00.789793  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.793391  1569 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
W20260709 13:04:00.818953  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 388031us
I20260709 13:04:00.819437  1566 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 22 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:00.822832  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:00.823256  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.823731  1568 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:04:00.820474  1532 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:00.825016  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:00.825904  1562 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:00.829068  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.830127  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.832516  1565 raft_consensus.cc:2393] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:00.831130  1564 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 13: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:00.788636  7715 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:00.836248  1561 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:00.836632  8228 consensus_queue.cc:1048] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:00.838212  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:00.839061  8215 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:04:00.841019  8165 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:00.855814  1568 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:04:00.841454  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:00.862481  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.863372  1566 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 23: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:00.864411  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.831796  8147 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:00.848918  7764 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.727s	user 0.011s	sys 0.000s
I20260709 13:04:00.849699  8064 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:00.851442  8218 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:04:00.802773  8048 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:00.819202  7818 consensus_meta.cc:287] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.543s	user 0.006s	sys 0.005s
I20260709 13:04:00.920236  8072 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.921088  8072 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:00.923930  8072 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:00.854581  1569 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:00.865257  1561 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:00.848168  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:00.938735  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:00.853219  8174 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:04:00.896076  1562 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:04:00.942526  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.905063  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.944521  1564 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:00.852948  7715 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:00.911029  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.938875  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.947302  1569 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:00.948541  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.950253  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.951750  1567 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 19: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:00.940294  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.907569  8220 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:00.903466  8221 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:00.908842  8165 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:00.982105  1569 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:00.983203  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:00.964205  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:00.956490  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:00.985809  1562 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:00.986068  1561 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 13: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:00.965152  1683 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:00.966756  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.987084  1566 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:04:00.995620  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.996526  1566 raft_consensus.cc:2410] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:04:00.942299  7818 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:01.000739  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2758) took 1347 ms. Trace:
W20260709 13:04:01.006388  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.958422  8071 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:00.994891  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:01.001787  8225 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:04:01.011544  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.954257  7715 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:01.017485  1716 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:04:01.018150  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:00.905553  8147 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.005278  1713 rpcz_store.cc:276] 0709 13:03:59.652249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.652406 (+   157us) service_pool.cc:224] Handling call
0709 13:03:59.652733 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:59.652998 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:04:00.167943 (+514945us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.575299 (+407356us) log.cc:844] Serialized 14 byte log entry
0709 13:04:00.999085 (+423786us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:00.999139 (+    54us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:00.999186 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.999247 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:00.999350 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":662602,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":13235,"raft.queue_time_us":450599,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":25361,"spinlock_wait_cycles":34058496,"thread_start_us":1344744,"threads_started":3,"wal-append.queue_time_us":601186}
I20260709 13:04:00.909582  7764 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.012194  1569 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:04:01.019053  1566 raft_consensus.cc:2410] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
W20260709 13:04:01.000126  8112 consensus_meta.cc:287] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 1.055s	user 0.012s	sys 0.004s
I20260709 13:04:01.016773  1562 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:01.025908  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:01.026660  1562 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
W20260709 13:04:01.019611  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 296801us
I20260709 13:04:01.027940  1682 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:00.995616  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:01.021629  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 125699us
I20260709 13:04:01.021733  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:01.022027  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:01.033973  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:01.034948  1562 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:04:00.998893  8239 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:01.006160  8220 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:01.011983  8229 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.993517  8165 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.049995  1534 leader_election.cc:400] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17.
I20260709 13:04:01.050572  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:01.052771  1534 leader_election.cc:403] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:01.039469  8071 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.050302  8235 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.007s
W20260709 13:04:01.057220  8227 log.cc:927] Time spent T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:04:01.057694  8243 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:01.058044  8233 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:04:01.058254  8243 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.058887  8036 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:01.059473  8036 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:01.060616  8243 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.061152  8243 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.061510  8036 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:01.061614  8243 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:01.033679  8112 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:01.063083  1714 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:04:01.064628  8112 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:01.066299  7869 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.066684  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.067641  8238 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:01.069540  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.056931  8071 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.070576  8241 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:01.072379  8071 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.077313  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.078514  8072 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:01.080170  8064 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.080230  7887 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:01.080782  8223 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:01.083830  8038 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.084411  8038 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.086557  8038 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:01.087616  8214 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:01.040812  8210 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:01.040491  8102 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.086311  1534 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:00.952953  8104 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.087186  8201 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:01.096052  8224 log.cc:927] Time spent T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.337s	user 0.000s	sys 0.001s
W20260709 13:04:01.091077  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 317372us
I20260709 13:04:01.146209  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.094848  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:01.096493  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 66331.6us
I20260709 13:04:01.148200  1681 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:04:01.099566  1453 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ca83cd5e36eb4e22b75011b87bfcd49d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6208c2fa49f643b5a50ab8f6d292fab2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:01.149024  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:01.149523  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 63350.3us
I20260709 13:04:01.149737  1566 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:04:01.149974  7764 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.143126  8214 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.096864  8232 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:01.151656  8232 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.151888  8165 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.152648  8165 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:01.152889  8214 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:01.153061  8165 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.153576  8165 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:01.154327  8234 log.cc:927] Time spent T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:04:01.096632  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 68825.5us
I20260709 13:04:01.155172  8201 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.156721  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.157545  8201 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:01.157948  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.115365  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.145977  8097 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.124691  1639 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:01.123152  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:01.161216  8097 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.039933  8247 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.162068  8147 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.162010  8247 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.097225  8112 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:01.163282  8112 raft_consensus.cc:697] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:04:01.094340  8105 consensus_meta.cc:287] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.920s	user 0.010s	sys 0.000s
I20260709 13:04:01.164296  8112 consensus_queue.cc:237] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.165807  1568 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:01.126737  1562 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:01.103732  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 3178) took 1641 ms. Trace:
I20260709 13:04:01.125023  8046 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:04:01.167349  1644 raft_consensus.cc:2410] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:04:01.047457  7545 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:01.130443  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.124042  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.158836  1718 raft_consensus.cc:2410] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 13:04:01.163615  8220 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:01.144970  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.125433  7869 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17.
I20260709 13:04:01.168519  1713 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:04:01.164350  8247 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.150856  1534 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.167073  1642 rpcz_store.cc:276] 0709 13:03:59.462548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:59.462686 (+   138us) service_pool.cc:224] Handling call
0709 13:04:01.103450 (+1640764us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001c7354. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:01.103647 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2593581952}
I20260709 13:04:01.173049  8048 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:01.149140  1453 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 49805.7us
I20260709 13:04:01.173306  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.174706  1565 raft_consensus.cc:2410] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:04:01.133396  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:01.151993  8232 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.049552  8239 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:01.172394  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2667) took 1107 ms. Trace:
I20260709 13:04:01.175573  1534 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.056255  8229 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.180079  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:01.180723  1562 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:01.181406  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:01.191812  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.182011  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:01.183288  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:01.193284  1637 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:01.194160  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:01.184309  1641 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:01.182783  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:01.169945  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 75212.1us
I20260709 13:04:01.204057  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:01.183987  7887 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:01.230194  1560 raft_consensus.cc:1275] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:04:01.231843  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 56357.4us
I20260709 13:04:01.232910  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:01.198791  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:01.235201  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 270123us
I20260709 13:04:01.229358  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:01.236131  1635 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:01.236579  1641 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:04:01.222182  1711 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 23.
I20260709 13:04:01.199954  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:01.237859  1639 raft_consensus.cc:2393] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:01.238826  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:01.239087  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:01.239265  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:04:01.240000  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 91899.5us
I20260709 13:04:01.240736  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:01.186565  1715 rpcz_store.cc:276] 0709 13:04:00.067610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.088810 (+ 21200us) service_pool.cc:224] Handling call
0709 13:04:00.089267 (+   457us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:00.106201 (+ 16934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.106249 (+    48us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:00.128948 (+ 22699us) log.cc:844] Serialized 78 byte log entry
0709 13:04:00.420837 (+291889us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:00.420903 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.421013 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.172166 (+751153us) raft_consensus.cc:1685] finished
0709 13:04:01.172187 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.172321 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9373,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":2187,"thread_start_us":255331,"threads_started":1,"wal-append.queue_time_us":529452}
I20260709 13:04:01.229866  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:01.236370  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:01.240810  1681 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.189074  8201 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:01.247253  1641 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:04:01.248559  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.243796  1635 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:04:01.244475  1642 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 16: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:01.250022  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:01.251060  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:01.244746  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:01.239527  1639 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
W20260709 13:04:01.245260  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 53577.2us
I20260709 13:04:01.245630  2268 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:01.253434  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:01.229259  7906 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:01.243924  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:01.264724  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:01.259550  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:01.259029  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:01.186190  8046 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:04:01.282696  1637 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:04:01.256605  2269 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:01.240120  8168 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:01.268827  1715 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:04:01.284839  1642 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:01.285647  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:01.243993  8173 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:04:01.301247  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 51226.3us
I20260709 13:04:01.302466  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.302675  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.243659  8249 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:01.302771  1569 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:04:01.305821  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:01.306253  1713 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:01.306885  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.308482  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:01.308821  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:01.313206  2296 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:01.246351  8147 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:01.296449  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:01.271636  8184 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:01.331156  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 81104.4us
I20260709 13:04:01.269749  7906 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:01.263613  7576 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:01.246282  8248 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:01.310058  8249 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.333595  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.344156  1635 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 23.
I20260709 13:04:01.275934  7545 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:01.345609  8036 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:01.350211  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:01.339488  7497 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:04:01.376329  2302 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:01.357877  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:01.376654  1635 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:04:01.259891  7518 consensus_queue.cc:1059] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
W20260709 13:04:01.359125  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 118409us
I20260709 13:04:01.298547  8201 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:01.361141  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 58816.8us
I20260709 13:04:01.258613  7764 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:04:01.359467  1683 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.382925  1635 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:04:01.362178  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:01.315057  8173 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.276120  8181 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:04:01.331995  8102 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:01.283315  8245 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:01.275595  8123 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 13:04:01.368134  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.402799  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:01.403556  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.400748  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:01.364579  8236 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:01.364851  8248 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.372179  8249 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:01.331879  8223 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:04:01.346503  8243 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:01.406258  7818 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:01.350240  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:01.377589  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:01.353160  7906 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:01.378120  2301 raft_consensus.cc:2393] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 23: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:01.366568  8230 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:01.431260  8214 consensus_queue.cc:1048] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:01.400352  8102 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.379957  1681 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:01.419448  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.389712  1565 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:01.396371  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.396723  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.348865  7715 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:01.397460  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.385798  1561 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:04:01.373529  8239 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:01.375625  7545 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:01.368208  8105 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:01.416627  8257 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:01.402472  8245 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.404711  1683 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:01.379974  8036 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:01.380434  8201 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:01.409365  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:01.397265  8173 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:01.380678  1682 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:01.445614  8230 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:01.442802  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:01.398968  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:01.408515  8120 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:01.408505  8147 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:01.446192  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:01.418303  1532 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:01.459067  7545 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:01.454398  8104 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:01.495656  2302 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 20: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:01.455956  8192 consensus_queue.cc:1059] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:01.458103  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:01.408934  8248 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.442274  1718 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:04:01.404443  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.501359  1710 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:01.427299  7576 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:04:01.465409  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 60801.6us
I20260709 13:04:01.467042  1563 raft_consensus.cc:2368] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19.
I20260709 13:04:01.446484  8102 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.444375  7872 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:04:01.474130  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 93528.7us
W20260709 13:04:01.441208  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 61347.6us
I20260709 13:04:01.495019  8236 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:01.494446  2300 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19. Ops: [16.13-18.14]
I20260709 13:04:01.437424  7518 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:04:01.463306  8105 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:04:01.498378  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 80194.1us
W20260709 13:04:01.527908  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 86909.7us
I20260709 13:04:01.528515  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.467522  8229 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.468245  8245 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:01.534056  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.503491  8120 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:01.475042  8201 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:01.454437  8184 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:01.516726  7872 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:01.523279  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.524583  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.526463  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.526854  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.527127  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.527179  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:01.542419  1711 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:04:01.543212  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:01.546119  7872 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:01.554967  7872 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:01.556977  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1811) took 1140 ms. Trace:
I20260709 13:04:01.557401  2298 rpcz_store.cc:276] 0709 13:04:00.416657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.427228 (+ 10571us) service_pool.cc:224] Handling call
0709 13:04:00.427778 (+   550us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:00.456366 (+ 28588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.456406 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:01.380528 (+924122us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b2c00047a38. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:04:01.381073 (+   545us) log.cc:844] Serialized 52 byte log entry
0709 13:04:01.466443 (+ 85370us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:01.466497 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.466591 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.556646 (+ 90055us) raft_consensus.cc:1685] finished
0709 13:04:01.556669 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:01.556916 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7031,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":75348,"spinlock_wait_cycles":3445120,"thread_start_us":60633,"threads_started":1,"wal-append.queue_time_us":154350}
I20260709 13:04:01.540023  8229 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:01.528141  8192 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:01.530010  8236 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:01.473726  7818 raft_consensus.cc:2804] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:01.509361  7887 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:01.567231  8236 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:01.567633  7887 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.568198  7887 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:01.514848  8104 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.505682  7576 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:01.535493  1681 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:01.493937  8147 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.549932  8184 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.529398  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.530273  1560 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:01.531746  8105 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:01.584841  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 55527.9us
I20260709 13:04:01.595829  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:01.583638  7906 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:01.598182  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.592691  8264 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.606525  7576 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.603593  7906 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:01.607355  8264 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.607798  7906 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:01.607834  7576 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:01.604113  8062 raft_consensus.cc:3055] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:01.609303  8062 raft_consensus.cc:740] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:01.610219  8264 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:01.611299  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 75847.6us
I20260709 13:04:01.612154  8237 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:01.612738  8237 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.613106  8237 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:01.613217  8266 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:04:01.604846  1607 leader_election.cc:400] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17.
I20260709 13:04:01.604210  8104 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:01.594283  8147 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:01.606734  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2668) took 1538 ms. Trace:
I20260709 13:04:01.612242  8184 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:01.605186  8239 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:01.527927  8112 consensus_queue.cc:1059] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:04:01.625902  8112 raft_consensus.cc:3055] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:01.626284  8112 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:01.627138  8112 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.628629  8112 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:01.629443  7545 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:01.630649  1643 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 16: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:01.631273  8223 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:04:01.610164  8062 consensus_queue.cc:260] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.635861  8062 raft_consensus.cc:3060] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:01.636955  8249 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.620191  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.622117  1714 rpcz_store.cc:276] 0709 13:04:00.068073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.093043 (+ 24970us) service_pool.cc:224] Handling call
0709 13:04:01.606536 (+1513493us) spinlock_profiling.cc:243] Waited 969 ms on lock 0x7b5400239e54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:01.606651 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2228150528}
I20260709 13:04:01.639720  1607 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.657265  8239 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:01.600040  8035 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:01.645881  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:01.598371  7818 raft_consensus.cc:697] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:01.667758  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:01.668965  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:01.642477  8223 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:01.640007  8228 consensus_queue.cc:1175] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" responder_term: 15 status { last_received { term: 15 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 15 index: 12 } }
I20260709 13:04:01.680667  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:01.683022  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 224989us
W20260709 13:04:01.687803  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 74725.5us
I20260709 13:04:01.663335  8035 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:01.551908  8036 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:01.671969  8239 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:01.697386  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:01.618957  8165 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.677160  7764 raft_consensus.cc:2764] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 13:04:01.719651  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.724606  8222 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.735306  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:01.737413  1533 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:01.703006  7715 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:01.738425  7715 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:01.739410  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:01.740929  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:01.700428  8239 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.743515  8239 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:01.744082  8053 raft_consensus.cc:3055] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:01.744556  8053 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:01.745440  8053 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.746498  8053 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:01.752454  8248 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.755254  1715 raft_consensus.cc:2393] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 21: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:01.756660  8245 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:01.758191  1644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 1501) took 1588 ms (client timeout 1763 ms). Trace:
W20260709 13:04:01.758701  1644 rpcz_store.cc:269] 0709 13:04:00.170039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.170287 (+   248us) service_pool.cc:224] Handling call
0709 13:04:01.757968 (+1587681us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b54001955d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:01.758126 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2274240384}
I20260709 13:04:01.700651  8035 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:01.709157  8123 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:01.775694  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:01.774295  2298 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:01.675150  8120 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.777390  1533 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:04:01.773278  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 68608.6us
I20260709 13:04:01.720914  7872 consensus_queue.cc:1048] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:04:01.782843  7872 consensus_queue.cc:1243] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:04:01.782946  8201 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.784281  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:01.786494  2300 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:01.768982  7715 raft_consensus.cc:697] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:01.789618  7715 consensus_queue.cc:237] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:01.721251  8236 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:01.770622  7818 consensus_queue.cc:237] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.776453  8274 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:04:01.793859  8236 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:01.796821  8165 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.798327  8201 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:01.775738  8072 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:01.808455  8072 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:01.799347  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:01.809494  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:01.799685  8097 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:01.807013  8248 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.703083  8112 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:04:01.811271  8097 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:01.816545  8097 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:01.817160  7545 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:01.817493  8112 consensus_queue.cc:1243] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:01.818038  8112 raft_consensus.cc:3055] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:01.818377  8112 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:01.819257  8112 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:01.820504  8112 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:01.821372  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:01.833633  2296 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.810194  7518 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.843794  7518 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.845628  8097 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:01.824599  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:01.827101  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:01.858755  2298 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:01.813644  2268 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:01.828042  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:01.829574  8249 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:01.832365  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:01.832878  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:01.833492  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:01.824703  7887 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:01.833813  1533 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:01.875420  1566 raft_consensus.cc:2410] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:04:01.823211  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:01.836851  1609 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.826995  8105 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.847429  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:01.851226  8245 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:01.809055  8072 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:01.860571  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 48806.1us
I20260709 13:04:01.874788  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:01.880550  1565 raft_consensus.cc:2393] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:01.880889  1718 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:01.882627  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:01.886142  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 49421.6us
I20260709 13:04:01.887400  1609 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:01.836154  8232 consensus_meta.cc:287] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.600s	user 0.010s	sys 0.004s
W20260709 13:04:01.899952  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 66229.9us
I20260709 13:04:01.901805  1711 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:01.904681  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:01.909552  1718 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:04:01.901085  1533 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:01.859515  7518 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:01.927353  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.928261  1711 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:04:01.929328  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.930505  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:01.933784  1711 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:04:01.937798  1561 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:04:01.912135  2298 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:04:01.870025  2305 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
mode: GRACEFUL
new_leader_uuid: "3f64b76180a54492bb015e405caeacec"
 from {username='slave'} at 127.0.0.1:54964
I20260709 13:04:01.939296  2305 raft_consensus.cc:606] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Received request to transfer leadership to TS 3f64b76180a54492bb015e405caeacec
I20260709 13:04:01.940062  2305 raft_consensus.cc:614] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:01.918949  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:01.940573  2296 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:04:01.941111  2304 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:01.920035  8046 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:01.920342  1609 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:01.920564  8220 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:01.921062  7764 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:01.882148  8239 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.815186  8173 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.898887  7545 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:01.876307  8038 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:01.914888  8249 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:01.965016  2268 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:01.890108  8236 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:01.956841  1609 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:01.944067  8278 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:04:01.914255  8256 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:01.960546  7872 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:04:01.982487  8257 consensus_meta.cc:287] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.000s
I20260709 13:04:01.888928  8035 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.982412  7872 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.957913  7518 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:01.909469  8232 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:01.944976  8165 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 13:04:01.990038  1638 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:04:01.994105  2304 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:04:01.958637  8223 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:01.965162  8249 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.006793   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:04:01.937526  8102 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:01.987056  1714 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:01.951735  7576 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:04:02.005872  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:02.011667  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:02.044703  1712 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 21: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:01.974581  8267 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.019843  8120 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.983032  8257 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:02.034494  1644 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:01.983958  8062 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:01.984941  8036 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:02.038025  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:02.049521  1640 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:04:02.043537  1635 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:01.996253  7764 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:01.994894  8274 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:02.053186  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:02.053956  1635 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:01.986277  8104 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:04:02.013768  7545 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.975762  8236 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:02.035853  8249 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:01.998085  8214 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:04:02.054510  8046 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 23 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:02.051673  8120 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.064036  1682 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:02.065565  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.064896  1640 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:04:02.048787  8036 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:02.016114  7518 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.059134  7764 raft_consensus.cc:697] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:02.030357  8072 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:02.125228  8105 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:02.053198  8267 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:02.117717  8237 consensus_meta.cc:287] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.500s	user 0.010s	sys 0.003s
I20260709 13:04:02.079463  8257 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.142388  1636 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:02.113767  8282 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:02.118636  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:02.193646  1640 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:04:02.181217  1532 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:02.078856  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 98232.3us
I20260709 13:04:02.188447  1635 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
W20260709 13:04:02.059813  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 132541us
I20260709 13:04:02.063504  7545 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:02.133311  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 56364.8us
I20260709 13:04:02.196439  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:02.214716  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1178) took 1274 ms. Trace:
I20260709 13:04:02.221115  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:04:02.205686  8288 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 13:04:02.192708  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 171201us
I20260709 13:04:02.227054  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:02.148363  8105 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:02.225062  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:02.194608  8097 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:02.215214  1564 rpcz_store.cc:276] 0709 13:04:00.940613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.296220 (+355607us) service_pool.cc:224] Handling call
0709 13:04:01.331466 (+ 35246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.331672 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:04:01.331941 (+   269us) log.cc:844] Serialized 14 byte log entry
0709 13:04:01.757398 (+425457us) spinlock_profiling.cc:243] Waited 9.37 ms on lock 0x7b1800434bc0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a 00007fd7a1caba2a
0709 13:04:02.214338 (+456940us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:02.214398 (+    60us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:02.214446 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.214519 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.214649 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24348,"prepare.queue_time_us":130,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":6486,"raft.queue_time_us":109506,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":5334,"spinlock_wait_cycles":21562112,"thread_start_us":431585,"threads_started":2,"wal-append.queue_time_us":638149}
I20260709 13:04:02.245651  1534 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:02.249181  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:02.213780  8035 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:02.160106  8237 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:02.247442  8062 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:02.270448  1638 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:04:02.270539  1639 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:02.237483  8220 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:02.237597  7906 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.236358  8064 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:02.270586  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:02.261603  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:02.199510  8257 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.276400  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:02.284073  1641 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:04:02.277395  7497 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:02.285190  7497 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.269348  1636 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:02.253620  8241 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:02.246896  8294 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:02.250568  8239 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:02.284518  1639 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:04:02.223618  8296 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.079s
I20260709 13:04:02.307744  1713 raft_consensus.cc:1240] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:04:02.236927  8104 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 13:04:02.335268  1641 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:02.257608  8097 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:02.312255  7497 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:02.315007  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:02.315927  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:02.316396  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2731) took 1541 ms. Trace:
I20260709 13:04:02.277153  8223 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.319398  8257 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:02.289345  8035 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:02.327378  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2779) took 1083 ms. Trace:
I20260709 13:04:02.357481  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:02.358173  1713 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:02.357682  1643 rpcz_store.cc:276] 0709 13:04:01.243902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.254255 (+ 10353us) service_pool.cc:224] Handling call
0709 13:04:02.327181 (+1072926us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b54001950d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:02.327316 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783377280}
I20260709 13:04:02.331113  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:54954 (request call id 1) took 1026 ms. Trace:
W20260709 13:04:02.331516  8284 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.261s	user 0.001s	sys 0.001s
I20260709 13:04:02.203660  8036 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.334161  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:02.370467  1635 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 19: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:02.265635  8297 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.247457  8105 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:02.342159  1563 rpcz_store.cc:276] 0709 13:04:00.775367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.775505 (+   138us) service_pool.cc:224] Handling call
0709 13:04:02.316179 (+1540674us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b54000a78d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:02.316330 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1587335424}
I20260709 13:04:02.279587  8220 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.324218  8286 consensus_queue.cc:1059] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:02.290851  8064 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:02.306295  8241 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.359372  2303 rpcz_store.cc:276] 0709 13:04:01.304544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.304701 (+   157us) service_pool.cc:224] Handling call
0709 13:04:02.330805 (+1026104us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b54003a2d54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca7735 00007fd7a1c9f9fa 00007fd7a6c674c9 00007fd7a01e3c54 00007fd7a01e3be2 00007fd7a01e3b11 00007fd7a01e3a8c 00007fd7a01e2d42 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:02.331023 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1433370880}
I20260709 13:04:02.362416  7872 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:02.250777  8046 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.349037  8102 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:02.345914  8238 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:02.288304  7906 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.266603  8295 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 13:04:02.263630  7764 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:02.370033  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 174674us
I20260709 13:04:02.361513  8097 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:02.384174  8267 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:02.361937  8302 consensus_queue.cc:1059] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:04:02.384615  8064 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:02.370350  8035 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.387871  8241 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:02.360289  8303 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:02.384457  8287 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
W20260709 13:04:02.442508  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.422765  1713 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:04:02.423647  7872 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:02.448590  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:02.448807  1453 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ca83cd5e36eb4e22b75011b87bfcd49d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6208c2fa49f643b5a50ab8f6d292fab2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:02.385239  8236 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:02.393544  8290 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:04:02.359557  8223 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:02.393211  8293 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:04:02.387141  8232 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:02.396510  8298 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:04:02.390221  8036 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:02.435091  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:02.472123  2302 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 16: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:02.379716  8220 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:04:02.425391  8289 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
W20260709 13:04:02.425103  8285 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:04:02.441110  8296 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:04:02.479682  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:02.431203  8238 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:02.438060  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:04:02.438664  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.398952  8105 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:02.407227  8173 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:02.491724  2303 raft_consensus.cc:2393] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:02.407398  8072 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:02.441022  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 179495us
I20260709 13:04:02.407683  7518 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:02.441324  2299 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:04:02.443886  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:04:02.452720  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 129061us
I20260709 13:04:02.383477  8297 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:02.449903  1681 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:02.455359  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:02.495875  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:02.457217  7872 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.461980  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:02.513540  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:02.463989  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:02.467036  1533 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:02.467319  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:02.469358  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:02.470382  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:02.473513  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:02.476933  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:02.393779  7887 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:02.459111  8302 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:02.405071  8282 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:02.481334  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:02.483184  1563 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:02.484442  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:02.485483  1566 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:02.460400  8294 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.407492  8120 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:02.509608  2299 pending_rounds.cc:85] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 14
W20260709 13:04:02.452183  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 295076us
I20260709 13:04:02.510303  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:04:02.509447  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 54163.4us
I20260709 13:04:02.504988  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:02.535224  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:04:02.592813  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 130975us
I20260709 13:04:02.593113  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:02.594074  2299 pending_rounds.cc:107] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:04:02.594509  2299 raft_consensus.cc:2889] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:02.595237  1636 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:04:02.515331  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:02.543821  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:02.558279  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:02.598021  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:02.598766  1565 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:02.478068  8062 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:02.473161  8214 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:04:02.538409  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:02.596359  2297 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:04:02.535225  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:02.543335  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:02.490553  8237 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:02.624655  1562 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:04:02.622289  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:02.627722  1714 raft_consensus.cc:2393] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:04:02.630393  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 163480us
I20260709 13:04:02.491031  8102 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:02.634426  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:02.623138  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:02.540823  8112 consensus_meta.cc:287] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.720s	user 0.011s	sys 0.000s
W20260709 13:04:02.534022  8038 consensus_meta.cc:287] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.569s	user 0.008s	sys 0.000s
I20260709 13:04:02.540372  7906 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:02.644726  8038 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:02.652172  7872 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:02.653291  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:02.653995  1569 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:02.654815  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:02.655817  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:02.621578  7518 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:04:02.657374  7518 raft_consensus.cc:3055] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:02.657724  7518 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:02.660271  8297 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:02.535475  8223 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:02.592876  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:02.639235  1563 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:04:02.639457  1716 raft_consensus.cc:2410] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:04:02.639739  1565 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:04:02.634032  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:02.602277  2269 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:02.497630  8147 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.838s	user 0.011s	sys 0.002s
I20260709 13:04:02.535876  8035 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:02.664240  2303 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:04:02.669701  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:02.598542  8282 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:02.620822  8247 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.601632  8296 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:02.668179  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 65972.2us
I20260709 13:04:02.515601  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:02.667013  1710 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 13:04:02.681448  1710 pending_rounds.cc:85] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 11
I20260709 13:04:02.681811  1710 pending_rounds.cc:107] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:04:02.682104  1710 raft_consensus.cc:2889] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:02.664723  1561 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:04:02.683812  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:02.557888  8308 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:02.670892  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:02.696565  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:02.696987  1609 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:02.697026  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:02.697751  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:04:02.698087  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 105318us
I20260709 13:04:02.667142  1636 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:04:02.699416  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:02.699823  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:02.700202  1636 raft_consensus.cc:2410] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
W20260709 13:04:02.700879  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 191744us
I20260709 13:04:02.701545  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:04:02.701974  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.705070  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:02.698486  1565 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 16: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:02.663383  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:02.623020  8236 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:02.547171  8307 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.514372  8222 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:02.602411  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 152636us
I20260709 13:04:02.628721  8053 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.065s
I20260709 13:04:02.685376  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:02.643200  8303 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:02.707399  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:02.672961  8184 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:02.718828  1561 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:02.709383  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:02.629123  8173 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 13:04:02.712704  1682 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:02.708539  1714 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 16: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:02.713968  8036 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:02.719957  1565 raft_consensus.cc:1240] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:02.692548  8223 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:02.727239  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:02.733877  1565 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:02.741607  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:02.624711  8294 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.658538  7518 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.673627  8147 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.697095  8038 consensus_queue.cc:1059] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:04:02.693560  8247 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.667506  8102 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.718116  8296 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.769479  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:02.763094  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 268325us
I20260709 13:04:02.790463  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:02.774770  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:02.718307  8112 raft_consensus.cc:2764] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:04:02.725940  8036 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.781530  1561 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:04:02.783295  1709 raft_consensus.cc:1240] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:04:02.732451  8236 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:02.751283  8248 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:02.730985  8228 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:04:02.782895  8223 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:04:02.794766  1637 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:04:02.795979  1567 raft_consensus.cc:1217] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Deduplicated request from leader. Original: 11.12->[13.13-17.15]   Dedup: 13.13->[15.14-17.15]
I20260709 13:04:02.809144  1567 pending_rounds.cc:85] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 13
I20260709 13:04:02.809434  1567 pending_rounds.cc:107] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:04:02.809744  1567 raft_consensus.cc:2889] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:02.720101  8105 consensus_queue.cc:1059] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:04:02.720399  7872 consensus_queue.cc:1059] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 13:04:02.810678  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:02.811827  1642 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:04:02.813766  8258 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:04:02.727208  8062 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:02.812511  8105 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:02.720567  8307 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.704825  8308 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:02.825403  1642 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:04:02.828374  1717 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2770) took 2488 ms. Trace:
I20260709 13:04:02.836156  1717 rpcz_store.cc:276] 0709 13:04:00.332296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.364059 (+ 31763us) service_pool.cc:224] Handling call
0709 13:04:00.400230 (+ 36171us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:00.851130 (+450900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.851177 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:00.851981 (+   804us) log.cc:844] Serialized 52 byte log entry
0709 13:04:01.192780 (+340799us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:01.253933 (+ 61153us) log.cc:844] Serialized 14 byte log entry
0709 13:04:01.349228 (+ 95295us) log.cc:844] Serialized 14 byte log entry
0709 13:04:01.364635 (+ 15407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:01.364711 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:01.431480 (+ 66769us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b1800242980. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:04:01.817579 (+386099us) spinlock_profiling.cc:243] Waited 3.97 ms on lock 0x7b1800242980. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:04:02.673355 (+855776us) spinlock_profiling.cc:243] Waited 7.49 ms on lock 0x7b1800242980. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:04:02.820023 (+146668us) raft_consensus.cc:1685] finished
0709 13:04:02.820046 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.820213 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100208,"prepare.queue_time_us":63077,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":9884,"raft.queue_time_us":474344,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":4647,"spinlock_wait_cycles":36045312,"thread_start_us":743790,"threads_started":2,"wal-append.queue_time_us":448331}
I20260709 13:04:02.838202  7872 raft_consensus.cc:3055] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:02.841415  8267 raft_consensus.cc:3055] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:02.841887  8267 raft_consensus.cc:740] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:02.842744  8267 consensus_queue.cc:260] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:02.844287  8267 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:02.844609  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:02.849155  7887 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.849766  7887 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:02.824568  8245 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:04:02.828992  8112 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.853585  8112 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.853045  1683 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:02.838675  7872 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:02.855652  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:02.856230  8238 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:02.856128  7872 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.857568  7872 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:02.857874  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:02.858608  1638 raft_consensus.cc:2368] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17.
I20260709 13:04:02.859753  1643 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19. Ops: [16.12-18.13]
I20260709 13:04:02.860417  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 1326) took 1736 ms. Trace:
I20260709 13:04:02.860783  1712 rpcz_store.cc:276] 0709 13:04:01.123717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.123825 (+   108us) service_pool.cc:224] Handling call
0709 13:04:02.860203 (+1736378us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b5400337cd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:02.860356 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2014681856}
I20260709 13:04:02.803881  8236 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.780761  8120 consensus_queue.cc:1059] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:04:02.801982  8036 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:02.799623  7764 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:02.886845  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2782) took 1491 ms. Trace:
W20260709 13:04:02.873594  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.871366  1638 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 15. (term mismatch)
I20260709 13:04:02.864425  1716 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:04:02.893033  1682 leader_election.cc:400] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17.
I20260709 13:04:02.965373  1682 leader_election.cc:403] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:04:02.824901  7518 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:02.970834  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:02.936450  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 184411us
I20260709 13:04:02.947249  1715 rpcz_store.cc:276] 0709 13:04:01.395720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.397238 (+  1518us) service_pool.cc:224] Handling call
0709 13:04:02.886626 (+1489388us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b54003395d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:02.886778 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716160256}
I20260709 13:04:02.971526  2302 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:02.965936  1638 pending_rounds.cc:85] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 14
I20260709 13:04:02.972690  1638 pending_rounds.cc:107] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 16.15
I20260709 13:04:02.973037  1638 raft_consensus.cc:2889] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:02.900863  8228 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:04:02.975334  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 82457.2us
I20260709 13:04:02.976286  2302 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:04:02.900520  8065 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:02.977568  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:02.978116  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:02.978426  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:02.990993  2304 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:02.979099  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:02.901189  8236 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:02.798899  8296 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:02.992033  2299 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 20: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:02.939831  8053 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:02.939858  8105 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:02.971767  8184 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:03.010875  2304 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:04:03.014027  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:03.029764  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:03.030841  1636 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:03.017974  7887 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:03.062779  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 82428.7us
I20260709 13:04:03.036579  8286 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:03.014523  8053 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.046023  8257 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:03.046907  8112 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.077889  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:03.014700  8105 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.063144  8038 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:03.065114  8294 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:03.088771  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 73273.5us
I20260709 13:04:03.108964  7497 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:03.109643  7497 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:03.110101  7497 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.076433  8307 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.124388  1635 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:03.083699  8173 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:03.143739  8065 raft_consensus.cc:697] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:03.147775  8065 consensus_queue.cc:237] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:03.148356  8318 log.cc:927] Time spent T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
W20260709 13:04:03.151865  8313 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:04:03.129194  8247 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.153818  8296 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:03.136610  8249 consensus_meta.cc:287] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.070s	user 0.015s	sys 0.000s
I20260709 13:04:03.155793  8249 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:03.158385  8249 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:03.129426  8307 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.159235  1563 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 21: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:03.159053  8307 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:03.161413  8307 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:03.088449  8184 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.114277  8038 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.165642  8308 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:03.166263  8038 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:03.167434  8102 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.125638  8347 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:04:03.172339  8347 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:03.172853  8347 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:03.173945  8347 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.175680  8173 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:03.176116  8053 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:03.176481  8173 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.180123  1640 raft_consensus.cc:2410] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:04:03.180763  8347 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:03.188987  8036 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.190521  8173 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:03.132011  8120 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
W20260709 13:04:03.192477  8220 consensus_meta.cc:287] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.711s	user 0.007s	sys 0.006s
I20260709 13:04:03.196398  8236 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.199473  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2789) took 1326 ms. Trace:
I20260709 13:04:03.199849  8286 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17.
I20260709 13:04:03.200367  8303 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:03.201342  8248 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:03.322041  1644 raft_consensus.cc:2368] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 20. Current term is 21.
I20260709 13:04:03.202767  8105 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.209760  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:03.340065  1643 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:04:03.209512  8282 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:03.211583  8241 consensus_meta.cc:287] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.764s	user 0.001s	sys 0.012s
I20260709 13:04:03.226490  1714 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:03.225230  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:03.229710  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.233533  7497 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.345440  1712 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:04:03.236033  8274 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:03.236706  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.237577  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.348701  1717 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:03.238437  8345 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:03.347041  1607 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.002s	sys 0.011s
W20260709 13:04:03.360515  1607 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.002s	sys 0.011s
I20260709 13:04:03.137194  8228 raft_consensus.cc:697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:03.240474  8042 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:04:03.242434  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 620641us
I20260709 13:04:03.307721  8307 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.365082  2268 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:03.251329  8165 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.261691  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:03.359433  8062 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.002s	sys 0.012s
W20260709 13:04:03.267640  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:03.376283  1711 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.002s	sys 0.018s
W20260709 13:04:03.385551  1711 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.002s	sys 0.018s
W20260709 13:04:03.213637  8064 consensus_meta.cc:287] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.769s	user 0.008s	sys 0.004s
I20260709 13:04:03.270401  8301 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:03.270987  8267 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.275045  1609 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:03.392771  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 118130us
I20260709 13:04:03.393057  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:03.290577  8147 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.294145  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1343) took 1359 ms. Trace:
I20260709 13:04:03.297101  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:03.397406  1566 raft_consensus.cc:2393] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:03.298664  1560 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:04:03.398156  1560 pending_rounds.cc:85] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 13
I20260709 13:04:03.398523  1560 pending_rounds.cc:107] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:04:03.396864  1713 rpcz_store.cc:276] 0709 13:04:01.935016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.693617 (+758601us) service_pool.cc:224] Handling call
0709 13:04:02.694015 (+   398us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:02.724572 (+ 30557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.724603 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:02.729161 (+  4558us) log.cc:844] Serialized 52 byte log entry
0709 13:04:02.998357 (+269196us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:03.293281 (+294924us) log.cc:844] Serialized 14 byte log entry
0709 13:04:03.293707 (+   426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.293819 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:03.293874 (+    55us) raft_consensus.cc:1685] finished
0709 13:04:03.293892 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.294074 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58011,"raft.queue_time_us":31241,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":8660,"spinlock_wait_cycles":43691392,"thread_start_us":469232,"threads_started":2,"wal-append.queue_time_us":293309}
I20260709 13:04:03.302449  1709 rpcz_store.cc:276] 0709 13:04:01.872623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.875314 (+  2691us) service_pool.cc:224] Handling call
0709 13:04:01.875656 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.798596 (+922940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.199123 (+400527us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b540033aed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:03.199393 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2079374080}
I20260709 13:04:03.202111  1637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1519) took 1213 ms. Trace:
W20260709 13:04:03.341286  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 256961us
I20260709 13:04:03.399624  1637 rpcz_store.cc:276] 0709 13:04:01.989097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.990287 (+  1190us) service_pool.cc:224] Handling call
0709 13:04:02.005532 (+ 15245us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.809002 (+803470us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.201784 (+392782us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b5400155254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:03.202039 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1814613376}
I20260709 13:04:03.400815  1683 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:03.184120  8264 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:03.349788  7497 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.217686  8305 raft_consensus.cc:3055] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:04:03.361178  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 570796us
I20260709 13:04:03.366205  1642 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 21: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:03.262423   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:03.367404  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:03.367995  2269 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:03.412431  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 44559.9us
I20260709 13:04:03.346169  8338 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.382004  1569 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:04:03.361707  8274 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:03.388546  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:03.374552  8307 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:03.389729  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:03.464213  2296 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:03.406379  8248 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:03.206521  7545 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.391229  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:03.302973  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:03.398838  1560 raft_consensus.cc:2889] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:03.471428  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 168583us
I20260709 13:04:03.472731  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:03.366577  8042 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:03.309036  8220 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.402565  1683 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:03.403154  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:03.406328  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.408907  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.480876  1717 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.404546  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:03.481830  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:03.375530  8341 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:03.486042  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 83579.1us
I20260709 13:04:03.486457  1560 raft_consensus.cc:2368] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:04:03.403460  8064 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:03.431730  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 64407.3us
I20260709 13:04:03.391814  8173 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:03.391259  8286 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:04:03.385070  8062 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.002s	sys 0.012s
I20260709 13:04:03.438037  2298 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:03.441648  8192 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:03.448454  8303 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:04:03.442368  8332 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:03.446576  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.371349  8345 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:03.360399  8241 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:03.426106  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.403288  8102 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:04:03.451175  2270 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.388623  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:03.374372  8165 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.441406  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:03.602413  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 161105us
I20260709 13:04:03.603529  1607 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:03.439599  1532 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:03.465648  1609 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:03.403502  8308 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:03.411352  8305 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:03.449539  8228 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.619899  8305 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.621116  8305 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:03.473732  1681 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:03.623628  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 150012us
I20260709 13:04:03.624564  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:03.625371  8343 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:03.625921  8165 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.426260  8239 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:03.452322  8331 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.299s	user 0.003s	sys 0.008s
I20260709 13:04:03.462777  8214 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:03.627674  1710 raft_consensus.cc:2410] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:04:03.627885  8214 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.628361  8214 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:03.453364  8336 log.cc:927] Time spent T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.306s	user 0.002s	sys 0.006s
I20260709 13:04:03.484601  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:03.466977  8338 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.449270  8354 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:03.648976  8354 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:03.429447  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:03.651163  8354 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:03.500236  7872 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.654233  7872 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.403127  8301 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:03.502352  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:03.503161  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.659370  1709 raft_consensus.cc:2393] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:03.658977  8308 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.659979  8301 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:03.660333  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.661095  1709 raft_consensus.cc:2368] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:04:03.661273  8147 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:03.661550  8308 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.661790  8147 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:03.661861  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.662356  8147 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:03.662201  8350 consensus_queue.cc:1059] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:04:03.510042  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:03.663072  8248 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:03.663479  8350 raft_consensus.cc:3055] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:03.508602  8246 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:04:03.511924  8352 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:04:03.663879  8350 raft_consensus.cc:740] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:03.664206  8246 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:03.664976  8350 consensus_queue.cc:260] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:03.665275  8246 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.666268  8350 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:03.666436  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1188) took 1177 ms. Trace:
I20260709 13:04:03.513985  8064 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.514887  1569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2754) took 1014 ms. Trace:
W20260709 13:04:03.663884  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 153949us
I20260709 13:04:03.626784  8239 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:03.547072  8286 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.667215  1569 rpcz_store.cc:276] 0709 13:04:02.500397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.655621 (+155224us) service_pool.cc:224] Handling call
0709 13:04:03.514707 (+859086us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b5400216454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:03.514832 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1620134272}
I20260709 13:04:03.668437  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:03.669203  8239 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:03.515316  1562 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:03.516865  8220 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.517925  1641 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:03.517660  8236 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.501084  8347 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.671154  8347 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.671663  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:03.663846  8326 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:03.672973  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:03.673704  8347 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:03.519883  8323 raft_consensus.cc:3055] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:03.674407  8323 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:03.675295  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41282 (request call id 2757) took 1161 ms. Trace:
I20260709 13:04:03.675733  1563 rpcz_store.cc:276] 0709 13:04:02.513477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.667490 (+154013us) service_pool.cc:224] Handling call
0709 13:04:03.675089 (+1007599us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b540041ffd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:03.675232 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060007296}
I20260709 13:04:03.521674  1640 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:03.677323  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:03.475476  8346 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:03.522230  1561 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:04:03.678442  8346 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:03.679684  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1190) took 1190 ms. Trace:
I20260709 13:04:03.680135  1561 rpcz_store.cc:276] 0709 13:04:02.489484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.799771 (+310287us) service_pool.cc:224] Handling call
0709 13:04:02.800362 (+   591us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:03.678668 (+878306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.679395 (+   727us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54001fed54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:03.679608 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1659598080}
I20260709 13:04:03.680945  8346 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:03.528426  8307 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:03.406586  8267 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.681690  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:03.529585  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1341) took 1597 ms. Trace:
I20260709 13:04:03.414857  8264 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.683075  1716 rpcz_store.cc:276] 0709 13:04:01.931798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.674308 (+742510us) service_pool.cc:224] Handling call
0709 13:04:02.674643 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.965106 (+290463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.529234 (+564128us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54001c1954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:03.529505 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435927296}
I20260709 13:04:03.684420  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.684844  8264 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:03.685067  1716 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:03.686081  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:03.530562  1715 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:04:03.687454  8302 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:03.688611  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.690032  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.532195  1717 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
W20260709 13:04:03.691407  8351 log.cc:927] Time spent T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:04:03.691671  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:03.692373  1717 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.693194  1564 raft_consensus.cc:2393] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 21: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:03.694205  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 3297) took 1211 ms. Trace:
I20260709 13:04:03.480795  8036 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.537701  1714 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:04:03.695892  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.639886  7545 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:03.547508  2304 raft_consensus.cc:2368] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:04:03.626191  8343 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:03.428977  8304 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:03.536671  8173 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.711145  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:03.711979  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:03.479748  8255 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:04:03.512774  8355 log.cc:927] Time spent T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:04:03.639298  8062 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:03.675578  8323 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:03.719146  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:03.719563  8323 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:03.677974  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:03.694536  1564 rpcz_store.cc:276] 0709 13:04:02.482940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.483095 (+   155us) service_pool.cc:224] Handling call
0709 13:04:03.694018 (+1210923us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400325854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:03.694147 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2527765248}
W20260709 13:04:03.605417  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 165906us
I20260709 13:04:03.607445  8102 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:03.703888  2298 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:04:03.736881  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:03.737710  1640 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 23.
I20260709 13:04:03.737746  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:03.702415  1641 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:04:03.699165  1711 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1346) took 1258 ms. Trace:
I20260709 13:04:03.666837  1565 rpcz_store.cc:276] 0709 13:04:02.488931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.751303 (+262372us) service_pool.cc:224] Handling call
0709 13:04:02.751756 (+   453us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:02.770625 (+ 18869us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:03.055467 (+284842us) log.cc:844] Serialized 14 byte log entry
0709 13:04:03.320489 (+265022us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:03.331605 (+ 11116us) log.cc:844] Serialized 52 byte log entry
0709 13:04:03.331832 (+   227us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:03.350231 (+ 18399us) log.cc:844] Serialized 14 byte log entry
0709 13:04:03.398372 (+ 48141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.398451 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:03.666196 (+267745us) raft_consensus.cc:1685] finished
0709 13:04:03.666235 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.666380 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55387,"prepare.queue_time_us":401350,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":75,"raft.queue_time_us":69220,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":11351,"spinlock_wait_cycles":24832,"thread_start_us":492097,"threads_started":2,"wal-append.queue_time_us":288789}
I20260709 13:04:03.669487  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:03.740762  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:03.741386  1641 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:04:03.673872  1560 raft_consensus.cc:2410] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 20 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:04:03.743634  1560 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:04:03.708436  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 69794.7us
I20260709 13:04:03.703627  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:03.710294  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:03.698729  1714 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:04:03.744748  1607 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:03.711136  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:03.646086  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 180537us
I20260709 13:04:03.745616  1569 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:03.662688  1709 raft_consensus.cc:2393] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 20: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:03.746991  1569 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:04:03.713534  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:03.748208  1569 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:03.749315  1569 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:03.751103  2303 raft_consensus.cc:2410] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:04:03.713717  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:03.711668  8343 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:03.641122  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:03.755432  1566 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:04:03.690680  1715 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:04:03.756088  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:03.756989  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:03.729437  1717 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:04:03.731700  2302 raft_consensus.cc:1275] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:03.739451  1711 rpcz_store.cc:276] 0709 13:04:02.440592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.722507 (+281915us) service_pool.cc:224] Handling call
0709 13:04:02.724649 (+  2142us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.724989 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.725014 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:02.753377 (+ 28363us) log.cc:844] Serialized 26 byte log entry
0709 13:04:03.385883 (+632506us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:03.385924 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.385990 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:03.690540 (+304550us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:04:03.698841 (+  8301us) raft_consensus.cc:1685] finished
0709 13:04:03.698860 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.699100 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":840,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":21810,"spinlock_wait_cycles":29590784,"thread_start_us":579670,"threads_started":1,"wal-append.queue_time_us":720544}
I20260709 13:04:03.752733  1565 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:04:03.741609  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:03.753424  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:03.754240  1607 leader_election.cc:400] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:04:03.756922  2297 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.761276  1607 leader_election.cc:403] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:03.765870  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:03.766255  1717 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:04:03.766803  1709 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:04:03.687243  8286 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.769768  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.769737  2268 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:03.770985  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:03.773391  2268 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:03.769809  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:03.775049  2305 raft_consensus.cc:2410] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:04:03.775867  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:03.776764  1715 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:04:03.783725  2297 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:04:03.812956  8181 raft_consensus.cc:3055] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:03.785414  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.785223  8064 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 13:04:03.785627  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.785940  8354 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:03.716787  7872 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.786334  8347 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:03.786193  8341 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:03.770985  8105 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:03.789527  1609 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:03.852811  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 63375.2us
I20260709 13:04:03.790078  8302 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:03.768764  8343 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:03.791973  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:03.794862  1607 leader_election.cc:400] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
W20260709 13:04:03.863888  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 69186.2us
W20260709 13:04:03.864293  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 72425.6us
I20260709 13:04:03.821314  8303 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:04:03.776083  8201 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:03.806195  2300 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:04:03.719137  8062 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:04:03.814832  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.819535  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:03.769858  8097 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:03.723353  8304 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.876713  1609 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:03.885939  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 66544us
I20260709 13:04:03.856119  7545 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:03.832230  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:03.803294  8326 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:03.784013  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:03.894632  1635 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:04:03.892449  8308 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:03.891724  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:03.895627  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 111678us
I20260709 13:04:03.786480  8301 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:04:03.837208  1711 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:04:03.835218  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:03.897769  2305 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.841722  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:04:03.827885  8340 log.cc:927] Time spent T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:04:03.847334  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.851804  8307 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:04:03.740821  8102 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.896721  8326 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:03.911468  8326 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:03.836915  8181 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:03.838887  8241 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.825176  8332 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:04:03.918054  8181 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.919443  8181 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.919713  8173 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:03.924489  8064 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:03.927318  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:04:03.930896  8291 log.cc:927] Time spent T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 1.722s	user 0.005s	sys 0.000s
I20260709 13:04:03.933241  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1335) took 2007 ms. Trace:
I20260709 13:04:03.933750  1718 rpcz_store.cc:276] 0709 13:04:01.925771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.925957 (+   186us) service_pool.cc:224] Handling call
0709 13:04:01.926458 (+   501us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:01.931466 (+  5008us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.931498 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:01.932177 (+   679us) log.cc:844] Serialized 52 byte log entry
0709 13:04:02.000777 (+ 68600us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:02.036222 (+ 35445us) log.cc:844] Serialized 14 byte log entry
0709 13:04:02.036496 (+   274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.036584 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:03.832497 (+1795913us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:04:03.932829 (+100332us) raft_consensus.cc:1685] finished
0709 13:04:03.932949 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.933147 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6575,"prepare.queue_time_us":35222,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":10771,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":6460,"spinlock_wait_cycles":8821504,"thread_start_us":68205,"threads_started":1,"wal-append.queue_time_us":140971}
I20260709 13:04:03.937952  2305 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:04:03.938063  8102 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.938678  8192 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:03.940786  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:03.848649  8347 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.942350  8347 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:03.918661  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:03.897188  8345 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:03.898964  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 57335.9us
I20260709 13:04:03.945670  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3211) took 1630 ms. Trace:
I20260709 13:04:03.945791  8321 raft_consensus.cc:3055] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:04:03.946301  8321 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:03.946779  8274 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.947347  8321 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.947831  1682 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:03.865844  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:03.948570  8321 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:03.949477  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 83768.8us
I20260709 13:04:03.950201  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:03.890081  8267 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.951032  8267 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:03.951328  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:03.951767  8267 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:03.952183  8267 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:03.952543  8267 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.952980  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:03.953295  1568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41274 (request call id 3293) took 2198 ms (client timeout 1749 ms). Trace:
I20260709 13:04:03.953863  8301 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:03.856284  8354 raft_consensus.cc:697] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:03.954838  8301 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:03.955312  8301 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:03.955530  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.955614  8354 consensus_queue.cc:237] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:03.959228  8256 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:03.960472  7872 consensus_queue.cc:1059] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:03.961210  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:03.961880  2304 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:03.962323  8232 raft_consensus.cc:3055] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:03.962718  8232 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:04:03.953728  1568 rpcz_store.cc:269] 0709 13:04:01.754512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.761097 (+  6585us) service_pool.cc:224] Handling call
0709 13:04:03.953073 (+2191976us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54002c5954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:03.953198 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2906292608}
I20260709 13:04:03.963527  8232 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:03.964771  8232 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:03.946094  1639 rpcz_store.cc:276] 0709 13:04:02.315266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.333303 (+ 18037us) service_pool.cc:224] Handling call
0709 13:04:02.333623 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.339993 (+  6370us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:02.340024 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:02.340489 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:04:02.661134 (+320645us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b18003b7100. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:04:02.771041 (+109907us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:02.771088 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.771609 (+   521us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:03.944051 (+1172442us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b240028a528. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79ea3a9c4 00007fd79ea38acb 00007fd79ebf3dc9 00007fd79ebf3a55 00007fd79ebf2fb0 00007fd7a1869488 00007fd7a1869be8 00007fd7a1841398 00007fd7a183c76a 00007fd7a183bbb6
0709 13:04:03.945450 (+  1399us) raft_consensus.cc:1685] finished
0709 13:04:03.945470 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.945602 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15748,"raft.queue_time_us":8963,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":13750,"spinlock_wait_cycles":76468864,"thread_start_us":109656,"threads_started":1,"wal-append.queue_time_us":464295}
I20260709 13:04:03.967392  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:03.871213  8123 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:03.968513  8237 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:03.968849  8123 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:03.900997  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:03.970477  2296 raft_consensus.cc:2393] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 21: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:03.901077  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:03.971149  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 70152.6us
I20260709 13:04:03.867897  8165 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:03.888550  8035 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:03.972512  8035 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:03.973045  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:03.974260  1718 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:03.973897  8035 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.907038  8308 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:03.975404  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:03.975849  8201 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:03.976302  8201 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.976665  8201 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:03.977232  8274 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:03.977372  1533 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:03.978307  8220 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:03.979466  8220 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:03.980019  8105 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:03.980568  8105 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:03.981146  1714 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 13:04:03.981518  1714 pending_rounds.cc:85] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 8
I20260709 13:04:03.981721  1714 pending_rounds.cc:107] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:04:03.982041  1714 raft_consensus.cc:2889] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:03.981787  8042 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:03.989279  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:03.989694  8042 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:03.909608  8104 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:03.994442  8264 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:03.881497  8062 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:03.881121  8097 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.999171  8097 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:04.001374  1636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3216) took 1266 ms. Trace:
I20260709 13:04:04.002067  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:04.002826  1565 raft_consensus.cc:2393] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 18: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:04.003564  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:04.003847  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:03.914556  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:03.941197  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3298) took 1457 ms. Trace:
I20260709 13:04:04.005482  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:04.007912  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:03.961333  8350 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:03.870075  8338 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:03.871709  8303 consensus_queue.cc:1243] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:04:04.014935  8304 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:03.904368  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:04.015982  1715 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:04:04.016978  7887 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:04.017498  1609 leader_election.cc:400] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 15. Current term is 16.
I20260709 13:04:04.017903  8303 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:03.985172  1682 leader_election.cc:400] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 20. Current term is 21.
I20260709 13:04:04.018838  1682 leader_election.cc:403] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:04.020933  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:04.023232  8338 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 20. Current term is 21.
I20260709 13:04:04.025841  8123 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.026342  8097 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:03.985168  8184 consensus_queue.cc:1059] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:03.908083  8294 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:03.978655  1533 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:03.975553  8308 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:03.987066  8282 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:03.987653  8237 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:03.988313  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:03.988636  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:04.005617  1567 rpcz_store.cc:276] 0709 13:04:02.483426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.483574 (+   148us) service_pool.cc:224] Handling call
0709 13:04:02.490850 (+  7276us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:03.055158 (+564308us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:03.301903 (+246745us) log.cc:844] Serialized 14 byte log entry
0709 13:04:03.426124 (+124221us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:03.427153 (+  1029us) log.cc:844] Serialized 52 byte log entry
0709 13:04:03.427304 (+   151us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:03.427359 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.940342 (+512983us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b18002e8a60. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb822a6 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419
0709 13:04:03.940891 (+   549us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54000f6254. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:03.940930 (+    39us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:03.940973 (+    43us) raft_consensus.cc:1685] finished
0709 13:04:03.940988 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.941141 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40614,"prepare.queue_time_us":116886,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":82114,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":36216,"spinlock_wait_cycles":708310144,"thread_start_us":540615,"threads_started":3,"wal-append.queue_time_us":183019}
I20260709 13:04:04.008100  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:04.008777  2269 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:04.009322  7576 consensus_queue.cc:1059] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:04.009527  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:04.001799  1636 rpcz_store.cc:276] 0709 13:04:02.735146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.037466 (+302320us) service_pool.cc:224] Handling call
0709 13:04:03.037855 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.447567 (+409712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.447603 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:03.447646 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:03.447680 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.001142 (+553462us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b5400125554. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:04.001188 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.001314 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941777280}
I20260709 13:04:04.011298  8309 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:04.028465  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:04.028960  8123 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:03.983062  8105 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:04.030639  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:04.032435  8257 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:04.123278  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:04.123982  2298 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:04:04.037700  2304 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
W20260709 13:04:03.831269  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 130033us
I20260709 13:04:04.124887  2302 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:04:04.039736  8201 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.042097  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:04:04.043123  8351 log.cc:927] Time spent T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.060s	user 0.000s	sys 0.006s
I20260709 13:04:04.043200  7545 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:04.049446  8042 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.044440  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:04.047773  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:03.965829  8214 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.054392  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:04.060792  2303 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:03.868822  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:03.942407  2305 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 23.
I20260709 13:04:04.115319  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:04.116396  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 137832us
I20260709 13:04:04.047253  8274 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:04.117486  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.098083  8362 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:04:04.119895  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 111215us
I20260709 13:04:04.075610  8256 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:04.120748  1683 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:04.121649  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 91089.4us
I20260709 13:04:04.126346  1609 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:04.095816  8102 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:04.137797  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.198426  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:04.043783  8361 log.cc:927] Time spent T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.177s	user 0.001s	sys 0.004s
I20260709 13:04:04.120512  8325 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:04.157085  1533 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:04.155365  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 111012us
I20260709 13:04:04.159956  2300 raft_consensus.cc:1217] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Deduplicated request from leader. Original: 17.12->[18.13-23.16]   Dedup: 18.14->[22.15-23.16]
I20260709 13:04:04.188849  2268 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:04.212798  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:04.199435  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:04.213223  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 171192us
I20260709 13:04:04.213760  2304 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
W20260709 13:04:04.201084  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 74843.2us
I20260709 13:04:04.129045  8064 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:04.206126  2269 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:04.207762  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:04.208283  2301 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:04.155567  8323 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:04.192215  8367 log.cc:927] Time spent T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:04:04.148461  1459 catalog_manager.cc:5697] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 16 to 18, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195), now has a pending config: VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193), VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194), VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195), VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 18 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } } pending_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:04.212646  2300 pending_rounds.cc:85] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 14
I20260709 13:04:04.221418  2300 pending_rounds.cc:107] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:04:04.221799  2300 raft_consensus.cc:2889] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:04.144454  8294 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:04.123598  8309 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:04.187005  8105 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:04.224072  1681 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:04.224751  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:04.210274  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:04.231817  1565 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 19: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:04.117297  8347 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:04.226727  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 106072us
I20260709 13:04:04.119938  8237 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:04.127960  8245 consensus_queue.cc:1048] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.090s
W20260709 13:04:04.240017  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 83015.4us
I20260709 13:04:04.242192  1533 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:04.214990  8036 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:04.246812  2299 raft_consensus.cc:1240] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:04:04.219115  8345 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:04.251366  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:04.162633  8042 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.223711  8181 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:04.237475  8246 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:04.249747  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:04.228322  8307 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.227890  8286 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:04.229461  8294 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.228001  8305 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:04.248155  8324 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.094s	user 0.007s	sys 0.011s
I20260709 13:04:04.234164  7872 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:04.256455  1566 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:04:04.234704  8105 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:04.264276  1562 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:04:04.280064  1462 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45142 (request call id 352) took 19736 ms (client timeout 15000 ms). Trace:
I20260709 13:04:04.280573  1533 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:04.284245  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:04.222461  8102 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:04:04.277604  1459 catalog_manager.cc:5697] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 9 to 10, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 10 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:04.253505  7715 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:04.243434  8035 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:04.263758  8036 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:04.236871  8062 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:04.307519  1459 catalog_manager.cc:5697] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:04.263460  8332 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:04.258805  8345 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:04.288290  8324 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.137s	user 0.007s	sys 0.011s
I20260709 13:04:04.233903  8323 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.279085  8341 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:04.285784  8294 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:04.307746  8364 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:04:04.310375  8105 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:04.263721  8343 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:04.293712  1462 rpcz_store.cc:269] 0709 13:03:44.520349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:44.520636 (+   287us) service_pool.cc:224] Handling call
0709 13:03:44.531281 (+ 10645us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b540029ff90. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7d0b2 00007fd7a7af6a89 00007fd7a7a8eda6 00007fd7a7a60088 00007fd7a7b5e37e 00007fd7a601f2a4 00007fd7a601f232 00007fd7a601f161 00007fd7a601f0dc
0709 13:04:04.257013 (+19725732us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:59.078276 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:59.242379 (+164103us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:59.252343 (+  9964us) write_op.cc:435] Acquired schema lock
0709 13:03:59.255350 (+  3007us) tablet.cc:662] Decoding operations
0709 13:03:59.363124 (+107774us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:59.363231 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:59.363259 (+    28us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:03:59.365504 (+  2245us) tablet.cc:701] Row locks acquired
0709 13:03:59.365599 (+    95us) write_op.cc:260] PREPARE: finished
0709 13:03:59.365854 (+   255us) write_op.cc:270] Start()
0709 13:03:59.366055 (+   201us) write_op.cc:276] Timestamp: P: 1783602239365759 usec, L: 0
0709 13:03:59.366121 (+    66us) op_driver.cc:348] REPLICATION: starting
0709 13:03:59.470199 (+104078us) log.cc:844] Serialized 7229 byte log entry
0709 13:04:01.153158 (+1682959us) op_driver.cc:464] REPLICATION: finished
0709 13:04:01.412351 (+259193us) write_op.cc:301] APPLY: starting
0709 13:04:01.412495 (+   144us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:01.658436 (+245941us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:01.658597 (+   161us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:01.806426 (+147829us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:01.806790 (+   364us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 13:04:01.806880 (+    90us) write_op.cc:312] APPLY: finished
0709 13:04:01.982023 (+175143us) log.cc:844] Serialized 164 byte log entry
0709 13:04:01.982271 (+   248us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:01.982458 (+   187us) write_op.cc:454] Released schema lock
0709 13:04:01.982549 (+    91us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:01.989660 (+  7111us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:01.995923 (+  6263us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3569638,"reported_tablets":46,"spinlock_wait_cycles":3846016,"child_traces":[["op",{"apply.queue_time_us":242448,"apply.run_cpu_time_us":28924,"apply.run_wall_time_us":570642,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79403,"num_ops":18,"prepare.queue_time_us":77334,"prepare.run_cpu_time_us":39484,"prepare.run_wall_time_us":1033310,"raft.queue_time_us":389454,"raft.run_cpu_time_us":79632,"raft.run_wall_time_us":154642,"replication_time_us":1786859,"thread_start_us":991610,"threads_started":4,"wal-append.queue_time_us":696890,"wal-append.run_cpu_time_us":25666,"wal-append.run_wall_time_us":2888108}]]}
W20260709 13:04:04.362663  8147 consensus_meta.cc:287] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.650s	user 0.005s	sys 0.007s
I20260709 13:04:04.391749  8309 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:04.365959  1459 catalog_manager.cc:5697] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 13 to 17, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 17 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:04.362275  8345 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:04.392781  8214 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:04.393729  8327 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.393308  1459 catalog_manager.cc:5697] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 9 to 10, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 10 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:04.394286  8327 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.380959  8201 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:04.349970  8035 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:04.377974  8097 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:04.391417  8147 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.290140  8307 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.394896  1459 catalog_manager.cc:5697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 13 to 16, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 16 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:04.395576  8062 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:04.417187  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:04.380206  8323 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:04.398224  8257 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:04:04.399188  1716 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:04.419646  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2696) took 1006 ms. Trace:
I20260709 13:04:04.419971  1716 rpcz_store.cc:276] 0709 13:04:03.413358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.723468 (+310110us) service_pool.cc:224] Handling call
0709 13:04:03.726472 (+  3004us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:04.418798 (+692326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.419436 (+   638us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b5400474854. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:04.419587 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546604288}
I20260709 13:04:04.390429  8301 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.420769  8267 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:04.421136  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:04.421782  1716 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:04.422031  8104 raft_consensus.cc:3055] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:04.422451  8104 raft_consensus.cc:740] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:04.404871  8237 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:04.406368  8036 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.423331  8104 consensus_queue.cc:260] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.423956  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:04.424399  8104 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:04.309983  7872 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:04.395782  8042 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:04.375703  8326 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.414789  8147 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:04.416909  8035 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:04.435552  1459 catalog_manager.cc:5697] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:04.438887  8302 raft_consensus.cc:3055] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:04.442024  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:04.453743  8307 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:04.454366  8366 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:04:04.454711  8246 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:04.455216  8246 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:04.455313  8347 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:04.461998  8345 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.463004  1713 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:04.433964  8301 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:04.444964  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:04.464545  8338 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.464864  8326 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:04.452661  8302 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:04.465112  8338 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:04.465704  2297 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:04.466276  8302 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.375589  7715 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:04.468381  1459 catalog_manager.cc:5697] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:04.422732  8267 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:04.480238  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 247482us
I20260709 13:04:04.481022  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:04.482471  1683 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:04.487023  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:04.486320  1459 catalog_manager.cc:5697] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 10 to 12, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 12 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:04.488291  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 276200us
I20260709 13:04:04.489523  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:04.489604  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:04.491806  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:04.488287  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:04.484115  8282 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:04.510878  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:04.514487  1711 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:04.516358  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:04.484478  7715 raft_consensus.cc:697] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:04.541435  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:04.501838  1459 catalog_manager.cc:5697] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 12 to 13, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 13 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:04.529498  1716 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:04:04.548331  1716 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:04:04.496101  8147 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:04.560763  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:04.553277  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:04.562879   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:04.558513  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:04.563030  1643 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:04:04.508459  8302 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:04.566128  1713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36066 (request call id 2808) took 1338 ms. Trace:
I20260709 13:04:04.612491  1713 rpcz_store.cc:276] 0709 13:04:03.228234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.406016 (+177782us) service_pool.cc:224] Handling call
0709 13:04:04.565973 (+1159957us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b54000483d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:04.566071 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2258556160}
I20260709 13:04:04.576772  8310 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:04.614125  8310 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.616990  8310 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:04.567298  1459 catalog_manager.cc:5697] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 11 to 14, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 14 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:04.577816  8241 consensus_queue.cc:1059] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:04.559592  8354 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.583707  7872 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:04.540060  8035 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:04.540442  8232 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:04.588953  7715 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:04.591744  8372 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.205s	user 0.004s	sys 0.000s
W20260709 13:04:04.593503  8371 log.cc:927] Time spent T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.207s	user 0.001s	sys 0.006s
I20260709 13:04:04.600356  8327 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:04.601897  8343 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:04.603832  8282 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:04:04.611627  8374 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:04:04.613579  1709 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:04:04.541198  8267 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.108s
I20260709 13:04:04.619107  1459 catalog_manager.cc:5697] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 6 to 11, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 11 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:04.620463  8232 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:04.683581  8232 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:04.618854  8294 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.682653  1459 catalog_manager.cc:5697] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 11 to 13, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 13 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:04.624238  8282 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.688184  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:04.688838  2296 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:04.632423  8105 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:04.691361  1459 catalog_manager.cc:5697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:04.693559  8105 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:04.693035  8232 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.693207  1459 catalog_manager.cc:5697] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 9 to 13, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 13 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:04.638428  8338 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:04.657976  7518 raft_consensus.cc:3055] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:04.695343  7518 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:04.696312  8232 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:04.696396  7518 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.661010  8035 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.660382  1639 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 22.
I20260709 13:04:04.659641  8345 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:04.672266  8102 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:04.660133  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 74445.6us
I20260709 13:04:04.660988  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:04.661181  7545 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:04:04.624179  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:04.619659  8354 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:04.627839  8381 log.cc:927] Time spent T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:04:04.661938  8267 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:04.715724  2296 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:04.745566  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:04:04.716711  1718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.007s	sys 0.008s
W20260709 13:04:04.753911  1718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.008s	sys 0.008s
I20260709 13:04:04.754189  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:04.508837  1683 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:04.755751  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 247039us
I20260709 13:04:04.716063  1459 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 12 to 14, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 14 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:04.730263  2303 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:04.686647  8294 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:04.703306  8035 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:04.732698  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 71800.5us
I20260709 13:04:04.733359  7906 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:04.732486  2299 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
W20260709 13:04:04.735119  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 454627us
I20260709 13:04:04.737995  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:04.768894  1639 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:04:04.751945  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:04.717373  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 493421us
I20260709 13:04:04.771533  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:04.740037  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:04.772692  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:04.777699  1642 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:04.770581  1635 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:04:04.773604  1639 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:04.782827  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:04.782889  1459 catalog_manager.cc:5697] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:04.783824  1711 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:04.733959  7518 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:04.805325  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:04.790874  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:04.792631  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:04.806902  1638 raft_consensus.cc:2393] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 21: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:04.806887  1639 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 20: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:04.809902  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:04.809202  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.003s	sys 0.016s
W20260709 13:04:04.811776  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.003s	sys 0.016s
W20260709 13:04:04.812542  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 587880us
I20260709 13:04:04.807606  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:04.813511  2269 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:04.819195  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:04.819461  1681 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:04.753036  8267 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:04.699939  8246 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.785377  8042 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:04.733981  8102 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.820008  1640 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:04:04.778892  7906 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.778075  8338 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:04.845278  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:04.855170  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:04.756196  8104 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 13:04:04.777335  8309 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:04.827379  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:04.815449  8184 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:04.821673  7887 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:04.787770  8257 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 13:04:04.787640  8147 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:04:04.821973  8384 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:04.831894  8245 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:04.878003  1715 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:04.885004  1714 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.023s	sys 0.077s
I20260709 13:04:04.876784  7887 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:04.893172  1714 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.024s	sys 0.077s
W20260709 13:04:04.891525  1712 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.003s	sys 0.021s
I20260709 13:04:04.863945  8042 raft_consensus.cc:697] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:04:04.888422  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 98445.6us
W20260709 13:04:04.893962  1712 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.003s	sys 0.022s
I20260709 13:04:04.853133  7906 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:04.884711  8305 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.894827  8354 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:04.895572  8309 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:04.896071  7887 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:04.938849  8309 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:04.940412  8102 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:04.887800  8267 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.895285  8384 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
W20260709 13:04:04.951540  8388 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:04:04.907047  8305 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:04.971763  8267 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:05.014602  1459 catalog_manager.cc:5697] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:05.020236  1459 catalog_manager.cc:5697] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 9 to 11, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 11 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:05.036715  1535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.000s	sys 0.014s
I20260709 13:04:04.875109  1635 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:04:05.043258  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:05.047528  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:05.038080  1532 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:05.067931  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 173322us
I20260709 13:04:05.070297  1712 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2818) took 1044 ms. Trace:
W20260709 13:04:05.071321  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 243985us
I20260709 13:04:05.071609  1716 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:05.070565  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1362) took 1044 ms. Trace:
W20260709 13:04:05.124917  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 77854.3us
W20260709 13:04:05.126434  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 88947.4us
I20260709 13:04:05.133620  2299 raft_consensus.cc:1275] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:04:05.137943  2301 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:04:05.123986  1535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.000s	sys 0.015s
I20260709 13:04:05.124766  1637 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:05.136368  2297 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:05.126514  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:05.119051  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:05.123988  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:05.133559  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:05.162729  8294 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:05.165664  1635 raft_consensus.cc:2393] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 21: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:05.130716  1712 rpcz_store.cc:276] 0709 13:04:04.026065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.026183 (+   118us) service_pool.cc:224] Handling call
0709 13:04:04.026470 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.028116 (+  1646us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:04.028371 (+   255us) log.cc:844] Serialized 14 byte log entry
0709 13:04:04.894459 (+866088us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:04.895199 (+   740us) log.cc:844] Serialized 26 byte log entry
0709 13:04:04.895358 (+   159us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:04.895395 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.895468 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.069627 (+174159us) raft_consensus.cc:1685] finished
0709 13:04:05.069672 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.069949 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2953,"prepare.queue_time_us":735,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":93769,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":7539,"thread_start_us":865476,"threads_started":2,"wal-append.queue_time_us":457571}
I20260709 13:04:05.167399  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:05.162576  2296 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:04:05.138581  1714 rpcz_store.cc:276] 0709 13:04:04.026348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.121114 (+ 94766us) service_pool.cc:224] Handling call
0709 13:04:04.124142 (+  3028us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:04.152403 (+ 28261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.152451 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:04.165702 (+ 13251us) log.cc:844] Serialized 52 byte log entry
0709 13:04:04.897148 (+731446us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:05.069367 (+172219us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.069879 (+   512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.070040 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.070109 (+    69us) raft_consensus.cc:1685] finished
0709 13:04:05.070124 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.070394 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35477,"prepare.queue_time_us":241501,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":89,"raft.queue_time_us":36668,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":6993,"thread_start_us":660956,"threads_started":1,"wal-append.queue_time_us":730429}
I20260709 13:04:05.169018  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:05.169754  1635 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:04:05.171734  8305 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.123724  8102 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:05.174875  1638 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:05.183545  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:05.185016  1638 raft_consensus.cc:1240] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:05.080662  7545 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:05.187126  7545 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:05.189460  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:05.189913  7545 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.082214  2269 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:05.092813  7872 consensus_queue.cc:1059] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:05.102257  8062 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.191730  8392 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:05.192437  8392 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:05.192230  8062 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.193353  8305 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:04.951076  8042 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.104461  8246 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:05.105386  8255 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:05.116694  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:05.175594  8102 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:05.192509  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 110888us
I20260709 13:04:05.103950  8220 consensus_queue.cc:1059] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:05.108851  8147 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:05.208413  7545 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.077566  1643 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:04:05.218240  8220 raft_consensus.cc:3055] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:05.218552  1637 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:05.218712  8220 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:05.218739  8062 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:05.226591  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 95339us
I20260709 13:04:05.181576  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:05.131422  7715 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:05.182873  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:05.194157  8392 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.118907  7906 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:05.237236  8382 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.347s	user 0.000s	sys 0.004s
I20260709 13:04:05.241511  8220 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.242116  7872 raft_consensus.cc:3055] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:05.242811  7872 raft_consensus.cc:740] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:05.244522  7872 consensus_queue.cc:260] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.246908  7872 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:05.247174  7545 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.255612  2302 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:05.256614  7887 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:05.257053  8324 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:05.259202  8324 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:05.259547  7887 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.261293  1641 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:05.180330  8282 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:05.265787  1639 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:05.235257  7715 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:05.270515  7887 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:05.271343  1644 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:05.271070  8307 consensus_queue.cc:1175] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5184c101be204393adf3ac8b31f25ad6" responder_term: 13 status { last_received { term: 13 index: 9 } last_committed_idx: 8 last_received_current_leader { term: 13 index: 9 } }
I20260709 13:04:05.262102  8324 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:05.281858  1718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 2819) took 1186 ms. Trace:
I20260709 13:04:05.282406  1718 rpcz_store.cc:276] 0709 13:04:04.095281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.163027 (+ 67746us) service_pool.cc:224] Handling call
0709 13:04:04.163455 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.163924 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:04.163964 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:04.164520 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 13:04:04.388376 (+223856us) spinlock_profiling.cc:243] Waited 9.73 ms on lock 0x7b480009ffd8. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a6bcaaa1 00007fd7a6bc1b37 00007fd7a6bc19ca 00007fd7a6cb926f 00007fd7a6cb8f37 00007fd7a6ccd55c 00007fd7a6ccd508 00007fd7a6ccd488 00007fd7a6ccd443 00007fd7a6ccc6ec 00007fd7a5bf6a96
0709 13:04:04.754553 (+366177us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:04.754615 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.754714 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.182569 (+427855us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:04:05.273768 (+ 91199us) raft_consensus.cc:1685] finished
0709 13:04:05.273790 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.281601 (+  7811us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48757,"raft.queue_time_us":985,"raft.run_cpu_time_us":2090,"raft.run_wall_time_us":202872,"spinlock_wait_cycles":25436672,"thread_start_us":589663,"threads_started":1,"wal-append.queue_time_us":678104}
I20260709 13:04:05.226231  8147 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:05.265578  8282 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:05.273130  7715 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:05.271565  7887 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.268836  8305 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:05.307906  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.312515  2297 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:05.313359  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:05.313766  8264 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:05.315078  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:05.316942  2299 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21. Ops: [20.15-20.15]
W20260709 13:04:05.323089  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 55018.6us
I20260709 13:04:05.315101  2296 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:05.338179  1635 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:05.294852  8392 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:05.345628  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 56423.8us
I20260709 13:04:05.354728  2269 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:05.383054  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 87460.7us
I20260709 13:04:05.390998  1642 raft_consensus.cc:1240] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:05.391346  8173 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:05.391074  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1654) took 1332 ms. Trace:
I20260709 13:04:05.391896  8173 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.400417  1714 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:05.403355  8282 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.404758  8282 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:05.407841  8383 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:05.400514  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 274113us
I20260709 13:04:05.408591  8383 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.420459  1682 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:05.431700  2298 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:04:05.428635  1639 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:04:05.436025  8345 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:05.436888  8345 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.438843  7518 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:05.440017  7518 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:05.440384  7518 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:05.377573  8264 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.384445  8305 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.388137  8309 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.132s
I20260709 13:04:05.410574  8173 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:05.393962  2300 rpcz_store.cc:276] 0709 13:04:04.048188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.048395 (+   207us) service_pool.cc:224] Handling call
0709 13:04:04.159582 (+111187us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:04:04.244540 (+ 84958us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:04:04.539136 (+294596us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.083424 (+544288us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:05.084832 (+  1408us) log.cc:844] Serialized 52 byte log entry
0709 13:04:05.085188 (+   356us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:05.085236 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.085389 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.175972 (+ 90583us) raft_consensus.cc:1685] finished
0709 13:04:05.175989 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.375775 (+199786us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54006ecdd0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca08ad 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:05.380444 (+  4669us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":201144,"prepare.queue_time_us":161259,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":228185,"raft.run_cpu_time_us":427,"raft.run_wall_time_us":7169,"spinlock_wait_cycles":247779072,"thread_start_us":559144,"threads_started":2,"wal-append.queue_time_us":418081}
I20260709 13:04:05.460861  8035 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.467298  8309 consensus_queue.cc:1243] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:04:05.468039  8309 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:05.468425  8309 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:05.468730  8104 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:05.469650  8309 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:05.409649  8255 consensus_queue.cc:1175] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5184c101be204393adf3ac8b31f25ad6" responder_term: 17 status { last_received { term: 17 index: 15 } last_committed_idx: 14 last_received_current_leader { term: 17 index: 15 } }
W20260709 13:04:05.469642  8409 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.169s	user 0.000s	sys 0.002s
I20260709 13:04:05.480475  8147 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.483862  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:05.485671  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:05.486413  8264 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:05.488271  8309 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:05.493311  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:05.493718  7887 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:05.494109  1714 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:04:05.495401  1681 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:05.503716  8305 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.506150  2302 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:04:05.455123  8220 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:05.507845  1607 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.523s (SENT) [suppressed 8 similar messages]
I20260709 13:04:05.508208  2299 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:04:05.508841  8062 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:04:05.518433  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 59290.4us
I20260709 13:04:05.524044  8390 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:05.524737  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 69179us
W20260709 13:04:05.525285  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 134642us
I20260709 13:04:05.526051  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:05.527002  8245 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:05.470835  7906 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.526273  8264 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 13:04:05.557201  1709 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:05.535422  8245 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.549793  1640 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:05.587881  8036 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:05.590384  2299 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:05.588531  8036 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:05.588563  8232 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:05.615836  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 120766us
I20260709 13:04:05.572351  8245 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:05.623499  8414 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:05.601495  8246 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:05.626765  8246 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.615993  8305 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 13:04:05.591491  7887 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.637732  7906 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:05.638459  7887 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:05.640441  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.641656  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.644840  7887 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:05.609249  8301 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:05.653760  8391 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.655172  8301 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:05.655474  8391 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:05.646075  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 200612us
W20260709 13:04:05.623904  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.626591  8414 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.655191  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:05.558436  8390 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.687963  8147 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.652132  8246 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:05.695079  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:05.688987  8147 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.698741  8147 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:05.694878  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.703060  1533 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:05.705255  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:05.706987  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.707955  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:05.718974  1681 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:05.638566  7906 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:05.719998  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 64947.5us
I20260709 13:04:05.718647  2300 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 13.
W20260709 13:04:05.724313  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 216591us
I20260709 13:04:05.734362  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:05.615206  8036 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:05.717444  8246 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.717298  8309 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:05.711208  8345 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:05.718636  8305 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.741765  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:05.756896  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 52178.9us
I20260709 13:04:05.730464  8062 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.756227  1683 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:05.721794  8392 consensus_queue.cc:1059] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
W20260709 13:04:05.772126  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 55209us
I20260709 13:04:05.758064  8324 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:05.778236  8324 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:05.778865  8324 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:05.792797  8324 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:05.794557  1713 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:05.776216  1715 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 15.
I20260709 13:04:05.772637  1682 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:05.754856  8147 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:05.805739  1712 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:04:05.808111  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 66134us
W20260709 13:04:05.808811  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 81131.5us
W20260709 13:04:05.810531  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 80938.7us
I20260709 13:04:05.776590  8036 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:05.822173  8384 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:05.822003  8105 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.066s
I20260709 13:04:05.822695  8384 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.823143  8384 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:05.825354  8267 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.826126  8267 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.827157  8232 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:05.827626  8332 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:05.828538  8332 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.767453  8246 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:05.831907  8301 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.763481  8305 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.835072  8385 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.835988  8385 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:05.827679  8232 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:05.837185  8232 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:05.800540  8392 consensus_queue.cc:1243] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:05.838465  8385 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:05.838907  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 63652us
I20260709 13:04:05.840984  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:05.842598  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:05.844862  8391 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.847210  8423 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:05.847921  8423 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:05.849269  8418 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.396s	user 0.001s	sys 0.001s
I20260709 13:04:05.851827  8414 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:05.852667  1682 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:05.853817  8173 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.855398  8105 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:05.855723  8392 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:05.856003  8105 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.856544  8392 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:04:05.857349  7872 consensus_meta.cc:287] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.610s	user 0.013s	sys 0.002s
I20260709 13:04:05.858316  8105 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:05.858384  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:05.860502  8390 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.861403  8246 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:05.865836  8423 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.801700  8062 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:05.870029  8062 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.870178  8147 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:05.835320  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:05.872226  1636 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 20: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:05.872445  8105 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.873338  8257 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:05.873154  8105 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:05.874154  8147 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:05.875855  8305 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:05.836018  1532 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:05.879810  8414 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:05.880492  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 64481.7us
I20260709 13:04:05.873973  8257 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:05.921579  8324 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:05.839231  8385 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:05.908483  8294 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:05.930326  1636 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:05.931355  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:05.933575  1639 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:04:05.883882  8332 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:05.930701  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 186703us
W20260709 13:04:05.937029  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:05.937489  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:05.938218  1638 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 23.
I20260709 13:04:05.938530  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:05.939271  1635 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:05.939312  1714 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:05.940019  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:05.943893  1717 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:04:05.940503  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:05.945518  1715 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:05.925381  8147 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.946740  8147 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:05.948397  8257 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.948969  8147 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.915724  8232 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:05.939765  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:05.923084  8105 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:05.939631  8173 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:05.950748  7715 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.606s	user 0.011s	sys 0.000s
I20260709 13:04:05.940943  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:05.969516  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:05.970244  1635 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:05.963032  2269 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:05.960000  2298 raft_consensus.cc:2955] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 3 to 14, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:05.969338  1718 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 12.
I20260709 13:04:05.973750  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:05.974462  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 116200us
I20260709 13:04:05.972177  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:05.979503  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:05.980592  1641 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 21: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:05.982216  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:05.983280  1568 raft_consensus.cc:2393] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 21: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:05.983520  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:05.984236  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:05.984994  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:05.985589  7872 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.986281  7872 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:05.987643  8415 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.508s	user 0.018s	sys 0.001s
I20260709 13:04:05.985616  1644 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:05.997483  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:05.998386  1715 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 21: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:05.958514  8385 raft_consensus.cc:697] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:06.001152  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:05.999755  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:05.968526  8294 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:05.977581  8423 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.977944  8305 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:06.013676  2299 raft_consensus.cc:2393] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 19: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:06.008795  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:06.011824  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:06.015952  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 345971us
I20260709 13:04:06.015981  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:06.016044  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.015799  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.025748  1715 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:06.003489  7872 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:06.030285  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:06.015542  8398 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:05.978080  7715 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.040999  8386 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:06.041466  1451 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ca83cd5e36eb4e22b75011b87bfcd49d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:04:06.038125  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.042048  1643 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:06.042297  1561 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:06.042887  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2809) took 1249 ms. Trace:
I20260709 13:04:06.043076  8431 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:06.043540  8422 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:06.043883  8434 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.076805  1641 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 13: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:06.047237  8385 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.080752  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.080123  1718 raft_consensus.cc:2955] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 3 to 14, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:06.092434  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.093869  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.094720  1641 raft_consensus.cc:2410] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:04:06.080507  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:06.095158  8384 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:06.095813  1563 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 19: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:06.067338  8392 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:06.020110  8305 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.097282  8305 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:06.097532  8384 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:06.051324  8309 raft_consensus.cc:3055] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:06.051841  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.052570  8232 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:06.007910  8173 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 13:04:06.048725  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:06.102551  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:06.104748  1643 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 20: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:06.082430  1644 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:04:06.080633  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.106976  1715 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:04:06.064110  1642 rpcz_store.cc:276] 0709 13:04:04.792964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.793169 (+   205us) service_pool.cc:224] Handling call
0709 13:04:04.793522 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.391613 (+598091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.038782 (+647169us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b540032c3d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:06.042818 (+  4036us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1137345920}
I20260709 13:04:06.052737  8423 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.107959  2269 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:06.100003  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:04:06.100456  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.114755  1714 raft_consensus.cc:2368] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18.
I20260709 13:04:06.116942  1639 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
W20260709 13:04:06.118316  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.120147  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.078068  8422 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:06.101064  8282 consensus_meta.cc:287] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.696s	user 0.012s	sys 0.000s
I20260709 13:04:06.099303  8332 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:06.135545  1641 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:04:06.070093  8062 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.101037  8309 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:06.138950  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.077888  8245 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.146220  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.146629  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:06.140617  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.147161  1711 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:06.148725  1566 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:06.148890  8442 consensus_queue.cc:1048] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:06.130996  8422 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.168111  1641 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:06.156502  1635 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:04:06.163584  1563 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 19: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:06.165556  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.170449  1715 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:04:06.183141  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.184082  1715 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 12.
I20260709 13:04:06.109540  8434 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:06.212366  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 104472us
I20260709 13:04:06.139635  8386 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:06.186656  8035 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:06.226078  2303 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:04:06.226171  1564 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:06.217859  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:06.129884  8294 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:06.203415  7545 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.211627  8390 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.172479  8062 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.231781  1715 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:06.230116  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.207315  8332 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:06.234866  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:06.229643  7887 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:06.167222  8431 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.255060  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.250280  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:06.257966  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 102910us
I20260709 13:04:06.255489  2297 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:04:06.053045  8097 raft_consensus.cc:2955] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 23 LEADER]: Committing config change with OpId 18.14: config changed from index 3 to 14, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:06.269672  2304 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:06.269656  1717 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 16: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:06.254619  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:06.280959  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.261287  8443 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:06.293576  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:06.298627  1568 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:06.315300  1715 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:06.319211  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:06.320365  2304 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:06.247589  8332 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.254745  8390 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.249356  8035 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.288822  7715 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.249550  8309 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.325886  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.341262  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:06.340471  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.341285  1717 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:04:06.343910  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.291474  7887 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:06.353731  8447 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 13:04:06.353392  8433 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:04:06.353152  8423 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:06.354683  8452 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:06.413705  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:04:06.411716  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 69923.7us
I20260709 13:04:06.331422  8462 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.327831  8439 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:06.355278  1609 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:06.363500  8440 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:06.364871  8448 log.cc:927] Time spent T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:04:06.396144  1561 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:04:06.396757  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 419680us
I20260709 13:04:06.427999  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.404139  1560 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:04:06.348769  8309 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:06.335346  8454 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:06.395864  7715 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:06.425787  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 104873us
I20260709 13:04:06.435734  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.436408  1715 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
W20260709 13:04:06.439414  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 181639us
I20260709 13:04:06.440519  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:06.339351  8443 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.450802  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:06.456230  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.435629  8036 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:06.433640  8439 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:06.428041  8386 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:06.462308  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.491461  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:06.464011  2305 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:06.445781  8434 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:06.439755  8452 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.459643  8454 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.450138  8173 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:06.489555  8465 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.493932  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.491567  1561 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:04:06.497694  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:06.498905  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:06.499611  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:06.499936  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.500277  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.500476  7518 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.500873  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:06.474881  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.503263  1636 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:06.491219  7906 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.502930  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:04:06.480105  8460 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.127s	user 0.000s	sys 0.006s
I20260709 13:04:06.502991  8062 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.439375  7715 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:06.475334  8450 log.cc:927] Time spent T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
W20260709 13:04:06.568058  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.508447  8422 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.475162  8463 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.464833  8245 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.459699  8036 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.545960  1641 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:06.652673  8465 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.653272  8431 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.648811  8463 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.650398  1562 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:04:06.656262  8463 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.625588  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.443511  8440 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:06.546432  1711 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:04:06.474221  8443 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.642349  1565 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:06.627918  1567 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:04:06.451488  8462 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.629999  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:06.648054  1568 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:06.653964  8472 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:06.612010  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:06.657107  1635 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:04:06.613086  8454 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.545378  8452 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.573426  8449 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.702976  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.703270  8449 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.705116  7715 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:06.705948  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:06.704104  1561 raft_consensus.cc:2410] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:04:06.653957  8431 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 13:04:06.679921  8460 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:04:06.696866  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:06.696759  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.698448  8332 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.649560  8035 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.699064  8467 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:04:06.700363  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:06.700011  8439 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.702598  8422 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:06.686178  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.709170  1569 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:06.726361  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.728358  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:06.729034  2301 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.731406  1635 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:06.728639  1561 raft_consensus.cc:2410] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 22 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 13:04:06.738262  8390 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:06.743336  1563 raft_consensus.cc:2410] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:04:06.743539  8267 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.743760  8384 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:06.711328  8245 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:06.746567  8245 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:06.747516  1640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 1552) took 1905 ms (client timeout 2003 ms). Trace:
W20260709 13:04:06.748122  1640 rpcz_store.cc:269] 0709 13:04:04.842261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.854813 (+ 12552us) service_pool.cc:224] Handling call
0709 13:04:06.747249 (+1892436us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54000d07d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:06.747424 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":909858560}
I20260709 13:04:06.749126  8245 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.750010  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:06.707583  2305 raft_consensus.cc:2393] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:06.706549  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:06.711052  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 211236us
I20260709 13:04:06.752396  1568 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 20: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:06.695129  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.754679  1560 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
W20260709 13:04:06.712759  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 162826us
I20260709 13:04:06.651722  8036 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:06.717700  8385 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:06.773609  8385 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.774048  8385 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:06.720216  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.775228  1641 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
W20260709 13:04:06.720734  8417 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 1.101s	user 0.001s	sys 0.004s
I20260709 13:04:06.756319  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:06.607398  8173 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.794559  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:06.795717  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.775571  8431 raft_consensus.cc:3055] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:06.805496  1569 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:06.772965  1565 raft_consensus.cc:2410] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:06.787467  1635 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
W20260709 13:04:06.758875  8305 consensus_meta.cc:287] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.661s	user 0.012s	sys 0.001s
I20260709 13:04:06.789654  8102 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:06.790548  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.774044  8463 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:06.757016  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 199091us
I20260709 13:04:06.797176  2268 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:06.813730  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.817545  1560 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:04:06.820272  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:06.816054  1568 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:04:06.821807  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.823750  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:06.806046  8431 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:06.829270  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:06.827672  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 71456.7us
I20260709 13:04:06.813838  8255 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:06.830703  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:06.831399  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:06.835341  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:06.835988  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 95681.8us
I20260709 13:04:06.838579  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.843413  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.844377  1563 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:04:06.837368  8104 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:06.853096  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:06.853266  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.853458  8432 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:06.853096  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.854367  1709 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:04:06.854758  8431 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:06.853489  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.853915  2298 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:06.857170  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.857393  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:06.858147  2298 raft_consensus.cc:2393] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 14: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:06.858466  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.858984  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:06.859714  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.855392  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.860407  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.856415  8431 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:06.861073  1713 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:04:06.860093  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.855383  8338 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:06.815554  8305 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.853925  1560 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:04:06.827409  8453 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.864588  1711 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 14.
I20260709 13:04:06.866786  1533 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:06.823324  8463 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:06.872706  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:06.872646  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.875411  1635 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:04:06.851366  8245 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:06.852128  8255 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:06.877403  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:06.878695  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:06.876942  8423 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:06.851583  8102 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.888553  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.884294  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:06.886487  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 55153.9us
I20260709 13:04:06.893711  1560 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:06.895588  1561 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:04:06.895955  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:06.908145  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.869858  8479 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:04:06.916354  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 84104.2us
I20260709 13:04:06.913328  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.917294  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:06.912068  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:06.917941  1560 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:06.923183  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:06.923623  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.924083  2298 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:04:06.915872  7872 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:06.927636  7872 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:06.930188  7872 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:06.931042  7545 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:06.931944  1641 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:06.932776  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:06.935992  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:06.893507  8478 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:06.944837  8423 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:06.950587  8104 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:06.954418  8423 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:06.954946  8423 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:06.958714  8104 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.961414  8104 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:06.962088  8432 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:06.963229  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:06.963862  1640 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:04:06.964882  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:06.965411  1640 raft_consensus.cc:2376] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19. Re-sending same reply.
W20260709 13:04:06.967674  8476 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:04:06.976647  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 94586.5us
I20260709 13:04:06.978089  7545 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:06.978649  1607 leader_election.cc:400] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18.
I20260709 13:04:06.985203  1607 leader_election.cc:403] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:04:06.979146  8173 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:06.888074  8338 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.894976  8102 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.987995  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:06.885311  8453 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:06.999423  2301 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:04:06.978582  8478 raft_consensus.cc:697] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:06.949836  8440 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:07.001451  2303 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
I20260709 13:04:07.002676  1635 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:04:07.004679  1640 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:07.007202  8490 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:07.007998  8490 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:07.015520  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 62332.5us
I20260709 13:04:07.013792  8338 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.015537  8267 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:04:07.023912  8267 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:07.024281  8267 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:07.024739  2298 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:07.024606  8398 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:07.025065  8267 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.025492  8398 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:06.996368  7545 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18.
I20260709 13:04:07.029966  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.030783  1640 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 21: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:07.030484  8478 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.026651  8267 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:07.020409  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:07.032922  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:07.033808  8482 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:07.034199  8482 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.034543  8482 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:07.035189  7545 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:06.926525  8479 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.036829  8035 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:07.038159  8479 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:07.039114  8305 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.039976  7518 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:07.041323  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:07.043253  2303 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:07.035627  7545 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.046850  8463 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.015084  8102 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.010869  8490 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.051316  8305 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:07.054988  8479 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:07.058633  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:07.062606  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:07.051977  8305 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.065233  1533 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:07.053297  7545 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:07.064045  8392 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:04:07.067185  8463 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:07.088256  8173 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.088959  8173 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.090315  8303 consensus_queue.cc:1059] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:07.091003  8102 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:07.091621  8102 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:07.092073  8309 raft_consensus.cc:3055] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:07.092223  8097 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:07.092566  8309 raft_consensus.cc:740] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:07.093582  8309 consensus_queue.cc:260] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.094879  8309 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:07.092108  8102 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:07.100198  1714 raft_consensus.cc:1240] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:07.101526  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.101856  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.102447  8305 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.070534  8479 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.102754  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.060014  8036 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.113597  1643 raft_consensus.cc:2368] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 21. Current term is 22.
I20260709 13:04:07.109359  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.114583  1713 raft_consensus.cc:2410] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:04:07.110827  8383 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.108444  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.115773  1711 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:04:07.108965  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.125239  2300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 1512) took 1614 ms. Trace:
I20260709 13:04:07.125766  2300 rpcz_store.cc:276] 0709 13:04:05.519141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.525755 (+  6614us) service_pool.cc:224] Handling call
0709 13:04:07.124975 (+1599220us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b540075e754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:07.125111 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428931968}
I20260709 13:04:07.127766  8449 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.129702  8102 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.133044  7887 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.135300  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.136683  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.104622  1715 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:07.142728  1682 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:07.147959  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.110736  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.154830  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:07.155196  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 56583.7us
I20260709 13:04:07.074275  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:07.069581  8453 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:07.060427  8035 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:07.158142  1532 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:07.149401  1714 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:04:07.118602  8432 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:07.180773  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 60110.3us
I20260709 13:04:07.181216  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:07.180835  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:07.183862  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.073472  8443 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.111831  8173 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:07.183872  1561 raft_consensus.cc:2410] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:07.181906  1568 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:07.177572  8392 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:04:07.187139  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.187150  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:07.188074  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:07.195317  1562 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:07.188190  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.196090  2301 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 14: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:07.184564  2300 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:04:07.191972  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:07.196708  1569 raft_consensus.cc:2376] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19. Re-sending same reply.
I20260709 13:04:07.183580  8449 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:07.197494  8042 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:04:07.198172  7906 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.178763  8035 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.199150  1567 raft_consensus.cc:2410] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:04:07.198235  8449 raft_consensus.cc:697] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:07.185163  8471 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.203855  1682 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:04:07.207610  1451 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ca83cd5e36eb4e22b75011b87bfcd49d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:04:07.208636  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:07.209254  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.200919  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.195185  1712 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
W20260709 13:04:07.206494  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 48455.3us
I20260709 13:04:07.158845  8385 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.190526  8465 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.212857  1682 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:07.214228  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:07.214622  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:07.227461  1713 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:04:07.216030  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3255) took 1119 ms. Trace:
I20260709 13:04:07.216369  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:07.229425  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:07.228348  1644 rpcz_store.cc:276] 0709 13:04:06.096744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.155306 (+ 58562us) service_pool.cc:224] Handling call
0709 13:04:06.188007 (+ 32701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.474432 (+286425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:06.474466 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:06.474509 (+    43us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:06.474548 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.215737 (+741189us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54001e9954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:07.215812 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.215959 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":634544512}
I20260709 13:04:07.213588  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.233898  1711 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:04:07.219800  1534 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:07.234860  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.236514  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:07.238693  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:07.241200  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:07.242102  1563 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:04:07.194983  8102 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.217029  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.245047  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:07.231972  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:07.241289  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.241228  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.245177  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:07.205629  8035 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:07.251086  1682 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:07.246860  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.252561  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:07.213215  1532 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:07.248095  7518 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:07.259454  7518 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:07.260005  7518 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:07.207337  8479 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.264982  1682 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:07.220956  8463 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:07.215461  8385 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:07.226012  8036 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.224625  8245 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:07.280123  7518 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.242165  7887 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.224635  8423 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.234133  8282 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:07.193523  8443 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.211588  8042 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:04:07.244827  8392 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:07.297924  8392 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:07.298561  8392 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:07.286418  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:07.303438  1565 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:04:07.304809  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:07.306334  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:07.259685  8384 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:07.282131  8338 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:07.289220  8453 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:07.307264  1714 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:07.307489  8338 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.307986  8384 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.308053  8338 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:07.308446  8384 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:07.309715  8423 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.294854  7887 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.281383  8391 consensus_queue.cc:1059] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:04:07.312022  8391 raft_consensus.cc:3055] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:07.312484  8391 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:07.312851  7887 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.313474  8391 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.281889  8345 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:07.268602  8465 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:07.293797  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 80702.3us
I20260709 13:04:07.277716  8463 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.320192  7518 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:07.290726  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 66500.5us
I20260709 13:04:07.217231  8449 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.224067  8471 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.325074  1710 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:07.325167  1532 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:07.328431  1713 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:04:07.290447  8282 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.317394  8062 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:07.339982  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 91074.7us
I20260709 13:04:07.324550  8465 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.329787  8463 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:07.352535  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:07.330796  8423 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.353682  1565 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:04:07.332279  8345 raft_consensus.cc:697] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:04:07.360780  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.343302  8309 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:07.290632  8440 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:04:07.344837  8062 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.361639  1532 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:07.365993  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.367547  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.373252  1635 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:07.368031  8423 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.373819  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.339051  8282 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:07.377992  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.383312  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.384208  1644 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:07.377323  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.385413  2304 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:07.373945  2268 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:07.372587  1637 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:04:07.375669  1639 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:04:07.353327  8465 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:07.379441  1681 leader_election.cc:336] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Timed out: RequestConsensusVote RPC to 127.0.107.193:36547 timed out after 1.720s (SENT)
I20260709 13:04:07.384793  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.390094  1638 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:04:07.383733  1636 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:04:07.375397  1532 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:07.363994  8391 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:07.392177  1642 raft_consensus.cc:1240] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:04:07.404193  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:07.374755  8062 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:07.388268  8501 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:04:07.404299  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:07.422428  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.355329  7887 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:04:07.421798  7518 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:04:07.428785  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 53433.6us
I20260709 13:04:07.425717  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.427150  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.430550  2300 raft_consensus.cc:2410] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:04:07.430457  8478 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:07.431386  7518 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:07.432466  8423 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.432355  7518 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.433020  1532 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:07.433795  8042 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:07.435123  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:07.409786  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.386407  8431 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:07.403301  8496 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:04:07.436777  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:07.437415  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:07.438196  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.440836  2299 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:07.374941  8097 consensus_queue.cc:1059] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:04:07.442184  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:07.440879  2296 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 17.
I20260709 13:04:07.442776  1717 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:07.446327  1532 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:07.402930  8491 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.158s
I20260709 13:04:07.460876  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:07.461800  1565 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:07.430788  8173 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:07.450138  1644 raft_consensus.cc:2955] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 3 to 14, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:07.471397  7545 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.474056  7545 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.483495  8431 raft_consensus.cc:3055] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:07.484066  8431 raft_consensus.cc:740] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:07.485116  8431 consensus_queue.cc:260] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.488662  8309 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.489241  8309 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.491183  8484 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.486593  8440 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.491750  8484 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.493849  8309 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.494088  8484 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.501353  1717 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
W20260709 13:04:07.498078  8464 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:04:07.505738  2299 raft_consensus.cc:2468] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 22.
W20260709 13:04:07.505889  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 140691us
I20260709 13:04:07.466379  8173 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:07.509589  1562 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:04:07.452452  8384 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.408133  8474 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:04:07.505753  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 65894.6us
I20260709 13:04:07.542831  8398 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.556737  1709 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:07.559357  7545 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:07.559751  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.560034  7545 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.560446  1642 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:04:07.562220  7545 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.562494  1609 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:07.564848  7545 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:07.564994  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:07.565796  8398 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.566597  1609 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:07.567998  7545 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:07.568440  7545 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.568907  7545 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:07.569773  1565 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 22.
W20260709 13:04:07.570863  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 132557us
W20260709 13:04:07.549154  8482 consensus_meta.cc:287] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.511s	user 0.004s	sys 0.008s
I20260709 13:04:07.582137  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:07.587628  1562 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:04:07.578788  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:07.513137  8440 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:07.541581  8173 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:07.588094  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:07.540915  8478 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:07.592911  1609 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:07.590855  7545 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.592744  2299 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:07.574681  8443 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.512166  8392 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.578403  8474 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:07.605284  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.605947  2296 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:07.606025  8392 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.595211  8035 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.606767  8309 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:07.607087  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3339) took 1460 ms. Trace:
I20260709 13:04:07.607260  8309 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.607666  8309 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:07.607462  1564 rpcz_store.cc:276] 0709 13:04:06.146954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.147133 (+   179us) service_pool.cc:224] Handling call
0709 13:04:06.147574 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.227714 (+ 80140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.606808 (+1379094us) spinlock_profiling.cc:243] Waited 9.57 ms on lock 0x7b54002a9c54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:07.607015 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22020864}
I20260709 13:04:07.608553  8035 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.591250  8482 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.612574  1639 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:07.613306  8490 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.613878  8490 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.616456  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.616842  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.617149  1640 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:07.617554  1635 raft_consensus.cc:2393] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 18: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:07.618054  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:07.618266  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.618624  1640 raft_consensus.cc:3060] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:07.504825  8345 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.619341  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:07.620048  1636 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 14: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:07.620862  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.621446  1636 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:04:07.622251  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.624805  1636 raft_consensus.cc:2410] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:04:07.623915  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.624933  8482 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.618810  1635 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:04:07.626792  1635 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:07.605620  8474 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.635493  2269 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:07.637068  8104 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:07.637501  8104 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.637856  8104 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:07.642903  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60378 (request call id 2834) took 1208 ms. Trace:
I20260709 13:04:07.643921  1643 rpcz_store.cc:276] 0709 13:04:06.434150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.434689 (+   539us) service_pool.cc:224] Handling call
0709 13:04:07.642697 (+1208008us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b5400124b54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:07.642835 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1522050304}
I20260709 13:04:07.644361  8384 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:07.616319  8490 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:07.627635  8487 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.649s	user 0.000s	sys 0.003s
I20260709 13:04:07.645939  8324 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:07.646608  8324 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.645939  2269 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:07.604923  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.656227  2269 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:07.657683  1533 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:07.601387  1565 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 21. Current term is 22. Ops: []
W20260709 13:04:07.656806  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.662107  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.661042  2269 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:07.665107  1533 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:07.665822  2269 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:07.667914  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:04:07.669700  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 223353us
I20260709 13:04:07.598611  8036 raft_consensus.cc:2804] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:07.669237  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.674999  2269 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:07.587419  8097 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:04:07.679956  1638 raft_consensus.cc:2410] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:04:07.682964  1710 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 14.
I20260709 13:04:07.688217  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:07.689033  1636 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:07.690238  7545 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.692349  7518 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:07.694018  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.694695  1713 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:04:07.694018  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:07.692981  7518 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.699457  7518 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:07.666097  8422 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:07.701141  1713 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:07.657228  8392 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:07.680570  8479 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:07.663756  8423 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:07.701802  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:07.600252  8440 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.679840  8432 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:07.714227  1640 raft_consensus.cc:2468] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:07.717159  2269 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:07.664678  8508 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:04:07.694519  8036 raft_consensus.cc:697] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 22 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:04:07.758368  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.710454  7887 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:07.710788  8422 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:07.763038  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 45933.5us
W20260709 13:04:07.713238  8451 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.359s	user 0.022s	sys 0.011s
I20260709 13:04:07.716176  8432 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.772078  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:07.746256  8398 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:07.768738  8324 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.781997  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.783248  1713 raft_consensus.cc:2410] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:04:07.783571  8241 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:07.784087  8241 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.762691  7887 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.785449  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:07.788843  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 98202.8us
I20260709 13:04:07.790249  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:07.791788  1711 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:04:07.767601  8451 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.414s	user 0.023s	sys 0.011s
I20260709 13:04:07.714035  8392 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.780217  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:07.800040  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:07.784264  8282 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.804175  8484 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:07.767732  8422 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:07.775483  8432 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:07.746225  8511 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:04:07.791749  8097 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:07.811306  8097 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.813719  8097 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:07.815423  8481 log.cc:927] Time spent T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.844s	user 0.004s	sys 0.000s
I20260709 13:04:07.819424  8036 consensus_queue.cc:237] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.823964  8440 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.770300  8479 raft_consensus.cc:2804] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:07.784495  8241 raft_consensus.cc:3060] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:07.833271  8338 consensus_meta.cc:287] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.001s
I20260709 13:04:07.795964  7518 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.833748  8338 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.834729  8392 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.786052  7887 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:07.836445  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:07.845274  1640 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:07.829432  8422 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.837019  7518 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:07.837121  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 57002.9us
I20260709 13:04:07.851120  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:07.842360  1636 raft_consensus.cc:2468] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:04:07.857997  1640 raft_consensus.cc:2468] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:07.861320  2269 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:07.847090  8484 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:07.826261  8104 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.828727  8309 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.829430  8062 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.864291  8479 raft_consensus.cc:697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:07.883909  8513 raft_consensus.cc:2804] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:07.903908  8282 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:07.849808  8449 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:04:07.872275  8264 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:04:07.938040  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 70370.3us
W20260709 13:04:07.938791  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 273585us
I20260709 13:04:07.944844  1533 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:07.946854  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:04:07.941273  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.951009  2302 raft_consensus.cc:2368] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19.
I20260709 13:04:07.961512  1533 leader_election.cc:400] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 21. Current term is 22.
I20260709 13:04:07.965291  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:07.945179  8518 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.110s	user 0.000s	sys 0.005s
I20260709 13:04:07.959782  8522 raft_consensus.cc:2764] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:04:07.980046  1717 raft_consensus.cc:1240] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:07.960179  8484 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:07.981040  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:07.981356  8484 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.968050  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.965355  8241 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:07.983952  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:07.984720  1717 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:07.984961  8241 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:07.950560  8391 consensus_meta.cc:287] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.004s
I20260709 13:04:07.971740  8414 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:07.986333  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:07.972050  8062 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.972836  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:07.974128  8338 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:07.976184  8309 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:07.963578  8513 raft_consensus.cc:697] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:07.994454  8513 consensus_queue.cc:237] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.981204  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:07.999662  8465 consensus_meta.cc:287] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.572s	user 0.011s	sys 0.000s
I20260709 13:04:07.999176  8432 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.000125  8465 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:08.002149  8465 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:08.004281  1534 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:08.006244   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:08.006326  8241 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:08.007274  8241 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:07.965960  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:08.013911  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:07.987221  1717 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:07.990533  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:07.986743  8414 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:07.947065  8506 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:08.002897  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:07.965677  8474 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:07.974426  8423 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:08.023116  2301 raft_consensus.cc:1240] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:08.027021  1640 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:08.042765  1683 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:08.041883  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 104455us
I20260709 13:04:08.047271  1637 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:08.051666  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:08.057363  2299 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:08.069614  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:08.070420  1640 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:08.070580  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:08.070447  1638 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:04:08.076628  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:08.078656  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.041338  8474 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.078660  2303 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:04:08.088466  1681 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:08.090178  1681 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:08.061048  8324 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:08.082960  8479 consensus_queue.cc:237] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.108872  1532 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:08.105356  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:08.114005  1681 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:08.114867  2268 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:08.082212  8432 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:08.088726  8474 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:08.119539  8529 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:08.046522  8506 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:08.090988  8246 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:08.125931  2305 raft_consensus.cc:1240] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:08.065503  8173 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.082485  8523 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:08.087386  8522 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:08.111217  8264 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:08.132277  1681 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:08.127625  1682 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:08.109086  8398 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:08.113665  8422 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:08.115427  8423 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:08.139398  8449 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:08.127673  8472 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:08.174398  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:08.175151  1644 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:08.128659  2299 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:08.149170  8384 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:08.150308  8523 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:08.180503  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 102778us
I20260709 13:04:08.160261  8465 raft_consensus.cc:2804] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:08.160825  1681 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:08.161314  8286 consensus_queue.cc:1059] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:04:08.163152  8506 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:08.164866  8042 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:08.182682  2297 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: []
W20260709 13:04:08.182946  8531 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:04:08.185827  8498 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.859s	user 0.000s	sys 0.004s
I20260709 13:04:08.187345  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:08.188020  1642 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:04:08.189462  8414 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:08.190701  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:08.131204  8390 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:04:08.195667  1642 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:04:08.197754  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:08.166854  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:08.198623  1637 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:04:08.198962  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:08.199844  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:08.200510  1644 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:04:08.200621  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:08.201545  1635 raft_consensus.cc:2393] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:08.142185  8515 consensus_queue.cc:1059] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:04:08.203418  8515 raft_consensus.cc:3055] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:08.197091  1682 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:08.205823  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:08.206118  2269 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:08.168105  7887 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:08.168869  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:08.170287  8471 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:08.170445  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:08.216410  1640 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:08.171446  8462 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:08.145532  8525 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:08.158396  8449 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.159235  8435 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:08.235642  1640 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:04:08.183001  8042 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:04:08.214532  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.215782  1644 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:08.219663  1642 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:08.203716  8472 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:08.222546  8386 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:08.151965  8522 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:08.203768  8515 raft_consensus.cc:740] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:08.213652  1636 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:08.236964  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:08.237421  2269 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:08.245289  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 84500.9us
I20260709 13:04:08.222489  8471 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:08.251521  1681 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:08.247643  1637 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:08.246737  1682 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:08.238072  8449 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:08.236497  8414 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.261034  8511 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:08.261694  8511 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:08.255745  8471 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.263638  8471 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:08.263852  8173 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:08.263659  8532 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:04:08.264385  8390 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:08.265745  8414 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.252463  8286 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 13:04:08.267313  8042 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:08.255020  8264 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.268821  8042 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:08.271649  8515 consensus_queue.cc:260] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:08.273240  8515 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:08.273087  8471 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.252256  8472 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.276511  8471 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:08.277330  8472 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:08.280988  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:08.281256  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:08.281713  1637 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:08.284507  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:08.286137  1681 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:08.244634  8465 raft_consensus.cc:697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:08.250028  8324 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:08.290710  8246 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:08.262727  8511 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.291225  8386 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:08.291857  8245 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:08.293596  8536 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:08.296360  7887 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:08.300700  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:08.312307  1709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1389) took 1109 ms. Trace:
I20260709 13:04:08.317919  2296 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 21. Current term is 22. Ops: []
W20260709 13:04:08.289572  8463 consensus_meta.cc:287] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.920s	user 0.005s	sys 0.013s
I20260709 13:04:08.328759  1709 rpcz_store.cc:276] 0709 13:04:07.202508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.202658 (+   150us) service_pool.cc:224] Handling call
0709 13:04:07.203218 (+   560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:07.557242 (+354024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.312016 (+754774us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b5400339fd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:08.312239 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":812033408}
W20260709 13:04:08.334831  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 47025.6us
I20260709 13:04:08.340137  1681 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:08.298887  8523 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:08.346163  2297 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:04:08.349354  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 102644us
I20260709 13:04:08.351543  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:08.330212  8536 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:08.319847  8245 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:08.319314  8479 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:08.365760  2297 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:08.367303  8547 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:08.367921  8414 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:08.368638  8545 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:08.369128  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:08.369429  8324 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:08.369165  8545 raft_consensus.cc:697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:08.330147  8463 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:08.364305  1637 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 22.
I20260709 13:04:08.376952  2297 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:08.376499  8547 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:08.380363  2269 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:08.381554  8547 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:08.383140  8453 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:08.383792  8453 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:08.384779  8453 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.392637  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:08.393373  1713 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:08.393200  8465 consensus_queue.cc:237] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:08.360526  8245 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:08.403092  8522 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:08.404032  8545 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
W20260709 13:04:08.405481  1607 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.227s	user 0.052s	sys 0.070s
W20260709 13:04:08.405829  1607 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.229s	user 0.052s	sys 0.070s
I20260709 13:04:08.411118  8463 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:08.371683  8479 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.419576  8036 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.420202  8036 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:08.423097  8538 consensus_queue.cc:1059] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:04:08.424577  8536 raft_consensus.cc:3055] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:08.424945  8536 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:08.425765  8536 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.392002  8386 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:08.431488  1642 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:08.433815  8264 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:04:08.437635  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.441771  8469 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.442442  8469 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:08.455336  8384 raft_consensus.cc:3055] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:08.455962  8384 raft_consensus.cc:740] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:04:08.456523  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.456943  8384 consensus_queue.cc:260] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.458551  8384 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:08.465521  8104 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:08.481112  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 1.40611e+06us
W20260709 13:04:08.481797  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.482079  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:08.483471  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:08.483647  1561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:08.501940  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:08.488952  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:08.504112  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:08.508975  1561 raft_consensus.cc:606] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:08.503170  1566 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:08.498883  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:08.518050  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:08.509621  1569 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:04:08.516139  1638 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:08.517084  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:08.509857  1561 raft_consensus.cc:614] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:08.525480  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:08.526288  1568 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:08.485877  8536 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:08.486084  8431 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:08.533578  1562 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:08.498382  8036 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:08.487571  8482 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:08.547047  1534 leader_election.cc:400] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19.
I20260709 13:04:08.547569  1534 leader_election.cc:403] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:04:08.548342  1713 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:04:08.509196  8506 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:08.498503  8479 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:08.552798  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:08.526408  8035 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:08.491025  8491 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 13:04:08.545604  8391 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:08.567503  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2848) took 1185 ms. Trace:
I20260709 13:04:08.565780  1532 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:08.567947  1639 rpcz_store.cc:276] 0709 13:04:07.381646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.392841 (+ 11195us) service_pool.cc:224] Handling call
0709 13:04:07.423572 (+ 30731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:07.612948 (+189376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.567251 (+954303us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400154d54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:08.567439 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434058240}
I20260709 13:04:08.550452  8482 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:08.577108  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 236928us
I20260709 13:04:08.578508  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:08.583254  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:08.565388  8390 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:04:08.593192  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.552902  8392 consensus_queue.cc:1059] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 13:04:08.595711  1609 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:08.549893  8525 consensus_queue.cc:1059] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:04:08.602975  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:08.681600  1709 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:08.611735  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:08.682561  1718 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:08.613998  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:08.683321  1715 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:08.683799  1637 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 22 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:08.679450  1710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33280 (request call id 1392) took 1379 ms. Trace:
I20260709 13:04:08.578661  8491 consensus_queue.cc:1243] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 22.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:04:08.607805  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:08.579531  8482 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:08.574780  8414 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:04:08.704104  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 186971us
I20260709 13:04:08.704936  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:08.705317  2302 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:04:08.684473  1710 rpcz_store.cc:276] 0709 13:04:07.208787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.209439 (+   652us) service_pool.cc:224] Handling call
0709 13:04:08.588229 (+1378790us) spinlock_profiling.cc:243] Waited 98.1 ms on lock 0x7b54003dc2d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:08.588334 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225615104}
I20260709 13:04:08.696973  2268 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:08.685249  1564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:47594 (request call id 2) took 1206 ms. Trace:
I20260709 13:04:08.700974  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:08.702185  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:08.708765  1564 rpcz_store.cc:276] 0709 13:04:07.386035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.608753 (+222718us) service_pool.cc:224] Handling call
0709 13:04:08.591947 (+983194us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b54001ffc54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca7735 00007fd7a1c9f9fa 00007fd7a6c674c9 00007fd7a01e3c54 00007fd7a01e3be2 00007fd7a01e3b11 00007fd7a01e3a8c 00007fd7a01e2d42 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:08.592120 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":869074688}
I20260709 13:04:08.709350  1637 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:04:08.711318  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:08.711515  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:08.712276  1644 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 15: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:08.717635  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:08.718458  1711 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:08.719420  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:08.571074  8264 consensus_queue.cc:1059] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.123s
I20260709 13:04:08.571513  8484 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.117s
I20260709 13:04:08.679754  8506 raft_consensus.cc:2764] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:04:08.749605  1532 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:08.718887  8487 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:04:08.587448  8398 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.151s
I20260709 13:04:08.594779  8390 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:08.746145  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:08.743732  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1690) took 1072 ms. Trace:
I20260709 13:04:08.586292  8474 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:08.773017  2303 rpcz_store.cc:276] 0709 13:04:07.671098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.671255 (+   157us) service_pool.cc:224] Handling call
0709 13:04:07.694330 (+ 23075us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.084230 (+389900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.743469 (+659239us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b5400516dd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:08.743669 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882841856}
I20260709 13:04:08.769860  1718 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 22.
I20260709 13:04:08.736949  8422 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:08.780450  8422 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:08.780915  8422 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:08.783684  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.784478  8474 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:08.784693  8511 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:08.785763  8542 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:04:08.785409  8511 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.743736  8414 raft_consensus.cc:697] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:08.787715  8414 consensus_queue.cc:237] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:08.719893  8469 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:08.778167  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:08.779538  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:08.780025  8544 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:04:08.818044  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 68490.8us
I20260709 13:04:08.854209  1566 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:08.808502  1712 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:08.712666  8324 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
W20260709 13:04:08.868456  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 163538us
I20260709 13:04:08.835618  1568 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:04:08.870155  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:08.762517  8036 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:08.881657  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:08.876072  2298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1898) took 1685 ms. Trace:
I20260709 13:04:08.767144  8525 consensus_queue.cc:1243] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:04:08.777820  8390 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:08.889940  1532 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:08.884713  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:08.891455  1710 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:08.845624  8449 consensus_meta.cc:287] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.570s	user 0.003s	sys 0.010s
I20260709 13:04:08.882506  2298 rpcz_store.cc:276] 0709 13:04:07.190424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.190582 (+   158us) service_pool.cc:224] Handling call
0709 13:04:07.192903 (+  2321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:07.193107 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:04:07.206200 (+ 13093us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.292926 (+1086726us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b18004b8da0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a 00007fd7a1caba2a
0709 13:04:08.858723 (+565797us) log.cc:844] Serialized 14 byte log entry
0709 13:04:08.875714 (+ 16991us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:08.875770 (+    56us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:08.875816 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:08.875883 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.876012 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16676,"prepare.queue_time_us":50511,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":4154,"raft.queue_time_us":20853,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":1911,"spinlock_wait_cycles":12306688,"thread_start_us":564994,"threads_started":2,"wal-append.queue_time_us":1364115}
W20260709 13:04:08.862694  8042 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.577s	user 0.015s	sys 0.000s
I20260709 13:04:08.844837  8523 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:08.897974  8449 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:08.890969  8035 raft_consensus.cc:2764] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:04:08.908010  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.882411  7887 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.886288  8547 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.975248  8547 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:08.863377  8462 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19.
I20260709 13:04:08.923962  1713 raft_consensus.cc:2410] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:04:08.897400  8525 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:04:08.980082  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.980304  8525 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.980800  8525 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:08.881444  8487 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:04:08.982702  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:08.983582  8435 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:08.983711  1568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:08.909787  8305 raft_consensus.cc:3055] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:08.984555  8305 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:08.985395  1566 raft_consensus.cc:3055] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:08.985824  1566 raft_consensus.cc:740] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:08.985518  8305 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.972832  8551 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:08.983597  2298 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 14: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:08.984611  1568 raft_consensus.cc:606] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:04:08.959815  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 78184.4us
I20260709 13:04:08.986559  1566 consensus_queue.cc:260] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:08.987639  1568 raft_consensus.cc:614] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:08.984059  8435 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:08.970892  1710 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
W20260709 13:04:08.988808  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.991740  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:08.992681  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:08.996930  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:08.992832  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:08.998036  1567 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:08.988152  1566 raft_consensus.cc:3060] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:09.000224  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.001142  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:09.000844  8435 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.002542  8435 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:09.019398  8534 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:08.986960  8305 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:09.020001  8534 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.022162  8534 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:09.023475  8391 raft_consensus.cc:3055] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:09.023932  8391 raft_consensus.cc:740] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:09.024827  1711 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:09.025760  8449 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:09.027824  1451 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ca83cd5e36eb4e22b75011b87bfcd49d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3f64b76180a54492bb015e405caeacec is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:09.028771  8558 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:09.031010  8550 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.031596  8550 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.033782  8550 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:09.034287  8544 log.cc:927] Time spent T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 13:04:09.036834  8515 consensus_meta.cc:287] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.763s	user 0.013s	sys 0.003s
I20260709 13:04:09.037539  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:09.038295  1644 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:04:09.039299  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:09.039918  1640 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:09.040401  1566 raft_consensus.cc:1275] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:04:09.040874  1566 pending_rounds.cc:85] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 12
I20260709 13:04:09.041210  1566 pending_rounds.cc:107] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:04:09.041576  1566 raft_consensus.cc:2889] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:09.042351  8529 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 13:04:09.007413  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:09.008065  1532 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:09.008091  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.046672  1565 raft_consensus.cc:2393] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 18: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:09.046684  1533 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:09.047041  1562 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 23.
I20260709 13:04:09.001735  1560 raft_consensus.cc:3060] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:09.003074  8484 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:09.047544  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 62163.1us
W20260709 13:04:09.053067  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.055480  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:09.056770  2301 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:04:08.954308  8511 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:09.059468  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 51490.9us
I20260709 13:04:09.069847  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:09.073611  1640 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:09.074263  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:09.050174  8245 consensus_meta.cc:287] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.654s	user 0.011s	sys 0.002s
I20260709 13:04:09.076331  8482 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.079082  8392 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:09.084268  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:09.102587  1560 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:09.103170  1565 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 15: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:09.103538  1533 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:09.066442  8536 consensus_meta.cc:287] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.520s	user 0.008s	sys 0.001s
W20260709 13:04:09.098146  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.106344  1562 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17. Ops: [14.11-16.12]
W20260709 13:04:09.087139  1638 consensus_meta.cc:287] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.546s	user 0.004s	sys 0.005s
I20260709 13:04:09.065992  8515 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:04:09.097553  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 339955us
I20260709 13:04:09.137190  1638 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:09.139505  1711 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:04:09.142784  1709 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:09.129714  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:09.144966  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 44394.2us
I20260709 13:04:09.144141  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:09.145115  1711 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:04:09.150851  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.009s	sys 0.011s
W20260709 13:04:09.151328  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.009s	sys 0.011s
W20260709 13:04:09.151671  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 799784us
I20260709 13:04:09.152397  1682 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:09.071460  8508 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:09.071267  8474 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:09.088326  8545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.071s	sys 0.074s
W20260709 13:04:09.158548  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 55062.6us
I20260709 13:04:08.996259  7887 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.182344  1565 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:09.184478  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:09.185474  1636 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:04:09.072041  8036 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:09.179577  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.096884  8042 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:09.191411  8392 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:09.192340  8515 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:09.182085  8558 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:09.193403  8515 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:09.193531  8558 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:09.199975  8431 consensus_meta.cc:287] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.650s	user 0.010s	sys 0.000s
I20260709 13:04:09.200839  1564 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:04:09.203042  8534 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:09.203636  8534 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.206902  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:09.208935  7887 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:09.210500  1643 raft_consensus.cc:1275] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:04:09.211339  8264 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:09.105813  8384 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:09.211939  8384 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.211975  8264 raft_consensus.cc:697] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:09.212469  8384 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:09.175244  8390 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.213574  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:09.215878  2296 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:09.216748  1681 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:09.215298  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:09.217723  1566 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:09.220081  1640 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:09.227758  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:09.228618  2304 raft_consensus.cc:2393] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:09.207916  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:09.231241  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:09.231876  2268 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:09.205809  8534 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:09.232658  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.233494  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:09.240836  1569 raft_consensus.cc:2368] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18.
I20260709 13:04:09.242023  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.234450  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:04:09.243048  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 90715.3us
I20260709 13:04:09.255479  1640 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:09.158521  8432 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.062875  8484 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:09.175740  8545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.074s	sys 0.085s
I20260709 13:04:09.196635  8558 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:09.260632  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 43931.7us
I20260709 13:04:09.263979  1560 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:04:09.176627  8097 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.098577  8245 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:09.270141  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:09.278544  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:09.177731  8523 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:09.285354  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.285987  1636 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:09.286844  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:09.287514  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:04:09.304965  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 73196.4us
I20260709 13:04:09.215356  8390 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:09.269337  1561 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 19: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:09.297941  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.264810  1717 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:04:09.307865  1569 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 23.
I20260709 13:04:09.216595  8469 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.306490  2268 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:09.301308  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:09.311363  1715 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:04:09.313791  1561 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:04:09.315397  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:09.316642  1567 raft_consensus.cc:2410] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 13:04:09.308133  2269 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:09.236780  7887 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.320179  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:09.320262  1639 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:09.329178  1711 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:09.331763  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.332263  1568 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:04:09.332438  1560 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:04:09.333273  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.333918  1564 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:04:09.280685  8173 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:09.334730  8173 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.337172  8173 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:09.337378  8338 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:09.310681  8324 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:09.338223  8324 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:09.324743  8451 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:09.331094  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:09.339900  1716 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 14: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:09.341037  8546 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:09.341549  8391 consensus_queue.cc:260] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.343277  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:09.229413  8423 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:09.337420  1681 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:09.344307  8423 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.346556  8398 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:04:09.347136  8398 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:09.347218  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:09.348130  1712 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 20: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:09.321272  1713 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:04:09.349470  8537 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.288990  8559 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:09.233628  8036 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.266929  8386 raft_consensus.cc:2804] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:09.289534  8432 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.329259  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:09.280831  8465 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:09.218211  8474 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.342697  8391 raft_consensus.cc:3060] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:09.213124  8264 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.316653  1640 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:04:09.369638  1715 raft_consensus.cc:2393] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:09.351039  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:09.372787  1569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:09.356771  8423 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:09.373793  8497 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.011s	sys 0.028s
I20260709 13:04:09.371532  8173 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:09.375384  8386 raft_consensus.cc:697] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:09.341665  8546 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:09.431567  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 254774us
W20260709 13:04:09.369911  8422 consensus_meta.cc:287] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.589s	user 0.007s	sys 0.003s
I20260709 13:04:09.444996  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:09.448110  8414 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:09.372663  8390 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.065s
I20260709 13:04:09.349990  8537 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.422657  1569 raft_consensus.cc:606] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:09.452118  1569 raft_consensus.cc:614] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:09.369311  8511 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:09.375900  8451 raft_consensus.cc:697] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:04:09.440989  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 283356us
W20260709 13:04:09.441581  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 334305us
I20260709 13:04:09.327909  8469 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:09.348142  8398 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.443831  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:09.432248  8559 raft_consensus.cc:697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:04:09.427392  8497 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.012s	sys 0.028s
I20260709 13:04:09.426534  8036 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:09.369073  7887 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:09.428062  8345 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.092s	sys 0.123s
I20260709 13:04:09.418172  1717 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 23: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:09.472100  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:09.477437  1716 raft_consensus.cc:2393] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 21: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:09.312649  8097 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:09.476320  1533 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:09.483485  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:09.486260  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.449754  8422 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.490983  1533 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:09.476269  8511 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:09.486660  8345 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.092s	sys 0.124s
I20260709 13:04:09.522639  1533 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:09.523442  2302 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:09.487057  8522 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:09.492040  8508 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:04:09.526556  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 208136us
I20260709 13:04:09.492238  8482 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:09.520291  8511 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:09.559701  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:09.560339  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:09.528656  8431 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:09.528609  8062 raft_consensus.cc:3055] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:09.568547  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:09.566316  1568 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:04:09.530503  8508 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:04:09.577710  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 271111us
I20260709 13:04:09.608398  2268 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:09.660315  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:09.666580  1715 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:04:09.687465  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.688239  1569 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:09.689110  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:09.606549  1607 connection.cc:256] RPC call timeout handler was delayed by -2.2078886749999924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
W20260709 13:04:09.691527  1607 connection.cc:256] RPC call timeout handler was delayed by -2.1437542699999881s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
W20260709 13:04:09.692232  1607 connection.cc:256] RPC call timeout handler was delayed by -2.0915117150000242s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
W20260709 13:04:09.692680  1607 connection.cc:256] RPC call timeout handler was delayed by -2.0696122649999893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
W20260709 13:04:09.693078  1607 connection.cc:256] RPC call timeout handler was delayed by -1.9916542699999695s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1694s for a response
W20260709 13:04:09.693475  1607 connection.cc:256] RPC call timeout handler was delayed by -1.8585579929999767s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2088s for a response
W20260709 13:04:09.693868  1607 connection.cc:256] RPC call timeout handler was delayed by -1.1194121750000363s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
W20260709 13:04:09.699225  8568 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 13:04:09.700901  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 48763.2us
I20260709 13:04:09.703013  8432 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:09.704154  8305 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:09.704420  8474 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:09.706427  8423 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:09.707268  8566 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:04:09.707283  8512 consensus_queue.cc:1048] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 13:04:09.711844  8537 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:09.539307  8386 consensus_queue.cc:237] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.726224  8546 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:09.686045  8462 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:04:09.732054  8571 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.219s	user 0.000s	sys 0.000s
I20260709 13:04:09.569468  8451 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.733703  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1592) took 1297 ms. Trace:
I20260709 13:04:09.734095  1642 rpcz_store.cc:276] 0709 13:04:08.435978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.436137 (+   159us) service_pool.cc:224] Handling call
0709 13:04:08.436477 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.436890 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:08.436918 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:09.028061 (+591143us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b2c001080d8. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:04:09.028459 (+   398us) log.cc:844] Serialized 26 byte log entry
0709 13:04:09.284201 (+255742us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:09.284324 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.284513 (+   189us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:09.733232 (+448719us) raft_consensus.cc:1685] finished
0709 13:04:09.733257 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.733636 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2627,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":115,"spinlock_wait_cycles":50482048,"thread_start_us":255161,"threads_started":1,"wal-append.queue_time_us":444638}
W20260709 13:04:09.736734  8572 log.cc:927] Time spent T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:04:09.738963  2299 raft_consensus.cc:1240] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:04:09.739256  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:04:09.740000  8564 log.cc:927] Time spent T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.324s	user 0.001s	sys 0.000s
I20260709 13:04:09.740307  1569 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:09.741736  8555 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 13:04:09.551360  8471 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.108s
I20260709 13:04:09.743074  8173 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:09.743461  8484 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:09.744367  8565 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.226s	user 0.000s	sys 0.000s
I20260709 13:04:09.643527  8508 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:09.649686  8560 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.327s	user 0.001s	sys 0.000s
I20260709 13:04:09.573426  8062 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:09.749716  8062 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.750506  2300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:54964
I20260709 13:04:09.751055  8062 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:09.751140  2300 raft_consensus.cc:606] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:09.751662  2300 raft_consensus.cc:614] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:09.753122   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:04:09.642077  8559 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:09.756107  8525 consensus_meta.cc:287] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.775s	user 0.012s	sys 0.001s
I20260709 13:04:09.756614  8525 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:09.767336   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:09.788856  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.790230  1566 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:04:09.795373  1569 raft_consensus.cc:2468] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:04:09.757982  8491 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:09.807715  1610 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.811493  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:04:09.802062  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 257925us
I20260709 13:04:09.823681  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.809769  8097 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:09.830417  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.831151  1565 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:09.832031  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.832808  1565 raft_consensus.cc:2410] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:04:09.816859  8338 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:09.808030  8491 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:09.860744  1565 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
W20260709 13:04:09.864610  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 162571us
I20260709 13:04:09.867662  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.839397  8338 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:09.861600  8529 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:09.869640  8523 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.839130  8482 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:04:09.885376  8338 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:09.895356  8583 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 13:04:09.896452  8581 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:04:09.896621  8384 consensus_meta.cc:287] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.684s	user 0.007s	sys 0.002s
I20260709 13:04:09.897125  8384 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.886705  8173 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:09.899344  8173 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.899933  8245 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:09.902172  8173 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:09.904896  8384 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:09.902858  8338 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.911295  8556 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:09.912870  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:09.913388  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:09.914033  1636 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:09.915727  1639 raft_consensus.cc:2393] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 21: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:09.911289  8338 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:09.889204  8584 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.884910  7887 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:09.943885  8582 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:04:09.953516  8491 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:09.956753  8324 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.958171  8324 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:09.920240  8529 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.960871  8529 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:09.921190  8523 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:09.965337  8523 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:09.969110  8479 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.969766  8479 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.970769  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:09.973795  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 85685.8us
I20260709 13:04:09.979338  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:09.980647  1636 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
W20260709 13:04:09.981477  1534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.990149  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:09.971086  8423 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:09.972865  8173 consensus_queue.cc:1048] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:04:09.992880  8423 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:09.980623  8305 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:04:09.995244  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:09.995638  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:09.998298  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:09.998843  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:10.000007  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:09.994261  8479 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:10.003314  8422 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:09.996831  8390 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:04:10.003834  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:10.036768  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:10.037961  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:09.997196  8432 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:10.004971  8036 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.039209  8432 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:10.039634  8036 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:10.039669  8432 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:10.040371  8036 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.041065  8036 raft_consensus.cc:2764] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:04:09.995582  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:09.938110  8584 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.006181  8423 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:10.006327  8482 raft_consensus.cc:2764] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:04:10.006763  8525 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:10.007323  8391 consensus_meta.cc:287] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.592s	user 0.003s	sys 0.012s
I20260709 13:04:10.044001  8584 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:10.045404  1563 raft_consensus.cc:2410] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:04:10.044277  8305 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:10.022058  8577 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:10.048839  8423 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.049368  8423 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.030655  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:10.051584  8423 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:10.051810  1568 raft_consensus.cc:2410] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:10.059073  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:10.063421  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:10.082111  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:10.062103  8482 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:10.083163  1568 raft_consensus.cc:2410] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:04:10.046701  8511 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.094410  2300 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 21: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:04:10.097443  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 59538.4us
I20260709 13:04:10.098956  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:10.108026  1717 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:04:10.128118  8511 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:10.130636  8432 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.130673  1562 raft_consensus.cc:1275] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:04:10.131685  1562 pending_rounds.cc:85] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 15
I20260709 13:04:10.131920  1562 pending_rounds.cc:107] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:04:10.132143  1562 raft_consensus.cc:2889] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:10.133392  8432 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:10.135113  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:10.136023  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:10.136960  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:10.137182  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:10.138082  1642 raft_consensus.cc:2410] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:04:10.138242  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:10.139019  2302 raft_consensus.cc:2393] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:10.139616  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:10.089704  8482 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.140268  1640 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:04:10.140776  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:10.141139  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:10.141376  1639 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:10.141750  1640 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:10.141902  8482 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:10.142304  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:10.098294  8508 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:10.108801  8036 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:10.144459  8036 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.144773  8508 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:10.145491  1641 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:04:10.146518  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:10.144840  8036 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:10.138821  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:10.142839  1639 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
I20260709 13:04:10.138681  8469 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:10.151656  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:10.150707  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:10.152818  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:10.153604  1639 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:10.152318  1563 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 23.
I20260709 13:04:10.157519  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:10.158265  1564 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:10.163499  2269 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:10.163694  2296 raft_consensus.cc:1217] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Deduplicated request from leader. Original: 20.14->[21.15-21.15]   Dedup: 21.15->[]
I20260709 13:04:10.164201  2296 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:04:10.165000  1682 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:10.165279  8453 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:10.165745  8453 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.166077  8453 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:10.168885  8386 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:10.157552  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:10.157673  8550 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.168906  1563 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:10.181066  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:10.181994  1562 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:10.182163  2268 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:10.182845  1563 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:10.184091  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:10.185249  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:10.189540  1640 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:04:10.194095  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:10.194718  1640 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
W20260709 13:04:10.201506  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 43974.3us
I20260709 13:04:10.166220  8036 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.206076  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:10.208343  1607 leader_election.cc:400] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18.
I20260709 13:04:10.201506  8264 raft_consensus.cc:2804] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:10.229303  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:10.230399  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:10.253314  1607 leader_election.cc:403] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:04:10.253643  8264 raft_consensus.cc:697] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:10.219705  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:10.232034  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:10.255051  2296 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:04:10.254829  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 52035.6us
I20260709 13:04:10.256893  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:10.235672  2268 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:10.240502  1562 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:04:10.243094  1564 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:10.254352  8595 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18.
I20260709 13:04:10.255470  8264 consensus_queue.cc:237] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:10.269466  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 66845.1us
I20260709 13:04:10.270187  1636 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 20: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:10.276566  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:10.206071  8482 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:10.277189  1643 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
W20260709 13:04:10.222852  8567 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.776s	user 0.001s	sys 0.004s
I20260709 13:04:10.183701  8384 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 13:04:10.280661  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:10.295871  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:10.283170  2268 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:10.284750  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.297673  1711 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:04:10.273968  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:10.231496  8432 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:04:10.289242  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 124248us
I20260709 13:04:10.289819  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:10.306869  1641 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 13.
I20260709 13:04:10.267241  8563 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.199872  8550 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:10.286321  8596 log.cc:927] Time spent T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:04:10.275079  8423 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:10.317832  2269 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:10.308554  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 77251.7us
I20260709 13:04:10.325807  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:04:10.311966  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 68311.7us
I20260709 13:04:10.269073  8338 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:10.310823  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:10.277776  1643 pending_rounds.cc:85] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 9
I20260709 13:04:10.331555  1643 pending_rounds.cc:107] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:04:10.331924  1643 raft_consensus.cc:2889] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:10.329028  1644 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:04:10.304423  8398 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.297366  8594 consensus_queue.cc:1059] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:04:10.336302  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:10.335476  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 65269.1us
I20260709 13:04:10.338178  2302 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:10.338121  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:10.305311  8432 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:10.228408  8453 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:10.342830  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 74355.8us
W20260709 13:04:10.343479  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 60535.1us
I20260709 13:04:10.324517  8469 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.347275  1639 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:10.347990  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:10.366247  8479 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:10.366712  8559 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:10.345561  8432 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:10.367616  8559 raft_consensus.cc:697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:10.369102  8559 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.338418  8338 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.340624  8398 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:10.372988  8338 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:10.373402  8398 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:10.336757  8173 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:10.381572  1714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1405) took 1777 ms. Trace:
I20260709 13:04:10.382077  1714 rpcz_store.cc:276] 0709 13:04:08.603956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.604089 (+   133us) service_pool.cc:224] Handling call
0709 13:04:08.711066 (+106977us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.739607 (+ 28541us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:04:08.739646 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:08.771418 (+ 31772us) log.cc:844] Serialized 26 byte log entry
0709 13:04:09.016574 (+245156us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b480009ffd8. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a6bcaaa1 00007fd7a6bc1b37 00007fd7a6bc19ca 00007fd7a6cb926f 00007fd7a6cb8f37 00007fd7a6ccd55c 00007fd7a6ccd508 00007fd7a6ccd488 00007fd7a6ccd443 00007fd7a6ccc6ec 00007fd7a5bf6a96
0709 13:04:09.136485 (+119911us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:09.136530 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.136597 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.222501 (+1085904us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b24001fd228. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79ea3a9c4 00007fd79ea38acb 00007fd79ebf3dc9 00007fd79ebf3a55 00007fd79ebf309d 00007fd7a182510e 00007fd7a183c88c 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221
0709 13:04:10.381315 (+158814us) raft_consensus.cc:1685] finished
0709 13:04:10.381350 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.381503 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48643,"raft.queue_time_us":50826,"raft.run_cpu_time_us":2252,"raft.run_wall_time_us":250873,"spinlock_wait_cycles":151201792,"thread_start_us":232988,"threads_started":1,"wal-append.queue_time_us":504996}
I20260709 13:04:10.284507  8451 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:04:10.298641  8422 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:04:10.316464  8595 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.402310  1533 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:10.415644  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:10.391469  8594 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 13:04:10.354426  8390 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:10.325656  8563 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:10.425226  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 87135.4us
W20260709 13:04:10.426644  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 77389.5us
I20260709 13:04:10.419137  1533 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:10.436197  2296 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:04:10.433674  1681 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:10.439895  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3377) took 1063 ms. Trace:
I20260709 13:04:10.437884  1533 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:10.441692  1635 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
W20260709 13:04:10.426045  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 169170us
I20260709 13:04:10.447175  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:04:10.456780  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 99199.8us
I20260709 13:04:10.445039  1533 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:10.460734  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:10.440315  1561 rpcz_store.cc:276] 0709 13:04:09.376842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.376973 (+   131us) service_pool.cc:224] Handling call
0709 13:04:09.377278 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:09.377654 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.377692 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:10.110023 (+732331us) spinlock_profiling.cc:243] Waited 907 us on lock 0x7b2c00000658. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:04:10.110473 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:04:10.264239 (+153766us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:10.264301 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.264385 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.418399 (+154014us) raft_consensus.cc:1685] finished
0709 13:04:10.418421 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.439818 (+ 21397us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":338,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":32553,"spinlock_wait_cycles":2142848,"thread_start_us":118414,"threads_started":1,"wal-append.queue_time_us":194484}
I20260709 13:04:10.461403  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:10.426030  8522 raft_consensus.cc:2764] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
W20260709 13:04:10.464083  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 117166us
I20260709 13:04:10.460506  1533 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:10.436764  8552 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:10.464334  1683 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:10.437461  8508 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:10.466032  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:10.392031  8469 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.467278  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:10.443619  8390 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:10.436311  8392 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:04:10.556015  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:10.526767  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:10.588896  1712 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:04:10.558005  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:10.584410  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:10.458186  8309 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.586131  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:10.597229  1717 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:10.492376  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:10.587579  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 120325us
I20260709 13:04:10.547712  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:10.589092  1635 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:10.542500  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:10.599555  1718 raft_consensus.cc:2410] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:04:10.483546  1711 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:04:10.522848  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 89196.7us
I20260709 13:04:10.601082  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:10.598698  1715 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
W20260709 13:04:10.525475  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 54464.5us
W20260709 13:04:10.480981  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 122338us
I20260709 13:04:10.501845  1710 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:04:10.603308  1709 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
I20260709 13:04:10.609726  1718 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:04:10.602567  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:10.463503  8431 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:10.480826  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:10.532652  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 66645.3us
I20260709 13:04:10.443434  8414 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:10.605548  1715 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:04:10.607345  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 74397.4us
I20260709 13:04:10.469007  8608 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:10.616617  8246 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.617102  8465 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:10.639233  8246 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.624559  8245 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.479288  8552 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:10.643759  8246 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:10.644311  8552 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:10.626772  8414 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.621538  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:10.623279  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:10.546022  8600 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.236s	user 0.001s	sys 0.001s
I20260709 13:04:10.473402  8525 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:10.473136  8594 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:10.525189  8345 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:10.612476  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:10.631680  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:10.629664  8608 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:10.570410  8482 raft_consensus.cc:3055] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:10.568809  8173 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:10.687778  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:10.688508  1567 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:10.473992  8386 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.477747  8062 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:10.637683  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:10.519992  8036 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:10.511191  8550 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:10.514755  8453 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:10.489470  8546 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.146s
I20260709 13:04:10.696949  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:10.596592  8599 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:10.697988  8599 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.698478  8599 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:10.485035  8607 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:10.702239  8422 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:10.703106  8390 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.703562  8608 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.704962  8608 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:10.653959  1561 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:10.648313  8469 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:10.707496  8465 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:04:10.708745  8621 log.cc:927] Time spent T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:04:10.710084  1643 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:10.710206  1560 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:10.659508  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:10.652637  2299 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
W20260709 13:04:10.691035  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 78649.9us
I20260709 13:04:10.711774  1568 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:04:10.712368  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:10.713052  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:10.661906  1712 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:04:10.663748  1639 raft_consensus.cc:1275] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:10.589748  2296 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:10.512517  8563 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:10.730046  2300 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:10.708678  8465 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.648861  8414 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:10.629545  8097 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.525794  8491 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.114s
W20260709 13:04:10.751996  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 59750.8us
I20260709 13:04:10.673707  1635 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:10.686654  8482 raft_consensus.cc:740] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:04:10.725224  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 112146us
I20260709 13:04:10.518764  8432 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.733491  1639 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:04:10.627090  8472 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 13:04:10.661293  8525 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.671967  8345 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.673858  8594 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:10.746625  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:10.674305  8556 consensus_queue.cc:1048] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.093s
I20260709 13:04:10.750425  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:10.756438  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 44135us
I20260709 13:04:10.739477  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:10.755556  1609 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:10.787950   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:10.788019  1644 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
W20260709 13:04:10.783538  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51092.9us
I20260709 13:04:10.804248  1683 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:10.795373  1567 raft_consensus.cc:2410] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:04:10.797395  1641 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:04:10.811466  2268 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:10.813244  1638 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:04:10.816227  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:10.783798  8525 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:10.784322  8559 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:10.825516  8324 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:10.783501  8431 raft_consensus.cc:697] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:10.827129  8605 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:10.827529  8536 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:10.818598  8422 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 13:04:10.818519  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:10.829324  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 221610us
I20260709 13:04:10.814651  2269 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:10.787278  8345 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:10.886755  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 261091us
I20260709 13:04:10.887185  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:10.825873  1682 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:10.826789  1681 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:04:10.835991  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 163154us
I20260709 13:04:10.796401  8594 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:10.829123  8423 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:10.839279  1460 catalog_manager.cc:5697] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 10 to 18. New cstate: current_term: 18 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:10.891147  1460 catalog_manager.cc:5697] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 16 to 17, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
W20260709 13:04:10.892954  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 78332.5us
I20260709 13:04:10.887876  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:10.844990  8482 consensus_queue.cc:260] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.802609  8173 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:04:10.894456  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:10.894673  8264 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:10.895323  8264 raft_consensus.cc:697] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:10.896268  8264 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.891950  8599 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:10.855661  8386 consensus_queue.cc:1048] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:10.831143  8484 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:10.832062  8451 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:04:10.828087  8545 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:10.877209  8508 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.813851  8595 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:10.926090  1533 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:10.892890  1460 catalog_manager.cc:5697] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 16 to 21, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 21 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:10.756076  8097 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.852989  8432 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:10.897600  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 71764.3us
W20260709 13:04:10.899927  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.913900  8338 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:10.902064  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:10.942106  1567 raft_consensus.cc:2368] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 20. Current term is 21.
I20260709 13:04:10.905946  1561 raft_consensus.cc:1240] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:04:10.906697  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:10.943269  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:10.943460  1532 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:10.852468  8431 consensus_queue.cc:237] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:10.909147  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 82378.5us
I20260709 13:04:10.881031  8471 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:04:10.915746  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:10.916553  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:10.919513  1560 raft_consensus.cc:1240] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:10.921872  1563 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:10.909667  8324 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:10.937888  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:10.901122  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:10.935146  1682 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:10.930258  1460 catalog_manager.cc:5697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 16 to 19, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 19 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:10.940991  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 114026us
I20260709 13:04:10.917069  8309 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:10.952595  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:10.950151  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:10.953765  1460 catalog_manager.cc:5697] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 16 to 19, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 19 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:10.954605  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:10.955596  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:10.956324  1459 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60118 (request call id 233) took 28686 ms (client timeout 15000 ms). Trace:
I20260709 13:04:10.964213  8628 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:10.969924  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:10.926390  8595 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:10.918466  8584 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:04:10.992702  1460 catalog_manager.cc:5697] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:10.941763  8338 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.924059  8469 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:04:10.951706  8556 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:10.951483  8324 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:11.004220  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 69117.8us
I20260709 13:04:10.952198  8309 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.964996  8522 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:11.013105  2302 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22. Ops: []
W20260709 13:04:10.987854  8622 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:04:11.004786  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 54937.7us
W20260709 13:04:11.034740  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 90050.1us
I20260709 13:04:10.934026  8508 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:10.965155  8630 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:10.973469  8631 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:10.998409  8608 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:10.950677  8414 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:10.956737  1459 rpcz_store.cc:269] 0709 13:03:42.269973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.270209 (+   236us) service_pool.cc:224] Handling call
0709 13:04:10.956218 (+28686009us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:06.284909 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:06.415967 (+131058us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:06.425201 (+  9234us) write_op.cc:435] Acquired schema lock
0709 13:04:06.433648 (+  8447us) tablet.cc:662] Decoding operations
0709 13:04:06.488552 (+ 54904us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:06.488635 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:06.488670 (+    35us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:04:06.490193 (+  1523us) tablet.cc:701] Row locks acquired
0709 13:04:06.490241 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:04:06.490392 (+   151us) write_op.cc:270] Start()
0709 13:04:06.490547 (+   155us) write_op.cc:276] Timestamp: P: 1783602246490328 usec, L: 0
0709 13:04:06.490588 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:04:06.652886 (+162298us) log.cc:844] Serialized 7745 byte log entry
0709 13:04:07.035630 (+382744us) op_driver.cc:464] REPLICATION: finished
0709 13:04:07.294161 (+258531us) write_op.cc:301] APPLY: starting
0709 13:04:07.326233 (+ 32072us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:08.126884 (+800651us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:08.146808 (+ 19924us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:08.498950 (+352142us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:08.499379 (+   429us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 13:04:08.499456 (+    77us) write_op.cc:312] APPLY: finished
0709 13:04:08.576024 (+ 76568us) log.cc:844] Serialized 172 byte log entry
0709 13:04:08.918694 (+342670us) spinlock_profiling.cc:243] Waited 8.93 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd79eb970ac 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b
0709 13:04:09.518999 (+600305us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:09.542246 (+ 23247us) write_op.cc:454] Released schema lock
0709 13:04:09.569621 (+ 27375us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:09.751521 (+181900us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:09.751560 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1352391,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":192527,"apply.run_cpu_time_us":83577,"apply.run_wall_time_us":2468428,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83585,"num_ops":19,"prepare.queue_time_us":412571,"prepare.run_cpu_time_us":16858,"prepare.run_wall_time_us":562496,"raft.queue_time_us":66844,"raft.run_cpu_time_us":66118,"raft.run_wall_time_us":177471,"replication_time_us":529097,"spinlock_wait_cycles":20532096,"thread_start_us":1268139,"threads_started":5,"wal-append.queue_time_us":1088336,"wal-append.run_cpu_time_us":12248,"wal-append.run_wall_time_us":2353312}]]}
I20260709 13:04:10.988286  8595 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:10.973917  8628 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:11.037745  8550 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:04:11.200304  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 161830us
I20260709 13:04:11.006893  8338 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:11.039101  8567 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:04:10.994506  1460 catalog_manager.cc:5697] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:11.038260  8309 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:11.027278  8324 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:10.976388  8036 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:10.975556  8624 consensus_queue.cc:1059] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.239s
I20260709 13:04:11.313670  1569 raft_consensus.cc:2368] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22.
I20260709 13:04:11.315196  8550 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.317427  1460 catalog_manager.cc:5697] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:11.318459  1568 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:11.320629  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:11.337234  8628 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:11.340603  8638 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:04:11.341288  8624 raft_consensus.cc:3055] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:11.341753  8624 raft_consensus.cc:740] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:11.341845  8338 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:11.342648  8624 consensus_queue.cc:260] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:11.343909  8624 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:11.343566  1460 catalog_manager.cc:5697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 16 to 18, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:04:11.345001  1460 catalog_manager.cc:5697] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:11.344084  8414 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:11.174760  8522 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:11.349138  8522 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:11.356864  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 150525us
I20260709 13:04:11.346432  1460 catalog_manager.cc:5697] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 11 to 17, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 17 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:11.363277  1460 catalog_manager.cc:5697] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 13 to 17, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 17 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:11.381685  1460 catalog_manager.cc:5697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 17 to 20, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 20 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:11.384779  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 414730us
I20260709 13:04:11.352276  8550 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:11.390344  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:11.393357  8631 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:11.393919  8508 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:11.394443  1607 leader_election.cc:400] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 20. Current term is 21.
I20260709 13:04:11.394795  1607 leader_election.cc:403] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:04:11.394867  8097 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:11.395553  8036 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 20. Current term is 21.
I20260709 13:04:11.373629  8525 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:11.404096  8599 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:11.404898  1453 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3f22f7f5cede49bea2083e8905932fb7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6208c2fa49f643b5a50ab8f6d292fab2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:11.405525  1453 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 72329.9us
I20260709 13:04:11.405655  8525 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:11.413801  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 428958us
I20260709 13:04:11.396775  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:11.417738  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:11.423586  1460 catalog_manager.cc:5697] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:11.447067  8062 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
W20260709 13:04:11.451426  8594 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.551s	user 0.010s	sys 0.000s
I20260709 13:04:11.451977  8594 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:11.453053  1636 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 23.
I20260709 13:04:11.453701  8629 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.455821  8594 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:11.454394  8629 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:11.456754  1460 catalog_manager.cc:5697] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 11 to 19. New cstate: current_term: 19 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:11.462220  8599 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:11.462828  8599 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:11.464953  8599 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:11.473040  1460 catalog_manager.cc:5697] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 9 to 17. New cstate: current_term: 17 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:11.475334  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:11.477068  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:11.476217  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:11.478147  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:11.428197  8508 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:11.478940  8508 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:11.479512  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:11.481564  8508 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:11.427978  8643 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.479934  1460 catalog_manager.cc:5697] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 15 to 21, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 21 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:11.485953  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.487474  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:11.430344  8626 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:11.489226  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:11.485584  8643 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:11.491853  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 288441us
I20260709 13:04:11.493547  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:11.492061  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:11.486660  8511 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:11.496649  1711 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:11.499801  1567 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:11.502008  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:11.502816  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:11.503047  1636 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:11.503628  1644 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:11.506682  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:11.506870  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 110095us
I20260709 13:04:11.507043  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:11.507586  1635 raft_consensus.cc:2468] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:04:11.507666  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:11.509712  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:11.529353  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:11.486403  1460 catalog_manager.cc:5697] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 3 to 14, term changed from 18 to 23, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193), VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194), VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195), VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 23 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:11.530794  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:11.526224  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:11.532904  1460 catalog_manager.cc:5697] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 13 to 15, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 15 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:11.537271  1460 catalog_manager.cc:5697] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:11.524741  1563 pending_rounds.cc:85] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 9
I20260709 13:04:11.541404  1563 pending_rounds.cc:107] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:04:11.541800  1563 raft_consensus.cc:2889] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:11.542588  1460 catalog_manager.cc:5697] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 12 to 14, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:11.525748  1569 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:04:11.519766  8345 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.615s	user 0.001s	sys 0.010s
I20260709 13:04:11.568609  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:11.551978  2299 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:11.552403  1607 leader_election.cc:400] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22.
I20260709 13:04:11.564567  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:11.569665  1607 leader_election.cc:403] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:04:11.508816  8511 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:11.546422  1460 catalog_manager.cc:5697] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 15 to 19, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 19 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:11.583628  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 380565us
I20260709 13:04:11.591077  1460 catalog_manager.cc:5697] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:11.526695  8607 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:11.596558  8414 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:11.595156  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:11.571316  8608 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:11.603678  1711 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 22.
I20260709 13:04:11.603044  1636 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 22.
W20260709 13:04:11.605078  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 52788.5us
W20260709 13:04:11.655738  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 60677.4us
I20260709 13:04:11.607667  8435 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.522743  8324 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:11.583145  8621 log.cc:927] Time spent T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:04:11.579851  8644 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:11.549919  8594 consensus_queue.cc:1059] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:04:11.521670  8626 consensus_queue.cc:1059] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.075s
I20260709 13:04:11.669353  1564 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:04:11.672956  1713 raft_consensus.cc:2368] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 19. Current term is 20.
I20260709 13:04:11.673759  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:11.682353  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:11.675119  2299 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:04:11.585137  8097 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:11.594730  8522 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:11.564311  8599 consensus_queue.cc:1059] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:04:11.594053  1638 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:04:11.712900  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:11.570502  8345 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:11.600564  8629 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:11.638933  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:11.617517  8338 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:11.605881  8465 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:11.657704  1566 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:11.661204  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:11.593355  8245 consensus_queue.cc:1059] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:04:11.682402  1636 raft_consensus.cc:3060] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:04:11.677100  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:11.674796  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:11.759938  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 85193.7us
I20260709 13:04:11.676558  1643 raft_consensus.cc:3055] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:11.761639  1643 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
W20260709 13:04:11.729187  8659 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:04:11.612314  2300 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:04:11.667898  1640 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:11.676645  8652 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:11.722652  1460 catalog_manager.cc:5697] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 9 to 15, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 15 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:11.776317  1460 catalog_manager.cc:5697] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 16 to 19, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 19 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:11.778491  1564 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 23.
I20260709 13:04:11.640143  8643 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:11.740157  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:11.742111  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:11.764904  8599 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:11.671169  8552 raft_consensus.cc:3055] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:11.672614  8324 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:11.670547  8386 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:11.751868  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 90726.1us
I20260709 13:04:11.687965  1644 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 22.
W20260709 13:04:11.729322  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 90508.4us
I20260709 13:04:11.837918  1643 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:11.710865  8264 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:11.667294  8650 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22.
I20260709 13:04:11.799365  1460 catalog_manager.cc:5697] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:11.752230  8550 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:11.746171  8655 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:11.830955  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 63981.8us
I20260709 13:04:11.674324  8644 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:11.670740  8435 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:11.677453  8626 raft_consensus.cc:3055] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:11.763049  8616 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:11.848140  1714 raft_consensus.cc:2368] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 23. Current term is 24.
I20260709 13:04:11.743191  8522 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:11.728454  8097 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:11.840247  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:11.804239  1636 raft_consensus.cc:2468] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:11.868609  1460 catalog_manager.cc:5697] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:11.654356  8511 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:11.880941  1643 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:11.832331  8608 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:11.832841  8449 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:11.758704  8465 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:11.915128  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:11.816211  8245 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.053s
W20260709 13:04:11.891709  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 327020us
I20260709 13:04:11.855221  8552 raft_consensus.cc:740] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:04:11.907794  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 67601.2us
I20260709 13:04:11.911671  1716 raft_consensus.cc:2368] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19.
I20260709 13:04:11.817058  8652 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:11.891230  8624 consensus_meta.cc:287] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.000s
I20260709 13:04:11.911896  1711 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:11.922302  1533 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:11.923689  1712 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:04:11.928236  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:11.928970  2300 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 14.
I20260709 13:04:11.935957  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:11.934638  1533 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:11.923906  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:11.885819  8626 raft_consensus.cc:740] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:11.876103  8264 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:11.936359  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:11.939798  2268 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.885605  8422 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:04:11.893168  8305 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 13:04:11.907709  8645 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:11.886452  8615 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:04:11.920033  8655 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.917048  8599 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:11.953362  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:11.975647  1637 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
I20260709 13:04:11.949266  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:11.960188  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:11.924052  8511 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:11.968070  8626 consensus_queue.cc:260] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:11.936091  8595 consensus_meta.cc:287] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.664s	user 0.011s	sys 0.000s
I20260709 13:04:11.983327  1643 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:04:11.919921  8608 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:11.945210  8574 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:11.942248  8664 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:11.977968  1711 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:04:11.925045  8449 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:11.960116  8525 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:12.014439  8594 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:04:12.016115  1637 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:04:11.989459  1533 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:11.971503  8616 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 13:04:12.003263  8599 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 23 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:12.004642  8630 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.098436  1718 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
W20260709 13:04:12.013432  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 298533us
I20260709 13:04:12.022528  8626 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:12.022956  8529 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:04:12.074357  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.100780  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 111354us
W20260709 13:04:12.106678  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 160055us
I20260709 13:04:12.111495  1607 leader_election.cc:400] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 19. Current term is 20.
I20260709 13:04:12.120359  1607 leader_election.cc:400] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 23. Current term is 24.
I20260709 13:04:12.112828  8529 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.972285  8645 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:12.049578  8338 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:11.979375  8655 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.053079  8664 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 22 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:11.928277  8552 consensus_queue.cc:260] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.003080  8595 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.177764  1643 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:12.157228  8529 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:12.177490  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 225239us
I20260709 13:04:12.181905  8615 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.182533  8615 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.120437  8574 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 22 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:12.157378  8584 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:12.127012  8646 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:04:12.188822  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.206082  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.231980  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.232244  1638 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:12.230587  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 164722us
I20260709 13:04:12.170459  8525 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.230679  8552 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:12.247633  8663 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:04:12.247998  8386 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.248566  8386 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:12.249245  8665 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
I20260709 13:04:12.251196  8097 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:12.251909  8392 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:04:12.253647  8668 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:04:12.256139  8536 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:04:12.257071  8648 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.784s	user 0.004s	sys 0.013s
W20260709 13:04:12.257808  8680 log.cc:927] Time spent T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:04:12.258693  1565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2821) took 1345 ms. Trace:
I20260709 13:04:12.332587  1643 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:12.276427  1638 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:12.259121  1565 rpcz_store.cc:276] 0709 13:04:10.913549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.913657 (+   108us) service_pool.cc:224] Handling call
0709 13:04:10.923867 (+ 10210us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.195092 (+271225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.195121 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.195691 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.330107 (+134416us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:11.380421 (+ 50314us) log.cc:844] Serialized 14 byte log entry
0709 13:04:11.417057 (+ 36636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.417176 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:11.472482 (+ 55306us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b1800365a40. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:04:12.256490 (+784008us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:04:12.258483 (+  1993us) raft_consensus.cc:1685] finished
0709 13:04:12.258504 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.258640 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94032,"prepare.queue_time_us":48540,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":96125,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":19335,"spinlock_wait_cycles":59111424,"thread_start_us":241003,"threads_started":2,"wal-append.queue_time_us":236296}
I20260709 13:04:12.338708  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:12.343902  1718 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:12.115027  8630 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.234201  8584 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:12.267432  8661 log.cc:927] Time spent T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.342s	user 0.000s	sys 0.000s
W20260709 13:04:12.358615  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.260083  8654 log.cc:927] Time spent T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.420s	user 0.006s	sys 0.000s
I20260709 13:04:12.259176  8386 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:12.231637  8435 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:12.270551  8550 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:12.175233  8338 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.329612  8097 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.338232  8036 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:12.244289  8462 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:12.264586  8345 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:12.235661  8453 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.161s
I20260709 13:04:12.331509  8525 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:12.424564  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.392657  8545 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:12.456957  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 117034us
I20260709 13:04:12.421568  8522 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:12.460788  8630 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:12.462652  8309 consensus_meta.cc:287] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 1.137s	user 0.006s	sys 0.006s
I20260709 13:04:12.416028  8036 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:12.463230  8309 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:12.465075  8036 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:12.467145  8655 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:12.384336  8664 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:12.469506  8595 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:12.424242  8462 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.455726  8264 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:12.396015  8584 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:12.340452  8599 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:12.509052  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 54506us
I20260709 13:04:12.511574  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:04:12.512243  8678 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:04:12.544064  1451 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ca83cd5e36eb4e22b75011b87bfcd49d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3f64b76180a54492bb015e405caeacec is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:12.545907  1563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3396) took 1047 ms. Trace:
I20260709 13:04:12.546167  8462 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:12.546296  1563 rpcz_store.cc:276] 0709 13:04:11.498110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.498260 (+   150us) service_pool.cc:224] Handling call
0709 13:04:11.524519 (+ 26259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.542862 (+ 18343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.542936 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.559149 (+ 16213us) log.cc:844] Serialized 26 byte log entry
0709 13:04:12.099685 (+540536us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:12.099724 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.099803 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.545683 (+445880us) raft_consensus.cc:1685] finished
0709 13:04:12.545707 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.545844 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":489301,"threads_started":1,"wal-append.queue_time_us":664209}
I20260709 13:04:12.513882  8482 raft_consensus.cc:3060] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:12.518205  8449 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:12.520769  8656 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.688s	user 0.000s	sys 0.009s
I20260709 13:04:12.521446  8338 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:12.244846  8646 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:12.413467  8097 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.526207  2300 raft_consensus.cc:1240] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22. Ops: []
W20260709 13:04:12.531312  8675 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 13:04:12.509649  1682 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:12.539119  8608 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:12.518455  8529 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.561667  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:12.584175  1568 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:12.570298  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:12.576572  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:12.591104  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:12.481310  8309 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:12.542480  1718 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 15: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:12.581030  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:12.596791  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 87173.8us
I20260709 13:04:12.587594  8392 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:04:12.660054  8696 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:12.660681  8696 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:12.578940  8595 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:12.661639  8595 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.660069  8694 raft_consensus.cc:2804] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:12.605207  8646 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:12.607702  8097 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:12.616472  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 45836.8us
I20260709 13:04:12.669219  1609 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:12.617177  8685 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:12.640201  8691 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 13:04:12.641569  8675 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:12.644124  8529 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:12.579147  8264 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.663486  8694 raft_consensus.cc:697] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:12.649009  1563 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 15: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:12.654153  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:12.664665  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 434406us
I20260709 13:04:12.582073  8036 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.537458  8545 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.683281  1534 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:12.416675  8574 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:12.726526  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:12.729444  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:12.729892  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:12.729092  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:12.680737  8245 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:12.733006  1640 raft_consensus.cc:2393] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 15: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:12.667207  8696 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:12.738286  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 55083.1us
I20260709 13:04:12.741721  1534 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:12.740522  2298 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:12.740700  1568 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 24.
I20260709 13:04:12.744596  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:12.744776  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2747) took 1284 ms. Trace:
I20260709 13:04:12.745427  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:12.715278  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:12.683874  8595 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:12.744064  1609 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:12.745476  1716 rpcz_store.cc:276] 0709 13:04:11.472651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.479235 (+  6584us) service_pool.cc:224] Handling call
0709 13:04:12.744549 (+1265314us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b54000b0ed4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:12.744714 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":975926016}
I20260709 13:04:12.666194  8338 consensus_queue.cc:1059] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:04:12.759461  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:12.729167  8646 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:12.765062  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:12.768435  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:12.735039  8615 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:12.772203  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:12.776268  8563 raft_consensus.cc:3055] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:12.748111  8392 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:12.778748  8645 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:12.779245  8694 consensus_queue.cc:237] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:12.781366  8640 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:12.784885  8698 log.cc:927] Time spent T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.272s	user 0.003s	sys 0.000s
W20260709 13:04:12.787312  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 206957us
I20260709 13:04:12.829792  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:12.791630  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:12.830502  2296 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:12.830617  2298 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:12.728330  8264 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.778297  8559 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.725000  8036 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.741832  8245 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.814620  1681 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:12.769543  8595 raft_consensus.cc:2764] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 13:04:12.849645  8674 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.101s
I20260709 13:04:12.749575  8529 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:12.836764  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:12.855782  2303 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:12.837328  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:12.838778  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:12.856611  2296 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:12.844897  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:12.699796  8685 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.806663  8599 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.800956  8563 raft_consensus.cc:740] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:12.858487  1637 raft_consensus.cc:3060] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:04:12.807912  8392 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:12.866070  2298 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:04:12.871083  1641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2895) took 1915 ms. Trace:
I20260709 13:04:12.868024  2304 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:12.833796  8645 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:12.875857  2303 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:04:12.871596  1641 rpcz_store.cc:276] 0709 13:04:10.955210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.959314 (+  4104us) service_pool.cc:224] Handling call
0709 13:04:10.960821 (+  1507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.387447 (+426626us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.387481 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.398484 (+ 11003us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.539789 (+141305us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:11.539824 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.854947 (+1315123us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54002d8054. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:12.855003 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.856576 (+  1573us) raft_consensus.cc:1685] finished
0709 13:04:12.856600 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.871006 (+ 14406us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2315,"raft.queue_time_us":15437,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":39921,"spinlock_wait_cycles":553023360,"thread_start_us":326393,"threads_started":2,"wal-append.queue_time_us":222376}
W20260709 13:04:12.881199  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 66653.2us
I20260709 13:04:12.884548  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:12.884909  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.871186  8435 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:12.886994  8563 consensus_queue.cc:260] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:12.889623  8392 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:12.851469  8559 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:12.885228  1565 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:04:12.895628  8670 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.109s
I20260709 13:04:12.896750  1637 raft_consensus.cc:2468] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:04:12.898559  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:12.899385  1637 raft_consensus.cc:2410] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:04:12.893385  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:12.900586  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:12.901697  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:12.867528  8689 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:04:12.907307  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:12.873066  8599 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:12.908005  1637 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:04:12.908895  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:12.910295  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:12.911201  1641 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
I20260709 13:04:12.880030  8645 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:12.901031  8465 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:12.908017  8626 consensus_meta.cc:287] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.778s	user 0.005s	sys 0.005s
I20260709 13:04:12.908440  8689 consensus_queue.cc:1243] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:13.011768  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:12.910065  8563 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:13.011874  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:13.016355  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.017357  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:13.025503  8685 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.026335  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:12.943220  8624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.006s	sys 0.006s
W20260709 13:04:13.027149  8624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.006s	sys 0.007s
I20260709 13:04:13.027232  1637 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 20: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:13.028153  8599 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:13.028224  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:13.031899  1568 raft_consensus.cc:1240] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 22 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:13.011270  8595 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:13.014285  8545 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.041138  1564 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
W20260709 13:04:13.028368  8511 consensus_meta.cc:287] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 1.040s	user 0.009s	sys 0.002s
I20260709 13:04:13.048527  8645 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:13.019060  8642 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:13.059861  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:13.060722  1637 raft_consensus.cc:2410] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 13:04:13.031245  8709 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:04:13.058902  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:13.083321  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1268) took 2389 ms. Trace:
I20260709 13:04:13.033639  8559 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:13.061177  2302 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
W20260709 13:04:13.058138  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 240625us
I20260709 13:04:13.083874  1562 rpcz_store.cc:276] 0709 13:04:10.670483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.670667 (+   184us) service_pool.cc:224] Handling call
0709 13:04:10.671149 (+   482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.307044 (+635895us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.307090 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.490660 (+183570us) spinlock_profiling.cc:243] Waited 5.19 ms on lock 0x7b2c00000658. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:04:11.508516 (+ 17856us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.827292 (+318776us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:11.827336 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.252116 (+424780us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b48000030d8. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a6bcaaa1 00007fd7a6bc1b37 00007fd7a6bc19ca 00007fd7a6cb926f 00007fd7a6cb8f37 00007fd7a6ccd55c 00007fd7a6ccd508 00007fd7a6ccd488 00007fd7a6ccd443 00007fd7a6ccc6ec 00007fd7a5bf6a96
0709 13:04:13.059566 (+807450us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54002986d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:13.059624 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:13.059675 (+    51us) raft_consensus.cc:1685] finished
0709 13:04:13.059687 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.059903 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":244394,"raft.run_cpu_time_us":8908,"raft.run_wall_time_us":1036830,"spinlock_wait_cycles":696649600,"thread_start_us":610201,"threads_started":2,"wal-append.queue_time_us":422903}
I20260709 13:04:13.044173  8640 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:13.068202  8482 consensus_meta.cc:287] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.511s	user 0.006s	sys 0.005s
I20260709 13:04:13.092705  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:13.093211  1641 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 15.
I20260709 13:04:13.095132  1643 raft_consensus.cc:1275] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:04:13.091193  8584 consensus_meta.cc:287] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.000s
W20260709 13:04:13.093955  8703 log.cc:927] Time spent T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.215s	user 0.001s	sys 0.001s
I20260709 13:04:13.056931  8595 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:13.090057  8511 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:04:13.095114  8705 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:04:13.096596  8697 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.084570  8642 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.098826  8264 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:13.114078  1642 raft_consensus.cc:1240] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
W20260709 13:04:13.120218  8671 log.cc:927] Time spent T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 1.139s	user 0.015s	sys 0.000s
I20260709 13:04:13.109427  8511 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.129482  8595 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:13.130028  8511 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:13.092428  8640 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.101862  8584 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.108819  8717 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.114063  8697 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.133270  8717 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.135532  8717 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:13.136224  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.136327  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.137015  1709 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 20: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:13.137907  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.138470  1709 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:04:13.139212  1560 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:13.132639  8640 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.132675  8584 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.144562  1637 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:04:13.159279  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3310) took 1570 ms. Trace:
I20260709 13:04:13.130697  8642 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.159832  1635 rpcz_store.cc:276] 0709 13:04:11.589150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.589308 (+   158us) service_pool.cc:224] Handling call
0709 13:04:11.589777 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.597093 (+  7316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.597131 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.610252 (+ 13121us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.903873 (+293621us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:11.903915 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.903983 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:13.061537 (+1157554us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:04:13.152619 (+ 91082us) raft_consensus.cc:1685] finished
0709 13:04:13.152649 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.159207 (+  6558us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":14390,"spinlock_wait_cycles":2325376,"thread_start_us":252762,"threads_started":1,"wal-append.queue_time_us":349871}
I20260709 13:04:13.161720  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:13.162320  1635 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:13.162024  8644 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.178625  1635 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:13.180413  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.181092  1713 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:13.171286  8511 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:13.178901  8697 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.187135  8511 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.190313  8511 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:13.210322  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:13.212831  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.220090  1681 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:13.233634  1607 leader_election.cc:400] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19.
I20260709 13:04:13.234831  1607 leader_election.cc:403] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:13.210842  8697 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:13.211938  8650 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:13.216001  8482 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.231266  8435 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:13.242960  8599 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19.
I20260709 13:04:13.254727  8559 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:13.255069  8697 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:13.265659  8559 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:13.267241  8690 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.754s	user 0.003s	sys 0.001s
I20260709 13:04:13.267532  8697 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:13.267961  8559 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:13.269248  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 49271.6us
I20260709 13:04:13.269519  8684 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:13.270005  8684 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.270347  8684 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:13.271077  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 377557us
I20260709 13:04:13.271860  1683 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:13.272692  8699 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:13.273265  8699 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:13.274211  8465 consensus_queue.cc:1059] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:13.275398  8465 raft_consensus.cc:3055] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:13.275828  8465 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:13.276690  8465 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.278007  8465 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:13.280148  1683 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:13.281423  1683 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:13.282193  8345 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:13.285089  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:13.285629  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:13.289534  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:13.290115  1711 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36078 (request call id 2752) took 1613 ms (client timeout 2086 ms). Trace:
W20260709 13:04:13.290642  1711 rpcz_store.cc:269] 0709 13:04:11.676699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.676835 (+   136us) service_pool.cc:224] Handling call
0709 13:04:13.289931 (+1613096us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54000479d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:13.290044 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350262016}
I20260709 13:04:13.292624  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.293064  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:13.293578  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.255688  8482 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:13.295007  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.295270  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:13.296046  2305 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 22: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:13.293282  1709 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:04:13.292927  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:13.297621  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.298426  1717 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:04:13.301270  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.302840  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.304999  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.307020  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.288542  8552 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:13.318086  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.319190  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:13.321290  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:13.323742  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.322834  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:13.296720  8713 raft_consensus.cc:2804] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:13.296715  8482 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:13.255991  8650 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.328485  1607 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:13.274484  8699 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.339720  8713 raft_consensus.cc:697] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:13.340844  8652 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:13.340766  8713 consensus_queue.cc:237] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.342300  8036 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.345530  8650 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:13.346510  8688 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.935s	user 0.001s	sys 0.001s
I20260709 13:04:13.329419  8706 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:13.347666  8706 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:13.348042  8706 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:13.351102  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:13.351804  8647 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 1.887s	user 0.003s	sys 0.006s
I20260709 13:04:13.351130  8594 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:13.352602  8594 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.352972  8594 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:13.353154  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:13.353842  2305 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 22.
I20260709 13:04:13.354364  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:13.354559  8435 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:13.355679  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3307) took 2557 ms. Trace:
I20260709 13:04:13.324259  8599 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:13.325973  8697 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:13.356040  1639 rpcz_store.cc:276] 0709 13:04:10.798108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.799857 (+  1749us) service_pool.cc:224] Handling call
0709 13:04:10.800171 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:10.800499 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:10.800529 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:10.819839 (+ 19310us) log.cc:844] Serialized 26 byte log entry
0709 13:04:10.974061 (+154222us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b18004e5ce0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:04:11.307707 (+333646us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:11.307789 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.307904 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:13.331471 (+2023567us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:04:13.355426 (+ 23955us) raft_consensus.cc:1685] finished
0709 13:04:13.355447 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.355578 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17066,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":18937,"spinlock_wait_cycles":136610560,"thread_start_us":333426,"threads_started":1,"wal-append.queue_time_us":576873}
W20260709 13:04:13.355152  8662 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 1.450s	user 0.010s	sys 0.023s
I20260709 13:04:13.358263  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:13.373229  8660 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 1.413s	user 0.007s	sys 0.002s
I20260709 13:04:13.374271  8697 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 20 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:13.385449  8594 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.389357  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:13.389976  1712 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:04:13.390997  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1439) took 2461 ms. Trace:
I20260709 13:04:13.395925  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:13.397977  2296 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:13.375257  8697 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:13.357980  8482 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:13.391700  1715 rpcz_store.cc:276] 0709 13:04:10.909838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.909975 (+   137us) service_pool.cc:224] Handling call
0709 13:04:10.910352 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:10.911204 (+   852us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:10.911243 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:10.915683 (+  4440us) log.cc:844] Serialized 26 byte log entry
0709 13:04:11.408383 (+492700us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b480009ffd8. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a6bcaaa1 00007fd7a6bc1b37 00007fd7a6bc19ca 00007fd7a6cb926f 00007fd7a6cb8f37 00007fd7a6ccd55c 00007fd7a6ccd508 00007fd7a6ccd488 00007fd7a6ccd443 00007fd7a6ccc6ec 00007fd7a5bf6a96
0709 13:04:11.480257 (+ 71874us) spinlock_profiling.cc:243] Waited 917 us on lock 0x7b18001213c0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1 00007fd7a1c5ca0f
0709 13:04:11.711194 (+230937us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:11.711259 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.711362 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:13.354404 (+1643042us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:04:13.370708 (+ 16304us) raft_consensus.cc:1685] finished
0709 13:04:13.370732 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.370883 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142596,"raft.queue_time_us":944,"raft.run_cpu_time_us":10704,"raft.run_wall_time_us":494399,"spinlock_wait_cycles":19087872,"thread_start_us":230725,"threads_started":1,"wal-append.queue_time_us":944819}
I20260709 13:04:13.370808  8650 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:13.427260  2296 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 23.
I20260709 13:04:13.433229  1712 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 23.
I20260709 13:04:13.429839  1609 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:13.421006  8594 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.442022  8613 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:13.442239  1716 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:13.441557  8684 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:13.444005  1709 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:13.445007  8684 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:13.446650  8607 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:13.437876  8036 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:13.447293  8607 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.427820  8482 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:13.455600  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:13.457846  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.435029  8650 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.463752  1609 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:13.460485  8713 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:13.467599  8713 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.453774  8607 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.469303  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.470502  8713 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.472385  8482 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:13.472707  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:13.454152  8036 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 23 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:13.442628  8613 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.471601  8607 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:13.474069  8036 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.470490  8650 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.470876  8706 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:13.476884  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.482391  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 3316) took 1804 ms. Trace:
I20260709 13:04:13.482945  1638 rpcz_store.cc:276] 0709 13:04:11.678430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.712518 (+ 34088us) service_pool.cc:224] Handling call
0709 13:04:13.482179 (+1769661us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b54003250d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:13.482322 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191734656}
I20260709 13:04:13.478852  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:13.484879  8706 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:13.487630  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:13.488368  1710 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:13.488576  1609 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:13.489588  8697 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:13.490366  8673 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 1.512s	user 0.003s	sys 0.003s
I20260709 13:04:13.492882  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1945) took 1894 ms. Trace:
I20260709 13:04:13.493306  2301 rpcz_store.cc:276] 0709 13:04:11.598455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.598655 (+   200us) service_pool.cc:224] Handling call
0709 13:04:11.599115 (+   460us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:11.605157 (+  6042us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.605190 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:11.609093 (+  3903us) log.cc:844] Serialized 52 byte log entry
0709 13:04:11.903149 (+294056us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:11.903226 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.903370 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:13.489725 (+1586355us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:04:13.492511 (+  2786us) raft_consensus.cc:1685] finished
0709 13:04:13.492534 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.492813 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":160,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":8896,"spinlock_wait_cycles":7706880,"thread_start_us":219390,"threads_started":1,"wal-append.queue_time_us":308751}
I20260709 13:04:13.497488  1710 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:13.499029  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:13.499274  8613 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:13.500351  8697 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:13.500912  8697 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:13.501962  8697 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:13.536454  8683 log.cc:927] Time spent T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 1.144s	user 0.000s	sys 0.004s
I20260709 13:04:13.560420  8525 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:13.575764  8392 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:13.580226  8245 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.583266  8529 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:13.584050  8529 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.588145  8529 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.589088  8651 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:13.590065  8651 consensus_queue.cc:1243] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:13.590500  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.591449  1568 raft_consensus.cc:2410] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
W20260709 13:04:13.592751  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 477248us
I20260709 13:04:13.595628  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:13.597293  8431 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:13.597915  8431 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.600382  8431 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.603503  1533 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:13.603888  8645 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:13.604465  8645 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.606141  8574 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:13.606750  8574 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.609016  8574 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.609553  8574 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:13.610762  8642 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:13.611289  8642 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.611676  8642 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:13.615914  8645 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.618710  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:13.619509  1568 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:13.627416  8608 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.628255  8608 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.629485  1682 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:13.630448  8245 raft_consensus.cc:2764] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:04:13.633550  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:13.633957  8726 log.cc:927] Time spent T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.355s	user 0.002s	sys 0.000s
I20260709 13:04:13.634284  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:13.634900  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:13.635257  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:13.635377  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:13.636278  8608 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.636528  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:13.642506  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1643) took 1069 ms. Trace:
I20260709 13:04:13.642768  1641 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:04:13.642905  1643 rpcz_store.cc:276] 0709 13:04:12.572802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.574180 (+  1378us) service_pool.cc:224] Handling call
0709 13:04:12.574550 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:13.095961 (+521411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.642252 (+546291us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b54000d1454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:13.642432 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1195722624}
I20260709 13:04:13.644845  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.645018  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.647001  1568 raft_consensus.cc:2468] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:04:13.647485  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:13.648772  2303 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
W20260709 13:04:13.654219  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 50773.1us
I20260709 13:04:13.630769  1682 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:13.661393  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:13.661710  2268 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:33997 timed out after 1.541s (SENT) [suppressed 11 similar messages]
I20260709 13:04:13.656687  1533 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:13.590703  8651 raft_consensus.cc:3055] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:13.666445  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:13.674154  1638 raft_consensus.cc:2410] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 20 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:04:13.674551  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:13.676301  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.681389  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:13.682411  1635 raft_consensus.cc:3060] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:13.670956  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.687731  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.687950  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.688988  1710 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:13.690407  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:13.692287  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 61562.1us
I20260709 13:04:13.694295  1532 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:13.610047  8574 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.696146  1533 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:13.697738  8462 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:13.698000  8655 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:13.695314  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:13.698812  8645 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:13.699131  8309 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:13.699357  1638 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:13.699705  8309 raft_consensus.cc:697] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:13.699872  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:13.698333  8462 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.700841  8508 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:13.700662  8309 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.701365  8508 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.702492  8462 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.703254  8478 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:13.706647  1561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1283) took 1017 ms. Trace:
I20260709 13:04:13.657935  8062 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:13.706991  1561 rpcz_store.cc:276] 0709 13:04:12.688897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.689036 (+   139us) service_pool.cc:224] Handling call
0709 13:04:12.689409 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:12.700384 (+ 10975us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:04:12.742513 (+ 42129us) log.cc:844] Serialized 14 byte log entry
0709 13:04:13.273132 (+530619us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b18002cb900. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb4522 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:04:13.538904 (+265772us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b18002cb900. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb96e07 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a184123a 00007fd7a1caba2a
0709 13:04:13.704330 (+165426us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:13.704698 (+   368us) log.cc:844] Serialized 26 byte log entry
0709 13:04:13.704857 (+   159us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:13.704889 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.704953 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:13.706502 (+  1549us) raft_consensus.cc:1685] finished
0709 13:04:13.706515 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.706603 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":484035,"prepare.queue_time_us":41080,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":5421,"raft.queue_time_us":200351,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":9753,"spinlock_wait_cycles":52251264,"thread_start_us":13168,"threads_started":1,"wal-append.queue_time_us":809792}
I20260709 13:04:13.710645  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.715842  1644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60378 (request call id 2904) took 1768 ms. Trace:
W20260709 13:04:13.767390  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 105640us
W20260709 13:04:13.782806  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.716331  1644 rpcz_store.cc:276] 0709 13:04:11.947422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.947550 (+   128us) service_pool.cc:224] Handling call
0709 13:04:11.947867 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.038943 (+ 91076us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:12.038991 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:12.039080 (+    89us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:12.039128 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.715599 (+1676471us) spinlock_profiling.cc:243] Waited 90.8 ms on lock 0x7b5400155754. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:13.715652 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:13.715780 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208754944}
I20260709 13:04:13.699278  8645 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:13.664990  8431 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:13.689937  8599 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.799016  1640 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:13.707373  8062 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.799758  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.672236  8651 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:13.708850  8245 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:13.804646  8508 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.672626  8608 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:13.798141  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 103370us
I20260709 13:04:13.939759  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.940644  1565 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:13.942405  2268 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:13.942937  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.818548  1635 raft_consensus.cc:2410] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 13:04:13.812126  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 134046us
W20260709 13:04:13.833252  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 136226us
I20260709 13:04:13.823086  8697 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:13.836714  8649 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.840358  8706 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:13.844636  8642 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.970523  8706 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:13.971354  8642 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.849995  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.851346  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:13.972790  1709 raft_consensus.cc:2410] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:04:13.853401  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.857517  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.858333  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.856873  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.710475  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:13.863865  8574 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:13.872679  1562 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:13.883858  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.864053  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.900173  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.980417  1564 raft_consensus.cc:2393] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:13.903455  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.980500  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.981195  1567 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:13.904834  1532 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:13.909407  1710 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:13.916679  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.922427  2298 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:04:13.925412  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:13.932464  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:13.887647  8645 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:13.949370  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:13.993444  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.994472  1565 raft_consensus.cc:2410] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:04:13.995251  1711 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 13:04:13.995666  1711 pending_rounds.cc:85] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 12
I20260709 13:04:13.959775  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3312) took 2368 ms. Trace:
I20260709 13:04:13.995922  1711 pending_rounds.cc:107] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:04:13.996255  1711 raft_consensus.cc:2889] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:13.996060  1642 rpcz_store.cc:276] 0709 13:04:11.591636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.591772 (+   136us) service_pool.cc:224] Handling call
0709 13:04:11.592084 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:13.120557 (+1528473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.959235 (+838678us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54000d16d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:13.959585 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3497883008}
W20260709 13:04:13.995673  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 285115us
I20260709 13:04:13.998354  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:13.998610  1534 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:13.996120  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 91324.4us
W20260709 13:04:14.000679  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 50132.7us
I20260709 13:04:14.001461  1712 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:13.927047  8615 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.949481  8713 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:14.008698  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:14.001480  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:14.012198  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.910755  8431 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.024197  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:13.993176  8390 raft_consensus.cc:2764] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:04:14.033126  1567 raft_consensus.cc:1240] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:04:14.014564  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:13.918661  8599 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.060076  1532 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:14.051898  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:14.006389  8469 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:14.069181  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:14.006103  8642 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:14.083710  1636 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:04:14.023425  8614 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:14.096388  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:14.100147  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 85629.1us
I20260709 13:04:14.101207  1534 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:13.979821  8697 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.104346  8584 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:13.960578  8608 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.102815  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:14.120390  8697 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:14.017251  8615 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.001152  8649 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:14.022156  8536 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.077s
W20260709 13:04:14.139653  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 73861.9us
I20260709 13:04:14.141606  2269 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:14.029827  8713 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:14.104797  8607 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:14.098445  8594 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:04:14.165799  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 96678.7us
I20260709 13:04:14.109099  8469 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:14.123328  8508 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:14.145345  8704 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:04:14.145056  8584 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:14.151291  8062 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:14.045445  8651 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.160622  8614 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:14.155805  8684 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.015s
W20260709 13:04:14.166142  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 146934us
I20260709 13:04:14.172609  8508 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:14.188663  8062 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:14.186573  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 83810.6us
I20260709 13:04:14.174392  8706 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:14.200747  8431 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:14.201140  8615 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:14.202489  8584 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:14.204221  1453 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3f22f7f5cede49bea2083e8905932fb7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6208c2fa49f643b5a50ab8f6d292fab2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:14.171317  8594 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.204241  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:14.206676  8594 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:14.195725  8508 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:14.214624  8614 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:04:14.215565  8747 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:04:14.217134  8746 log.cc:927] Time spent T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 13:04:14.196978  1534 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:14.218364  8599 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:14.218979  8062 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:14.219364  8062 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:14.219717  8062 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:14.219812  2268 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:14.223732  8479 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:14.224215  8479 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.224570  8479 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:14.224700  8608 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:14.227396  8649 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:14.215306  8614 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.232599  1565 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 13:04:14.259475  1565 pending_rounds.cc:85] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Aborting all ops after (but not including) 9
I20260709 13:04:14.259757  1565 pending_rounds.cc:107] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:04:14.259986  1565 raft_consensus.cc:2889] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:14.260324  8614 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:14.235317  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:14.235961  8432 consensus_queue.cc:1059] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:14.262902  8432 raft_consensus.cc:3055] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:14.263301  8432 raft_consensus.cc:740] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:14.264175  8432 consensus_queue.cc:260] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.265439  8432 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:14.237222  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:14.240630  8750 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:04:14.228484  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:14.257830  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.223732  8713 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:14.270699  2268 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:14.277047  8614 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:14.277413  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:14.271183  8761 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:14.284552  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:14.285266  1560 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:14.285796  8599 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:14.286036  8649 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:14.286448  8599 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.286628  8649 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.288858  8599 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:14.289268  8649 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:14.217242  8508 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.286773  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:14.301741  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 142326us
I20260709 13:04:14.313355  8651 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:14.317364  8449 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:14.317545  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:14.318040  8449 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:14.271644  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:14.274165  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:14.274662  8414 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:14.322752  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:14.322885  8692 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.323655  8414 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:14.324476  1564 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:04:14.324571  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:14.323751  8692 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:14.325788  8545 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:04:14.325662  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 54096.3us
I20260709 13:04:14.326551  8545 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.327494  8692 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:14.328572  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 44363us
I20260709 13:04:14.337069  1568 raft_consensus.cc:1275] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:04:14.338074  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:14.339824  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:14.340093  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:14.343824  1561 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:04:14.345321  8642 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:14.339313  8605 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:14.346704  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:14.347173  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:14.347604  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.347671  2304 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 22: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:14.349443  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:14.349805  8449 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:14.338092  2269 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:14.351214  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:14.353092  8682 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:14.356153  2299 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:14.356400  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:14.335664  8574 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:14.360634  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:14.359301  1681 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:14.368367  2268 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:14.387434  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:14.390450  1709 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:14.393052  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:14.334985  8645 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
W20260709 13:04:14.396677  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.397465  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:14.397980  1714 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:04:14.400154  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:14.382524  8699 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:14.379477  1681 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:14.357649  2269 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:14.407739  1716 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
W20260709 13:04:14.417557  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 59972.2us
I20260709 13:04:14.421521  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:14.406376  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 46573.4us
I20260709 13:04:14.422755  8761 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:14.419346  8758 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:14.403184  8479 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.419783  8547 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:14.425763  8508 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:14.374305  8345 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:04:14.416532  8245 raft_consensus.cc:2804] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:14.427819  8245 raft_consensus.cc:697] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:14.430248  1568 raft_consensus.cc:1240] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:14.401515  8545 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:14.411661  8682 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:04:14.405344  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 431540us
I20260709 13:04:14.434479  8508 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:14.430194  8479 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:14.445637  8508 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:14.421645  2269 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:14.448459  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:14.442615  8682 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.463444  1532 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:14.470356  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:14.477835  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:14.479486  1534 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:14.468261  8414 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:04:14.482393  1534 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:14.483240  8062 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.467702  8245 consensus_queue.cc:237] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:14.483774  8655 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:14.485002  8655 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.485395  8062 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:14.485426  8655 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:14.487106  8529 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:14.487285  8605 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:14.488456  2297 raft_consensus.cc:2410] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:04:14.464175  8682 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:14.489570  1712 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:14.491544  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:14.494725  1713 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:14.498416  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:14.458741  8640 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:04:14.503489  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:14.503664  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 82039.9us
I20260709 13:04:14.489666  8511 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:14.510561  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:14.512866  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:14.514556  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:14.498360  8642 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:14.518838  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:14.520073  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:14.524318  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:14.525722  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:14.527119  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:14.528077  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:14.528860  1643 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 22.
I20260709 13:04:14.530071  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:14.531881  2269 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:14.522321  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:14.513933  8761 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:14.534771  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:14.562359  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:14.537619  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:14.514771  8036 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:14.519822  8762 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.545105  8435 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.550906  1635 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:04:14.551203  2305 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:04:14.518365  8525 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:04:14.555769  8508 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:14.561583  2296 raft_consensus.cc:2410] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:04:14.537503  8707 consensus_queue.cc:1048] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 13:04:14.603983  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:14.605021  1643 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:14.609781  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:14.610409  1712 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:14.593679  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 84954.6us
I20260709 13:04:14.612655  1682 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:14.651666  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:14.651698  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:14.653475  1638 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 22.
I20260709 13:04:14.564039  8036 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:14.666560  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:14.677186  1560 raft_consensus.cc:3060] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:14.567551  8435 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.607937  8689 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:14.666640  8545 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:04:14.686007  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.686268  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:14.566165  8511 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:14.566269  8645 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:14.690538  1607 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.013s	sys 0.013s
W20260709 13:04:14.705389  1607 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.013s	sys 0.013s
W20260709 13:04:14.705780  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 524864us
I20260709 13:04:14.707409  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:14.709640  1712 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:14.709833  1607 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:14.711688  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:14.564226  8762 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.716545  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:14.718204  1640 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:04:14.726016  1638 raft_consensus.cc:1275] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:14.689916  8545 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:14.729563  1637 raft_consensus.cc:1275] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:04:14.731632  1643 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:14.734764  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:14.734867  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:14.740793  1644 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:14.698180  8713 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:14.744553  1560 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:14.746335  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:14.748447  8545 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:14.747596  8649 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:14.735926  8763 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:14.753398  8599 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:14.844363  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:14.937491  1641 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:14.752986  8713 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:14.848062  8726 log.cc:927] Time spent T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:04:14.862625  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:04:14.852893  8770 log.cc:927] Time spent T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:04:14.882023  1533 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:14.931963  2297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1741) took 1279 ms. Trace:
I20260709 13:04:14.940671   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:14.941943  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 289522us
I20260709 13:04:14.948482  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:14.952827  2300 raft_consensus.cc:2368] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21.
I20260709 13:04:14.943677  2297 rpcz_store.cc:276] 0709 13:04:13.652162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.656695 (+  4533us) service_pool.cc:224] Handling call
0709 13:04:13.949211 (+292516us) spinlock_profiling.cc:243] Waited 6.88 ms on lock 0x7b5800084698. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7ca22 00007fd7a7a95c09 00007fd7a6befee0 00007fd7a6cb90ab 00007fd7a6cb9131 00007fd7a6c670a0 00007fd7a6c66984 00007fd7a01e16d4 00007fd7a01e1662
0709 13:04:14.873525 (+924314us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b5400516dd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:14.931875 (+ 58350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1151970432}
I20260709 13:04:14.956343  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:14.876782  8594 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.958014  2301 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:04:14.960134  8036 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:14.961014  8036 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.939270  8599 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:14.977835  8599 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:14.961686  8036 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:14.996593  2300 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:04:14.992091  8435 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:15.002437  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 256028us
I20260709 13:04:15.006670  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:15.010886  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:15.011675  2305 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:14.981155  8763 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:14.971294  8594 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:14.974318  8655 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:15.055624  8414 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:15.056100  8414 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:15.056927  8414 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:15.058729  1709 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:04:15.060223  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:15.061017  1713 raft_consensus.cc:2393] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 21: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:15.062287  1712 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:04:15.068383  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 543799us
W20260709 13:04:15.046998  8766 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 13:04:15.037293  8761 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:04:15.051532  8769 log.cc:927] Time spent T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 13:04:15.080374  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 73794.7us
W20260709 13:04:15.080945  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 71532.6us
W20260709 13:04:15.074859  8765 log.cc:927] Time spent T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:04:15.088416  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:15.080825  1682 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:15.070542  8629 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:15.087261  1569 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:15.092255  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:15.101495  1567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1300) took 1035 ms. Trace:
W20260709 13:04:15.091179  8771 log.cc:927] Time spent T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:04:15.102056  1567 rpcz_store.cc:276] 0709 13:04:14.065676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.066412 (+   736us) service_pool.cc:224] Handling call
0709 13:04:14.066754 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.067019 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:04:14.492945 (+425926us) log.cc:844] Serialized 14 byte log entry
0709 13:04:15.101112 (+608167us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:15.101179 (+    67us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:15.101236 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.101313 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:15.101434 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":331384,"prepare.queue_time_us":38131,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":7338,"thread_start_us":669377,"threads_started":2,"wal-append.queue_time_us":452058}
I20260709 13:04:15.104255  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:15.081719  8761 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:15.107195  8773 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:04:15.108117  8776 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:04:15.090981  8614 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:15.110984  8780 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:04:15.113241  8594 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:15.114578  8655 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:15.140868  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 258850us
I20260709 13:04:15.102550  8763 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:15.137526  8774 log.cc:927] Time spent T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:04:15.147101  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:15.156941  1709 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:15.139963  8748 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.985s	user 0.000s	sys 0.016s
I20260709 13:04:15.140246  8713 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:15.149574  8629 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:15.144255  8794 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:15.176234  1533 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:15.180401  8309 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:15.181025  8309 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:15.181560  8469 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:15.183475  1534 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:15.185930  8630 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:15.165500  8775 log.cc:927] Time spent T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:04:15.149304  8642 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:15.172859  2268 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:15.201538  1709 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
I20260709 13:04:15.146632  8614 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:15.197528  8778 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.125s	user 0.000s	sys 0.003s
W20260709 13:04:15.220188  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 47474.4us
I20260709 13:04:15.227993  8305 raft_consensus.cc:2804] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:15.234014  8036 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:15.237536  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 61468.2us
W20260709 13:04:15.312860  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 129401us
I20260709 13:04:15.241448  1533 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:15.327142  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 85716.8us
W20260709 13:04:15.313220  8792 log.cc:927] Time spent T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 13:04:15.340628  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 113147us
W20260709 13:04:15.339382  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 155134us
I20260709 13:04:15.219518  8642 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:15.337985  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 243302us
I20260709 13:04:15.348796  1716 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:04:15.349220  1716 pending_rounds.cc:85] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 11
I20260709 13:04:15.349462  1716 pending_rounds.cc:107] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:04:15.349725  1716 raft_consensus.cc:2889] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:15.352232  1533 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:15.314237  8309 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:15.339129  8664 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:15.355995  1532 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:15.353535  1534 leader_election.cc:400] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21.
I20260709 13:04:15.366559  1534 leader_election.cc:403] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:04:15.367645  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:15.363102  8684 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:15.369992  8803 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:15.370182  8684 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:15.370527  8803 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:15.372413  8684 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:15.372471  8803 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:15.360226  8745 log.cc:927] Time spent T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 1.178s	user 0.006s	sys 0.005s
W20260709 13:04:15.385507  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 304610us
I20260709 13:04:15.357450  8305 raft_consensus.cc:697] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:15.391497  8614 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:15.392014  8804 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21.
I20260709 13:04:15.394244  8762 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:15.390276  8794 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:15.394995  8794 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:15.396656  8794 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:15.397586  8761 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:15.368769  8584 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:15.383497  8345 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
W20260709 13:04:15.399868  8682 consensus_meta.cc:287] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.900s	user 0.010s	sys 0.002s
I20260709 13:04:15.400236  8599 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:15.408313  8642 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:15.409260  8599 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:15.409839  1635 raft_consensus.cc:1240] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:15.410420  1641 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:15.410818  1641 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:15.411567  1641 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:15.412652  1641 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:15.419569  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:15.419790  8584 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:15.421972  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:15.425030  8547 consensus_queue.cc:1048] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:04:15.423705  8684 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:15.418927  8689 consensus_queue.cc:1048] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:04:15.420890  8794 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.010s
I20260709 13:04:15.431160  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1752) took 1078 ms. Trace:
I20260709 13:04:15.431630  2296 rpcz_store.cc:276] 0709 13:04:14.352566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.356844 (+  4278us) service_pool.cc:224] Handling call
0709 13:04:15.430901 (+1074057us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54005d2ad4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:15.431080 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459590528}
W20260709 13:04:15.423105  8777 log.cc:927] Time spent T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.442s	user 0.003s	sys 0.002s
I20260709 13:04:15.401818  8545 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:15.435883  8584 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:15.437350  1718 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:15.415916  8787 consensus_queue.cc:1059] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:04:15.418051  8682 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:15.444787  8787 raft_consensus.cc:3055] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:15.445207  8787 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:15.446007  8787 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:15.446899  2299 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 23: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:15.446033  8682 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:15.448112  8449 consensus_queue.cc:1175] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5184c101be204393adf3ac8b31f25ad6" responder_term: 15 status { last_received { term: 15 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 15 index: 10 } }
I20260709 13:04:15.448339  8787 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:15.461171  8545 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:15.437570  8305 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:15.544680  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 163020us
W20260709 13:04:15.546545  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 159161us
I20260709 13:04:15.548666  1534 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:04:15.549894  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 197683us
W20260709 13:04:15.552352  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.555758  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.558202  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:15.563982  8478 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:04:15.568769  8547 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:15.569911  8547 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:15.570647  8547 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:15.571398  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 149327us
I20260709 13:04:15.573698  8431 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:15.579789  8584 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:15.579965  8478 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:15.580729  8584 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:15.581629  8584 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:15.587018  8478 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:15.588693  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:15.589301  8478 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:15.591965  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:15.592402  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:15.595502  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:15.596386  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:15.596690  1568 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
W20260709 13:04:15.599506  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51325.3us
I20260709 13:04:15.606312  8414 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:15.627553  8508 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:04:15.599535  8431 consensus_queue.cc:1243] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:04:15.643776  8628 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:15.644945  8628 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:15.653362  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 92924.3us
I20260709 13:04:15.672860  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:15.697193  1641 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 13. (term mismatch)
I20260709 13:04:15.698398  1641 pending_rounds.cc:85] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 12
I20260709 13:04:15.707018  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:15.699209  1641 pending_rounds.cc:107] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 13:04:15.709650  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:15.710067  1641 raft_consensus.cc:2889] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:15.711439  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:15.712481  1640 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:15.733871  8594 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:15.754204  1643 raft_consensus.cc:1275] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:04:15.767244  1643 pending_rounds.cc:85] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 10
I20260709 13:04:15.770560  1643 pending_rounds.cc:107] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:04:15.770843  1643 raft_consensus.cc:2889] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:15.779767  8763 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:15.780046  8789 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:15.780184  8790 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:15.783802  1642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1664) took 1182 ms. Trace:
I20260709 13:04:15.783335  8790 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:15.784298  1642 rpcz_store.cc:276] 0709 13:04:14.600844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.601892 (+  1048us) service_pool.cc:224] Handling call
0709 13:04:14.602226 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.602472 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:04:15.017623 (+415151us) log.cc:844] Serialized 14 byte log entry
0709 13:04:15.771173 (+753550us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:15.783043 (+ 11870us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:04:15.783191 (+   148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.783316 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:15.783573 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":490738,"prepare.queue_time_us":328400,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":48,"raft.queue_time_us":611356,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":13450,"thread_start_us":603655,"threads_started":3,"wal-append.queue_time_us":210760}
I20260709 13:04:15.787750  8790 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:15.795202  8594 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:15.796728  1683 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:15.800045  8479 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.802671  8682 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:15.803535  8682 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:15.811569  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 605123us
W20260709 13:04:15.818835  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 87917.9us
I20260709 13:04:15.829101  8649 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:15.856880  8649 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:15.859797  8793 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.860059  8649 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:15.773530  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:15.872419  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:15.800891  8479 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:15.886509  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 87587.2us
I20260709 13:04:15.837666  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:15.838066  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:15.829233  2299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53446 (request call id 1573) took 1319 ms (client timeout 1740 ms). Trace:
I20260709 13:04:15.892781  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:15.894837  8699 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:15.896543  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:15.897372  1682 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:15.843905  1681 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:04:15.893033  2299 rpcz_store.cc:269] 0709 13:04:14.510103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.510226 (+   123us) service_pool.cc:224] Handling call
0709 13:04:15.828824 (+1318598us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b5400708fd4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:15.829030 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2140669312}
I20260709 13:04:15.828378  8790 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:15.905382  8790 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:15.908180  8790 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:15.855237  8822 log.cc:927] Time spent T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:04:15.860615  8793 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:15.879747  8479 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:15.881439  8594 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:15.925487  8788 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.926298  8788 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:15.926564  8803 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:15.926455  8547 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:15.927659  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 83978.6us
I20260709 13:04:15.928321  1718 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:15.926864  1682 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:15.927973  8536 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:15.946823  8536 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:15.928908  8788 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:15.947325  8536 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:15.947679  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:15.949093  8699 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:15.949602  8699 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:15.955799  1640 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 23.
I20260709 13:04:15.958978  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:15.969559  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:15.970278  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:15.971431  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:15.972425  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:15.972646  1683 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:15.975965  8788 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:15.981863  8577 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:15.992327  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.997224  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 115569us
I20260709 13:04:16.005411  8577 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.012548  8508 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:16.012893  8790 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.013751  8508 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:16.014160  8508 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:16.017395  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:16.013764  8790 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.031487  8390 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:16.030156  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:16.033071  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 106272us
W20260709 13:04:16.018858  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 787907us
I20260709 13:04:16.036360  8704 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:16.037304  8704 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:16.037458  1453 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3f22f7f5cede49bea2083e8905932fb7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6208c2fa49f643b5a50ab8f6d292fab2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:16.039295  8831 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:16.040107  8831 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.041657  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:16.042284  8704 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:16.042410  8831 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:16.044406  8790 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:16.045756  8432 consensus_queue.cc:1059] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:04:16.046327  1533 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:16.046891  8432 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:16.048342  1638 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:16.047125  8305 raft_consensus.cc:3055] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:16.059322  8547 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:16.064215  8529 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:16.064800  8529 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:16.065251  8529 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:16.066213  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:16.067732  8833 log.cc:927] Time spent T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:04:16.056470  8432 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.058322  8305 raft_consensus.cc:740] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:16.070026  8432 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:16.072880  8431 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:16.050591  8640 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:04:16.083555  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 49031.2us
I20260709 13:04:16.085572  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:16.061734  8435 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:16.098340  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:16.095273  8245 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:04:16.094334  8682 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:16.121536  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:16.096181  8390 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.140857  8787 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:16.141009  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:16.141472  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:16.141848  8787 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.142544  2297 raft_consensus.cc:2410] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:04:16.145622  8793 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:16.146096  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:16.159083  8790 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:16.159554  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:16.160878  2304 pending_rounds.cc:85] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 11
I20260709 13:04:16.161250  2304 pending_rounds.cc:107] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:04:16.133942  8508 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:16.166950  8508 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:16.171113  2269 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:16.172102  8834 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:04:16.093039  8305 consensus_queue.cc:260] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.178500  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:16.183627  2301 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 22.
I20260709 13:04:16.178584  8479 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:16.187785  8479 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.193918  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:16.148608  8628 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:16.177496  8787 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:16.161506  2304 raft_consensus.cc:2889] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:16.135118  8431 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:04:16.147009  8414 consensus_meta.cc:287] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.520s	user 0.012s	sys 0.000s
I20260709 13:04:16.222218  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:16.226430  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:16.227205  1642 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:16.228746  8783 raft_consensus.cc:2804] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:04:16.218092  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 81707.6us
I20260709 13:04:16.174852  8536 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.238787  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:16.258699  1567 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:04:16.262427  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:16.260883  8536 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:16.216343  8594 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:16.268430  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 163884us
I20260709 13:04:16.273890  1638 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:04:16.262218  2269 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:16.272795  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:16.238036  8783 raft_consensus.cc:697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:16.261617  8790 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:16.307086  1639 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:16.308012  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 48221.1us
I20260709 13:04:16.309988  2297 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:04:16.262837  8699 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.288506  8577 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:16.346944  1713 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:16.347657  1642 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19. Ops: []
I20260709 13:04:16.351243  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:16.302802  8431 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.346603  8616 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:16.306705  8594 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.356492  8390 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:16.358474  2268 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:16.359012  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:16.364568  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:16.366076  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:16.371037  8826 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:16.371384  8828 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:16.371826  8783 consensus_queue.cc:237] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.382524  1568 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:04:16.389838  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:16.390089  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:16.390758  1640 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:04:16.338881  8790 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:16.332798  8651 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.061s
W20260709 13:04:16.396183  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 46288.9us
I20260709 13:04:16.385386  8616 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.403487  8616 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:16.401567  8390 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:16.409734  8828 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.412737  8431 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:16.413102  8828 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:16.395094  8826 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.417111  8826 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:16.418134  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 104567us
I20260709 13:04:16.419271  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:16.421010  8584 consensus_meta.cc:287] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.839s	user 0.014s	sys 0.000s
I20260709 13:04:16.426555  8616 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.431665  8616 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:16.432994  8305 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:16.445389  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:16.446197  1642 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:04:16.421584  8584 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.460604  1639 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:04:16.452500  8826 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:16.468803  8479 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:16.475025  1560 raft_consensus.cc:3060] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:16.482064  8756 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:16.486575  8826 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:04:16.518121  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:16.527937  8793 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:16.565836  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 51398us
W20260709 13:04:16.576697  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 154966us
I20260709 13:04:16.579843  1607 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:16.580910  8828 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:16.584491  8790 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:16.585210  8584 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:16.587283  1560 raft_consensus.cc:2468] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 23.
I20260709 13:04:16.532624  8756 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.599134  8756 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:16.594825  8529 consensus_meta.cc:287] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.002s
I20260709 13:04:16.608974  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:16.609818  2269 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:16.618677  8808 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:16.619210  8808 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:16.619616  8808 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:16.632328  1718 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:04:16.614192  8529 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:16.638006  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 58282.3us
I20260709 13:04:16.643736  1718 raft_consensus.cc:1240] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:16.645536  1534 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:16.646697  8584 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:16.653507  8529 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:16.659621  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:16.664636  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:16.665280  2303 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:16.673568  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:16.678814  8529 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:16.679219  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:16.679623  8529 consensus_queue.cc:1243] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:16.685276  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:16.693113  8826 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:16.695322  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:16.707398  8449 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.708102  8449 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.712610  8808 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:16.728842  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 62535.1us
I20260709 13:04:16.630453  8850 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:16.739637  1567 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:16.727229  8390 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:16.749632  8850 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:16.750548  2304 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:04:16.750717  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:16.751495  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:16.744953  8449 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:16.823330  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:16.823570  8449 consensus_queue.cc:1048] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:16.825449  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:16.818706  1682 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:16.826581  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:16.835656  1639 raft_consensus.cc:2368] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 22. Current term is 23.
I20260709 13:04:16.814836  2301 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:16.827975  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:16.752962  8594 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:16.754738  8390 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:16.819339  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:16.820024  8545 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.822007  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:16.843969  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:16.844741  1635 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:16.849877  1642 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 23.
I20260709 13:04:16.868614  2303 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:04:16.871511  1644 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:16.873062  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:16.872102  1644 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:16.844455  8808 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:16.887631  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:16.891692  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:16.896142  2297 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 13:04:16.896477  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:16.896814  1715 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:04:16.896960  8245 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:16.897768  8245 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.899350  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:16.900849  8245 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:16.883482  8536 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:16.903316  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:16.903137  8390 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.905058  8390 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:16.883985  8545 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.908430  8545 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:16.909585  1644 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:16.910988  1644 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:16.912026  8783 consensus_queue.cc:1059] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:16.913828  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:16.915485  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:16.915423  8640 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:16.918112  8808 raft_consensus.cc:3055] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:16.918628  8808 raft_consensus.cc:740] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:16.919713  8808 consensus_queue.cc:260] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.931005  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:16.932456  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:16.932967  1568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:16.933179  1564 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:04:16.933800  1568 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 22.
I20260709 13:04:16.861800  8616 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.941576  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:16.944006  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 125344us
W20260709 13:04:16.947409  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.948714  1609 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:16.952255  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:16.957755  8616 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:16.960161  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:16.960518  8616 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:16.963254  1644 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 10. (term mismatch)
I20260709 13:04:16.963642  1644 pending_rounds.cc:85] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 9
I20260709 13:04:16.963845  1644 pending_rounds.cc:107] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 20.10
I20260709 13:04:16.964056  1644 raft_consensus.cc:2889] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:16.964989  8594 raft_consensus.cc:2764] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:04:16.967160  8545 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:16.967833  8545 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:16.968910  8545 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:16.972607  8850 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:16.975962  1681 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:16.978262  8805 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:16.978806  8805 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.963160  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:16.961328  1682 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:16.981832  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:16.962790  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:16.987391  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 171500us
W20260709 13:04:16.988471  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:16.989107  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.979305  8805 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:16.987142  8478 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:17.002699  8305 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.552s	user 0.007s	sys 0.003s
I20260709 13:04:16.986117  8245 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
W20260709 13:04:17.030439  1607 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:17.020581  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 132990us
I20260709 13:04:17.025260  8699 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:17.012374  8640 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:17.005015  8431 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:04:17.034934  1607 connection.cc:256] RPC call timeout handler was delayed by -1.1485252589999959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21130000000000002s for a response
I20260709 13:04:17.030898  8854 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:17.055980  8584 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:17.059434  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:17.060788  8529 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:17.065685  8684 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:17.066300  8684 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:17.068276  8431 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:17.049784  8640 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:17.088405  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:17.089265  1636 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:17.090982  8831 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:17.091599  1458 catalog_manager.cc:5697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 10 to 13, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } }
I20260709 13:04:17.093364  1458 catalog_manager.cc:5697] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 14 to 15, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 15 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:17.095196  8640 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:17.095963  8684 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:17.097308  8684 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:17.104286  8431 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:17.104203  8805 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:17.094846  1458 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 14 to 15, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 15 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:17.107067  1533 leader_election.cc:400] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 22. Current term is 23.
I20260709 13:04:17.107527  1533 leader_election.cc:403] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:04:17.111212  1644 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:17.112170  8431 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for earlier term 22. Current term is 23.
I20260709 13:04:17.112931  1643 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:04:17.115928  1458 catalog_manager.cc:5697] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 17 to 18, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:17.120740  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 132353us
I20260709 13:04:17.122543  8805 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:17.130136  2269 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:17.102747  1636 raft_consensus.cc:2468] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:17.115562  1533 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:17.138024  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 271006us
I20260709 13:04:17.139153  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:17.140456  2268 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:17.149783  1683 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:17.153997  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:17.154458  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 167212us
I20260709 13:04:17.155547  1681 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.156215  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:17.157254  2303 raft_consensus.cc:2410] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:04:17.155254  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:17.167618  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:17.157198  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 195855us
I20260709 13:04:17.181505  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:17.150197  8759 raft_consensus.cc:2804] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:17.194594  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.195693  1568 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:04:17.196053  8699 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:17.196326  1681 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.196774  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.197228  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:17.197716  8699 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:17.198040  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.198447  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:17.199090  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.199570  1683 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.199764  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:17.193724  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.201005  8245 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.201499  8245 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:17.203644  8245 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:17.203779  1460 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45142 (request call id 353) took 8221 ms. Trace:
I20260709 13:04:17.204213  8245 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:17.197718  1564 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 16: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:17.198172  8699 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:17.196094  1636 raft_consensus.cc:1240] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 20. Current term is 21. Ops: [17.12-20.13]
I20260709 13:04:17.155766  8640 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:17.151373  8793 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:17.206853  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.206838  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.160971  8390 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:17.209358  2269 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:17.155254  8594 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:17.168025  8861 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:17.216038  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.217206  1560 raft_consensus.cc:2393] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 23: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:17.217718  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.155465  8682 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 13:04:17.219856  8574 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:04:17.221629  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.222146  1569 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:17.223202  1709 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:04:17.227432  8245 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.228819  8245 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:17.231449  8245 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:17.120126  1458 catalog_manager.cc:5697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 13 to 18, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:17.234133  8861 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:17.204142  1460 rpcz_store.cc:276] 0709 13:04:08.982240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.983770 (+  1530us) service_pool.cc:224] Handling call
0709 13:04:17.203533 (+8219763us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:12.775446 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:12.776082 (+   636us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:12.776135 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:04:12.776157 (+    22us) tablet.cc:662] Decoding operations
0709 13:04:12.778048 (+  1891us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:12.778146 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:12.778174 (+    28us) tablet.cc:685] Acquiring locks for 27 operations
0709 13:04:12.784066 (+  5892us) tablet.cc:701] Row locks acquired
0709 13:04:12.784090 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:04:12.784468 (+   378us) write_op.cc:270] Start()
0709 13:04:12.784596 (+   128us) write_op.cc:276] Timestamp: P: 1783602252784409 usec, L: 0
0709 13:04:12.784621 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:04:12.900091 (+115470us) log.cc:844] Serialized 10604 byte log entry
0709 13:04:13.175729 (+275638us) op_driver.cc:464] REPLICATION: finished
0709 13:04:13.716187 (+540458us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b540013f954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c9ce28 00007fd7a56d7581 00007fd7a56d6d58 00007fd7a56db9f1 00007fd7a56db9a9 00007fd7a56db939 00007fd7a56db901 00007fd7a56dab2d 000000000038f354 000000000038f199
0709 13:04:14.172482 (+456295us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b18003729a0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb43f3 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:04:14.182619 (+ 10137us) write_op.cc:301] APPLY: starting
0709 13:04:14.187583 (+  4964us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:14.351004 (+163421us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:14.351041 (+    37us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:15.472078 (+1121037us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:15.472759 (+   681us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=54,mrs_lookups=0
0709 13:04:15.472839 (+    80us) write_op.cc:312] APPLY: finished
0709 13:04:15.473556 (+   717us) log.cc:844] Serialized 236 byte log entry
0709 13:04:15.489113 (+ 15557us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:15.489372 (+   259us) write_op.cc:454] Released schema lock
0709 13:04:15.489443 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:15.489953 (+   510us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:15.489991 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":34460,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":61136,"apply.run_cpu_time_us":60104,"apply.run_wall_time_us":1311452,"cfile_cache_hit":110,"cfile_cache_hit_bytes":117041,"num_ops":27,"prepare.queue_time_us":388504,"prepare.run_cpu_time_us":78714,"prepare.run_wall_time_us":1022947,"raft.queue_time_us":57477,"raft.run_cpu_time_us":46144,"raft.run_wall_time_us":57028,"replication_time_us":390942,"spinlock_wait_cycles":9196800,"thread_start_us":553736,"threads_started":5,"wal-append.queue_time_us":228913,"wal-append.run_cpu_time_us":15086,"wal-append.run_wall_time_us":2075107}]]}
I20260709 13:04:17.234383  1458 catalog_manager.cc:5697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 14 to 16, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:04:17.206503  8478 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:17.209595  8699 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:17.210170  8616 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:04:17.241899  8682 raft_consensus.cc:2804] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:17.242501  8682 raft_consensus.cc:697] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:17.243583  8594 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:17.256631  1458 catalog_manager.cc:5697] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:04:17.254307  8478 consensus_queue.cc:1059] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:04:17.258227  8861 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:17.259064  8594 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:17.260280  1458 catalog_manager.cc:5697] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 6 to 18, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:17.262301  1458 catalog_manager.cc:5697] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 13 to 19, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:17.264037  1458 catalog_manager.cc:5697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 12 to 18, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:17.243433  8682 consensus_queue.cc:237] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:17.267329  8431 raft_consensus.cc:3055] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:17.267796  8431 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
W20260709 13:04:17.274482  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 74949.3us
I20260709 13:04:17.275471  8478 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 13:04:17.282331  8808 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:17.273849  8431 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:17.294842  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 88876.1us
W20260709 13:04:17.294262  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 85981.3us
W20260709 13:04:17.302693  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.304210  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.311621  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.311926  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.312875  1564 raft_consensus.cc:2393] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 22: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:17.313809  8431 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:17.313272  1458 catalog_manager.cc:5697] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:17.315167  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:17.311697  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.345875  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:17.347256  1453 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3f22f7f5cede49bea2083e8905932fb7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6208c2fa49f643b5a50ab8f6d292fab2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:17.350459  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:17.351505  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.353804  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.354105  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.354715  1564 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 13:04:17.351505  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:17.356670  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.340101  8640 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:17.359330  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 159203us
I20260709 13:04:17.362669  1563 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 22.
I20260709 13:04:17.392042  1563 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:17.356103  8759 raft_consensus.cc:697] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:04:17.397042  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51668.4us
W20260709 13:04:17.398119  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 45315.1us
I20260709 13:04:17.398864  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.347097  8699 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:17.424069  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:17.426286  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:17.354905  8529 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 13:04:17.427554  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.433460  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:17.433507  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.436344  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:17.436465  8759 consensus_queue.cc:237] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:17.439277  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:17.440431  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:17.447266  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:17.448884  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:17.451192  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:17.451556  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.452337  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.453461  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:17.453584  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:17.454069  1561 raft_consensus.cc:3060] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:17.454295  1636 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:17.455327  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:17.459276  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.343547  1458 catalog_manager.cc:5697] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:17.466245  1561 raft_consensus.cc:2468] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 23.
I20260709 13:04:17.462026  1458 catalog_manager.cc:5697] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 15 to 16, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 16 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:17.480882  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 68591.2us
I20260709 13:04:17.483453  1709 raft_consensus.cc:1240] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:04:17.484273  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:17.486178  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 78882.4us
I20260709 13:04:17.485344  1458 catalog_manager.cc:5697] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 10 to 11, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:17.487308  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:17.488037  8866 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.265s	user 0.002s	sys 0.002s
I20260709 13:04:17.489037  8828 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:17.489451  1607 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.489856  8828 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:17.491611  8828 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:17.426990  8868 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:17.447273  8863 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:17.479163  8826 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:17.502905  8873 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:04:17.502153  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:17.517665  1607 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:17.542100  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 60291.5us
I20260709 13:04:17.546331  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.516152  8787 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:17.484328  8879 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:04:17.557355  8863 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:17.569211  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 154875us
I20260709 13:04:17.606479  1714 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:17.607031  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:17.607648  1716 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
W20260709 13:04:17.607031  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.566149  8787 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:17.619791  1718 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:17.576193  1458 catalog_manager.cc:5697] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 13 to 18, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 18 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:17.557699  8868 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:17.631089  8881 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:17.631929  8878 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:17.632652  8878 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:17.638305  8850 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:17.678179  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 160505us
I20260709 13:04:17.679055  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.680060  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.680099  8892 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:17.690552  1682 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.692265  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.693740  8684 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:17.695624  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:17.691874  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:17.697072  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:17.688652  8888 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:04:17.701967  8759 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:17.705848  8887 raft_consensus.cc:3055] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:17.706491  8887 raft_consensus.cc:740] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:17.683105  8640 consensus_queue.cc:1059] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:04:17.709309  8863 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:17.715138  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.717502  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:17.719698  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:17.720248  8682 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:17.720752  1569 raft_consensus.cc:1275] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:04:17.720842  8682 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:17.721928  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.699761  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:17.726760  2298 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:04:17.728945  8877 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:17.729179  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:17.730650  8878 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:17.730736  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 50721.4us
I20260709 13:04:17.687223  8881 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.735626  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:17.743134  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 46118.3us
I20260709 13:04:17.748467  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:17.773175  1718 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:04:17.725140  8756 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:17.776146  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.777482  1607 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.724510  8868 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:17.735425  8808 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:04:17.805965  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 72473.5us
I20260709 13:04:17.800413  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:17.801558  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 56890.8us
I20260709 13:04:17.803435  8682 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:17.809257  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 45645us
I20260709 13:04:17.842779  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.808252  8808 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:17.808795  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 60629.2us
I20260709 13:04:17.780289  8756 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.810751  8616 consensus_queue.cc:1059] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.006s
I20260709 13:04:17.819039  8713 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:17.820413  8895 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:04:17.822031  8787 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:17.804608  8594 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:17.835228  1563 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:04:17.832775  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 256586us
I20260709 13:04:17.839576  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
W20260709 13:04:17.801380  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 57318.9us
I20260709 13:04:17.840574  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:17.863911  8872 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:17.848508  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.810957  8850 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:17.811401  8828 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:17.810014  8878 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.052s
I20260709 13:04:17.861469  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:17.903671  1566 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:17.768671  8881 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:17.863417  8616 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:17.869685  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:17.864782  1560 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:17.905280  1560 raft_consensus.cc:606] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:17.905694  1560 raft_consensus.cc:614] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:17.881321  8889 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:17.885077  8831 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:17.852415  8756 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:17.855293  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:17.921389  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 66148.9us
I20260709 13:04:17.922312  2268 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:17.853888  8808 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:17.854033  8682 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.915436  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:17.943472  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 143078us
I20260709 13:04:17.874011  8872 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:17.941030  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:17.846343  8894 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:17.944613  8894 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:17.946008  8826 consensus_queue.cc:1059] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:17.947027  8889 raft_consensus.cc:3055] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:17.947403  8889 raft_consensus.cc:740] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:17.947638  8894 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:17.949039  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.953897  1569 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:04:17.952032  1607 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.962016  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:17.868616  8887 consensus_queue.cc:260] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:17.968757  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 155201us
I20260709 13:04:18.056880  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:17.943697  8874 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:17.970151  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:17.942005  8759 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:18.057889  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 112648us
I20260709 13:04:18.010478  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:18.011560  1562 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
W20260709 13:04:18.056952  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 216786us
I20260709 13:04:18.075244  1562 raft_consensus.cc:606] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:18.075798  1562 raft_consensus.cc:614] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:17.956745  8894 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:17.999483  8898 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:04:18.101753  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:18.000777  8699 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:18.088775  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 136776us
I20260709 13:04:17.971107  8584 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:17.971186  8870 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:17.970711  8831 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:18.095278  8529 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.098296  1636 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:04:17.953310  8682 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:18.105584  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:18.065554  8872 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:18.109238  1566 raft_consensus.cc:2468] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:18.111805  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:17.971000  8431 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.132s
W20260709 13:04:18.103468  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 117349us
W20260709 13:04:18.123634  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 161705us
I20260709 13:04:18.119135  8831 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:18.125697  1532 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:18.138278  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:18.139783  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 78626.8us
I20260709 13:04:18.140060  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:18.141717  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:18.140568  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 62162.5us
I20260709 13:04:18.130075  8861 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:18.119100  8529 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:18.009865  8889 consensus_queue.cc:260] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:18.149643  1638 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:04:18.161078  8805 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:18.163166  8907 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:04:18.130332  8910 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:18.118398  8584 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:18.169466  1566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a4760a45328493e87523c2ee8292599"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
W20260709 13:04:18.182085  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 60275.9us
I20260709 13:04:18.182731  1607 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:18.183665  1566 raft_consensus.cc:606] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:18.178620  8872 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:18.136703  8831 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:18.188483  8826 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:04:18.195122  8881 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:18.196278  8906 consensus_queue.cc:1048] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:18.196645  8831 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:18.185573  8892 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:18.198293  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:18.198422  8892 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:18.199651  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:18.200023  8892 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:18.201601  1644 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 24.
I20260709 13:04:18.201889  8910 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:18.202154  8903 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:18.202703  8903 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:18.205242  1607 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:18.230950  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 106703us
W20260709 13:04:18.226557  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 85115us
I20260709 13:04:18.234881  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:18.235714  2298 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:04:18.196576  8584 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:18.238382  8756 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:18.240648  8756 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:18.244227  8682 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:18.246884  8529 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:18.248046  8887 raft_consensus.cc:3060] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:18.196465  8881 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:18.252543  8903 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:18.253211  8831 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:18.255380  8910 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:18.235082  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 50931.5us
W20260709 13:04:18.289063  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 55483.8us
W20260709 13:04:18.289193  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 54362.1us
I20260709 13:04:18.290899  1607 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:18.294934  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 106078us
I20260709 13:04:18.315214  1566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c129438e9d58493a93890208b25765cf"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:18.315886  1566 raft_consensus.cc:606] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:18.316064  1681 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:18.316256  1566 raft_consensus.cc:614] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:18.317308  1681 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:18.256011  8910 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:18.319648  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:18.325250  8787 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:18.326138  8787 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:18.329169  8787 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:18.335958  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 71810.5us
I20260709 13:04:18.322971  1560 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:18.341804  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:18.344862  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:18.345645  2300 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:18.353747  1561 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:04:18.306689  8894 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:18.320267  8903 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:18.380141  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 89262.8us
I20260709 13:04:18.291779  8828 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:18.366356  1534 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:18.382395  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 244049us
I20260709 13:04:18.384196  2268 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:18.387854  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:18.388895  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:18.345597  8756 raft_consensus.cc:2804] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:18.391589  1568 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:04:18.382751  8888 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:04:18.394357  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:18.364099  8869 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 13:04:18.367208  8808 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:18.448462  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 82175.3us
I20260709 13:04:18.422223  8831 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:18.451810  8920 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:18.454504  8913 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 13:04:18.454957  8920 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:18.457494  8920 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:18.458079  8529 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:18.458566  8529 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:18.424468  8756 raft_consensus.cc:697] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:18.458878  8902 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:18.459074  8529 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:18.460026  8924 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:18.459858  8756 consensus_queue.cc:237] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:18.460515  8924 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.460920  8924 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:18.461190  8478 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:18.461803  8478 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:18.424564  8684 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:04:18.386744  8828 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:18.436178  8870 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.507136  1568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:18.445135  8894 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:18.532320  2302 raft_consensus.cc:1240] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 18. Current term is 19. Ops: [18.16-18.16]
W20260709 13:04:18.531442  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 140260us
I20260709 13:04:18.530405  1568 raft_consensus.cc:606] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:18.532989  1568 raft_consensus.cc:614] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:18.534153  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 93394.4us
I20260709 13:04:18.538861  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:18.525151  8684 consensus_queue.cc:1243] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:04:18.569072  1609 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.507882  8917 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:04:18.576035  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:18.578029  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 189068us
I20260709 13:04:18.579697  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:18.580613  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 376126us
I20260709 13:04:18.576745  2298 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:04:18.588798  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:18.596290  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:18.590328  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:04:18.611960  8922 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:18.538524  8870 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:18.584745  8902 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:04:18.629946  8910 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:18.632584  8808 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:18.599289  8932 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:18.681335  1643 raft_consensus.cc:2393] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 20: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:18.605388  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:18.635793  8478 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:18.639894  8889 raft_consensus.cc:3060] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:18.640158  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:18.641331  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 55754.9us
I20260709 13:04:18.642053  8863 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:18.644471  8875 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:04:18.596869  8868 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.658293  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:18.671270  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 83808.9us
I20260709 13:04:18.718109  1609 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:18.633880  8926 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.125s	user 0.002s	sys 0.000s
I20260709 13:04:18.721465  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:18.702877  1642 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
I20260709 13:04:18.709551  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:18.722895  8935 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:18.712354  2297 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:04:18.723600  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:18.727195  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 129850us
I20260709 13:04:18.728278  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:18.731176  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:18.735793  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:18.749915  2300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e6dc6e12b7924255b756439991155e88"
dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:54964
I20260709 13:04:18.750942  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:18.753396  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:18.706629  8863 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:18.750581  2300 raft_consensus.cc:606] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:18.777423  8863 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:18.725003  8862 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:18.723644  8935 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:18.736059  8594 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:18.796221  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:18.782598  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 54360.5us
I20260709 13:04:18.780668  8862 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:18.788993  1563 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 18: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
W20260709 13:04:18.794159  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 135903us
I20260709 13:04:18.795732  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:18.805075  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 77210.5us
I20260709 13:04:18.807114  1712 raft_consensus.cc:2393] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 17: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:18.808552  1639 raft_consensus.cc:1275] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:04:18.795418  8910 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:18.818137  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:18.852730  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:18.796337  8935 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:18.734200  8868 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:18.856137  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 54140us
I20260709 13:04:18.856369  1716 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:18.798413  8808 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:18.855177  1638 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
W20260709 13:04:18.860399  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 46424.1us
I20260709 13:04:18.876755  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:18.799259  8828 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:18.859591  8682 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:18.715374  8932 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:18.904142  8925 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:18.904951   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:18.904873  8925 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:18.906985   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:18.907089  8919 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.907747  8919 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:18.909108  8640 raft_consensus.cc:2955] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 23 LEADER]: Committing config change with OpId 23.17: config changed from index 14 to 17, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:18.910269  8919 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:18.913142  8927 log.cc:927] Time spent T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:18.915141  8478 consensus_queue.cc:1059] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:04:18.916042  8478 raft_consensus.cc:3055] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:04:18.916390  8478 raft_consensus.cc:740] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:18.917126  8478 consensus_queue.cc:260] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:18.918285  8478 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:18.924983  8431 raft_consensus.cc:993] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:04:18.925522  8431 raft_consensus.cc:1081] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:04:18.929131  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:04:19.030119  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 122859us
W20260709 13:04:19.032083  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 102998us
W20260709 13:04:19.034761  8895 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 13:04:19.035056  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 107477us
W20260709 13:04:19.036286  8933 log.cc:927] Time spent T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 13:04:19.039150  8868 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:19.059868  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.141975  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 81647.5us
W20260709 13:04:19.161140  1451 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ca83cd5e36eb4e22b75011b87bfcd49d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3f64b76180a54492bb015e405caeacec is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:19.031608  8826 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:19.169154  1451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 57364.4us
I20260709 13:04:18.896539  8529 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:19.138535  8924 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.677s	user 0.006s	sys 0.005s
I20260709 13:04:18.929297  8682 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.046854  8935 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.191723  8925 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:19.194888  8935 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.197103  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:19.195741  8930 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.456s	user 0.002s	sys 0.004s
I20260709 13:04:19.078047  8919 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:04:19.215807  8936 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.320s	user 0.000s	sys 0.000s
I20260709 13:04:19.215946  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.172420  1638 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a4760a45328493e87523c2ee8292599"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60378
I20260709 13:04:19.216960  1638 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:19.218041  1638 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:19.187201  8887 consensus_meta.cc:287] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.939s	user 0.010s	sys 0.003s
I20260709 13:04:19.216935  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.232623  1562 raft_consensus.cc:1275] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:04:19.208201  8915 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:19.250094  8938 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.350s	user 0.000s	sys 0.000s
I20260709 13:04:19.218035  8870 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:19.209632  8919 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:19.183951  8826 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:19.262719  2301 consensus_meta.cc:287] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.589s	user 0.004s	sys 0.001s
I20260709 13:04:19.290640  2301 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 23.
W20260709 13:04:19.284225  1682 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.018s	sys 0.039s
W20260709 13:04:19.293977  1682 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.018s	sys 0.039s
W20260709 13:04:19.294358  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 695599us
W20260709 13:04:19.294613  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 257591us
I20260709 13:04:19.299928  8874 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:19.206068  8924 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:19.257979  8940 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:04:19.308382  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:19.199072  8914 consensus_queue.cc:1048] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.212s
I20260709 13:04:19.337064  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.345479  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:19.291704  8919 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:19.306438  8869 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.307435  8862 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:19.384599  8892 consensus_meta.cc:287] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.184s	user 0.013s	sys 0.000s
I20260709 13:04:19.399507  8892 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:19.400362  1635 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:19.407760  8939 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.383561  2303 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:04:19.382259  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:19.378803  8529 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.409659  1637 raft_consensus.cc:2393] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 20: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:19.379627  8875 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:19.365315  8887 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:19.411259  8887 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.411952  8869 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.412465  1609 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:19.414686  8431 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:19.415473  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:19.416134  8584 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:19.415445  8431 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.312203  8874 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.420300  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:19.421108  2303 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 16.
I20260709 13:04:19.378986  2305 raft_consensus.cc:2955] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Committing config change with OpId 23.17: config changed from index 14 to 17, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:19.424136  8935 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:19.424927  8935 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:19.427879  8887 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.275352  8915 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:19.435976  1566 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:04:19.408453  8939 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.451004  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:19.451880  1567 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:04:19.454310  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:19.475944  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:19.416851  8584 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:19.487018  8431 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.472991  8682 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:19.501976  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.504343  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.503705  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.426162  8935 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.508302  1717 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 23: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:19.508361  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.487728  8932 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.488353  8869 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.509055  1714 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:04:19.489609  8529 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:19.519613  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.520991  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:19.519359  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.532554  1712 raft_consensus.cc:2393] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 21: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:19.544559  8892 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:19.544943  8944 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.545485  8892 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.545656  8944 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.548523  8944 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.548909  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.549810  8874 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:19.524739  1711 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:04:19.550657  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.556296  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:19.565950  1683 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:19.591404  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:19.592983  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:19.593814  1567 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:04:19.596585  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:19.639694  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 73768.6us
W20260709 13:04:19.644639  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 299143us
I20260709 13:04:19.647641  2303 pending_rounds.cc:85] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting all ops after (but not including) 10
I20260709 13:04:19.648033  2303 pending_rounds.cc:107] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:04:19.648350  2303 raft_consensus.cc:2889] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:19.638860  8850 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:19.649742  8850 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:19.652091  8850 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:19.652678  8850 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:19.653267  8850 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:19.653748  8850 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.623118  8529 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.577955  8826 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.661269  8529 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:19.662534  8863 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.885s	user 0.012s	sys 0.000s
I20260709 13:04:19.664381  1635 raft_consensus.cc:2468] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
W20260709 13:04:19.665666  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 69033.9us
I20260709 13:04:19.666229  1534 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:19.667342  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 145679us
I20260709 13:04:19.677907  1569 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:04:19.679519  1535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.622881  8945 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:19.644407  8699 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:19.681448  1638 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.685001  1681 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:19.695990  1643 raft_consensus.cc:1240] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:19.684054  8944 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:19.680898  8863 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:19.700814  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:19.701292  8863 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 25 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:19.702154  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:19.702834  2298 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:04:19.703015  8868 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:19.703532  8868 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.703670  1561 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 25 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:19.704080  8868 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:19.705688  8859 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:19.707050  8931 raft_consensus.cc:993] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:04:19.707520  8931 raft_consensus.cc:1081] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:04:19.707918  8584 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:19.708570  8850 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.711102  8924 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:19.671733  8805 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:19.714989  8640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:47594 (request call id 7) took 2476 ms. Trace:
I20260709 13:04:19.719483  1638 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.691360  8939 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.719130  8640 rpcz_store.cc:276] 0709 13:04:17.213426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.213872 (+   446us) service_pool.cc:224] Handling call
0709 13:04:17.243780 (+ 29908us) log.cc:844] Serialized 501 byte log entry
0709 13:04:17.602714 (+358934us) spinlock_profiling.cc:243] Waited 8.8 ms on lock 0x7b18003c2080. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb4522 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:04:17.714274 (+111560us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b4c0017a700. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ff6da20 00007fd79ff7ea29 00007fd79ff7f13f 00007fd79ff432ab 00007fd79ff70f3c 00007fd79ff713fe 00007fd7a01fe50b 00007fd7a1bfd381 00007fd7a1bfb26b 00007fd7a1c01d53 00007fd7a1c01cd9
0709 13:04:19.689554 (+1975280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39530,"raft.queue_time_us":149862,"raft.run_cpu_time_us":4168,"raft.run_wall_time_us":278953,"spinlock_wait_cycles":382123648,"thread_start_us":222260,"threads_started":3,"wal-append.queue_time_us":97645,"wal-append.run_cpu_time_us":2548,"wal-append.run_wall_time_us":1081962}
I20260709 13:04:19.744242  8892 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.715963  8915 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:19.697234  8945 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.745589  1562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1344) took 1440 ms. Trace:
I20260709 13:04:19.746016  1562 rpcz_store.cc:276] 0709 13:04:18.305484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.305812 (+   328us) service_pool.cc:224] Handling call
0709 13:04:18.312657 (+  6845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.233115 (+920458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.745339 (+512224us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b54003e7454. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:19.745509 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2109060352}
I20260709 13:04:19.747522  8945 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:19.748282  8889 consensus_meta.cc:287] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 1.036s	user 0.013s	sys 0.000s
I20260709 13:04:19.721973  8805 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.750855  8805 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.756232  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:19.758709  8245 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:19.759335  8245 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.759680  8245 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:19.762579  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:19.767048  8889 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.767619  8889 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:19.769629  8889 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:19.803190  8889 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:19.804049  8889 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:19.812431  1683 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:38715 timed out after 1.708s (SENT) [suppressed 1 similar messages]
W20260709 13:04:19.820643  2297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1786) took 1349 ms (client timeout 1705 ms). Trace:
W20260709 13:04:19.821143  2297 rpcz_store.cc:269] 0709 13:04:18.471106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.471338 (+   232us) service_pool.cc:224] Handling call
0709 13:04:19.820470 (+1349132us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54006077d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:19.820588 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396997888}
W20260709 13:04:19.839205  2301 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 2044) took 1801 ms (client timeout 1708 ms). Trace:
W20260709 13:04:19.839641  2301 rpcz_store.cc:269] 0709 13:04:18.037447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.037637 (+   190us) service_pool.cc:224] Handling call
0709 13:04:19.839029 (+1801392us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b54001112d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:19.839136 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1292963712}
I20260709 13:04:19.849292  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fb294ac919b44e56960745f79e83dfc4 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:19.851816  8850 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:19.852644  8850 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.856091  8850 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.856901  8850 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:19.857635  8850 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:19.864158  1635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 3390) took 1290 ms. Trace:
I20260709 13:04:19.875595  1635 rpcz_store.cc:276] 0709 13:04:18.573419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.573639 (+   220us) service_pool.cc:224] Handling call
0709 13:04:19.863949 (+1290310us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b5400246b54. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:19.864066 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1843605376}
I20260709 13:04:19.862876  8903 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:19.877576  8903 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.881816  8903 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.883198  8919 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:19.884332  1565 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 21. Current term is 22. Ops: []
W20260709 13:04:19.889978  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 417079us
I20260709 13:04:19.895686  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:19.896596  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.040s	sys 0.085s
W20260709 13:04:19.897542  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.040s	sys 0.085s
I20260709 13:04:19.900116  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: true dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:19.900220  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:19.900876  2296 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:19.901119  1635 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:04:19.908447   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:19.909194  2296 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 22.
I20260709 13:04:19.911214  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:19.915009  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:19.915706  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:19.894443  1532 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:19.872539  8850 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:19.932050  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:19.920123  8889 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:19.932974  1640 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:19.933462  8919 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:19.934003  8960 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:19.936888  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:19.925938  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:19.926422  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:19.926803  8918 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:19.924489  1643 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e6dc6e12b7924255b756439991155e88"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:53222
W20260709 13:04:19.937202  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 291420us
I20260709 13:04:19.944646  1643 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:19.945101  1643 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:19.917460  8245 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:19.938630  1639 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:19.939028  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 342324us
I20260709 13:04:19.946668  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:19.948949  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.948946  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.935170  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:19.954596  8956 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.955984  8877 raft_consensus.cc:3060] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:04:19.956252  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 62557.4us
I20260709 13:04:19.968255  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:19.968688  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:19.971063  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.973901  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.975345  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.975683  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:19.976548  1640 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:19.977787  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:19.981716  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:19.955363  8956 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:19.992877  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:19.993880  8901 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:19.997632  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:19.998368  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:20.002698  8901 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.003201  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:20.005043  8901 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:19.994135  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:20.012406  2269 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:20.013260  8901 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:20.013700  8901 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:20.014109  8901 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:20.014770  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:20.015336  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:19.993947  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:20.016317  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:20.018884  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:20.020918  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:20.023350  8868 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.025506  8868 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:19.995831  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 50665.5us
I20260709 13:04:19.996004  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:20.029042  1711 raft_consensus.cc:2955] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Committing config change with OpId 23.17: config changed from index 14 to 17, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:20.030000  2268 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:20.030839  8953 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:20.030972  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:20.032068  8906 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:20.032475  8906 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:20.032807  8906 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:20.036440  8245 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.986545  8899 consensus_queue.cc:1059] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.051s
I20260709 13:04:19.985389  8713 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:20.038841  8713 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.959919  8947 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:20.039253  1562 raft_consensus.cc:3055] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:20.039762  1562 raft_consensus.cc:740] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:20.039603  8947 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.040653  1562 consensus_queue.cc:260] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.041254  8906 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.042037  1562 raft_consensus.cc:3060] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:20.039251  8713 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:19.934823  8960 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.043949  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:20.044991  1643 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.045560  8960 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:19.986867  8893 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:20.046643  8893 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.047572  8545 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:20.047964  8545 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:20.048996  8893 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:20.048764  8545 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:20.050009  8545 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:20.051111  8956 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:20.052379  1643 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:20.053004  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:20.054548  1640 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 25.
I20260709 13:04:20.054908  1569 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:04:20.056764  1639 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:19.998715  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:20.059640  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:19.996711  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:19.965198  8850 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:20.001685  8915 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:20.067688  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:20.068459  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:20.069892  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:20.042657  8947 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:20.057633  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:20.088454  1563 raft_consensus.cc:3060] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 19
W20260709 13:04:20.074831  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 78216.5us
W20260709 13:04:20.000016  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 187517us
I20260709 13:04:20.075203  2269 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:20.083545  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:20.085269  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:20.086192  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:20.093658  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:20.109491  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:20.109915  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:20.110239  2302 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:20.113281  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:20.113940  1569 raft_consensus.cc:3060] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:20.114712  1683 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:20.117225  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:20.118553  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:20.113133  8903 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:20.118939  8713 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:20.120724  8935 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:20.121176  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:20.121342  8935 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:20.119423  2304 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 19: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:20.122233  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:20.122880  1636 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 23.
W20260709 13:04:20.125329  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 50188.3us
I20260709 13:04:20.127951  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:20.128703  1561 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:20.132651  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:20.133615  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:20.134285  1710 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:19.997602  8905 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.111s
I20260709 13:04:20.133724  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:20.139717  1636 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:04:20.140887  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:20.143072  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:20.145166  1713 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:20.065462  8960 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:20.148864  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:20.143854  2297 raft_consensus.cc:2376] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 23. Re-sending same reply.
I20260709 13:04:20.153762  1642 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:20.076824  8901 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.170006  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:20.092020  8889 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:20.156066  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:20.182720  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:20.155330  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:04:20.179220  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 49580us
I20260709 13:04:20.179718  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:20.180509  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:20.181443  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:20.146373  8935 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:20.186488  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:20.196990  2301 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:20.197330  2301 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:20.121210  8713 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:20.199970  1609 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:20.195578  8959 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:20.195647  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:20.216142  1641 raft_consensus.cc:2410] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
W20260709 13:04:20.188477  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 54808.3us
I20260709 13:04:20.217036  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:20.149225  8931 raft_consensus.cc:2804] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:20.153155  8699 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:20.079229  8850 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:20.221215  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 65983.9us
I20260709 13:04:20.221985  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:20.090456  8915 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.238673  1533 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:20.382128  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 143702us
I20260709 13:04:20.208841  8826 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:20.384280  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:20.219592  8956 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:20.392071  1561 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:04:20.393271  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:20.215703  8616 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.173s
I20260709 13:04:20.230018  8903 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:20.262521  2299 raft_consensus.cc:2393] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 23: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:20.405248  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:20.184453  8960 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.406536  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:20.247644  8889 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:20.240367  8906 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:20.264540  8931 raft_consensus.cc:697] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
W20260709 13:04:20.420236  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 203182us
I20260709 13:04:20.384138  8478 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:20.423738  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:20.432641  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:20.442543  1607 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:20.443845  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:20.444059  8915 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:20.444727  8892 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:20.445384  8892 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:20.447906  8959 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:20.448438  8959 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.449986  8861 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:20.450407  8861 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:20.451457  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:20.435470  1641 raft_consensus.cc:1275] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:04:20.458266  1641 pending_rounds.cc:85] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 12
I20260709 13:04:20.458623  1641 pending_rounds.cc:107] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:04:20.458935  1641 raft_consensus.cc:2889] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:20.435938  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.439606  8917 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:20.453823  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:20.407200  8918 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:04:20.434092  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 234071us
I20260709 13:04:20.476691  1609 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:20.478317  1609 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:20.415191  8574 raft_consensus.cc:2764] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:04:20.405253  8899 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:20.497295  1609 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:20.499140  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:20.440629  1681 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:04:20.526355  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 119842us
I20260709 13:04:20.528064  1562 raft_consensus.cc:2468] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 22.
I20260709 13:04:20.463893  8892 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:20.487854  8889 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:20.415338  8826 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.450461  8959 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:20.542734  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 102096us
I20260709 13:04:20.538885  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:20.463452  8903 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:20.534394  8478 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:04:20.534533  8946 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
W20260709 13:04:20.567833  8969 log.cc:927] Time spent T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:04:20.545960  1682 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:20.553062  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:20.572806  8699 raft_consensus.cc:2804] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:20.573504  8699 raft_consensus.cc:697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:20.554265  8787 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:04:20.574976  8973 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:04:20.575335  8787 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:20.576694  8862 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:20.577296  2302 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 22.
I20260709 13:04:20.577728  8787 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:20.564380  8977 raft_consensus.cc:2804] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:20.560374  8861 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:20.522959  8906 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:20.579779  8861 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:20.579564  8906 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.580515  2301 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.581785  8906 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:20.581959  2301 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:20.583356  8877 consensus_meta.cc:287] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.627s	user 0.009s	sys 0.001s
I20260709 13:04:20.583817  8877 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:20.591143  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:20.574555  8699 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:20.596699  8960 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:20.597973  8850 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:20.599345  8850 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:20.544519  8975 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:20.603606  8975 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.606184  8975 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:20.606863  8931 consensus_queue.cc:237] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:20.544812  8879 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
I20260709 13:04:20.560889  8756 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:20.609016  8756 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:20.610026  8756 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:20.493075  8918 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:20.591296  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:20.493803  8917 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:20.559628  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:20.524039  8899 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.524506  8902 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:20.617383  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:20.651966  1569 consensus_meta.cc:287] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.538s	user 0.007s	sys 0.003s
I20260709 13:04:20.656327  1569 raft_consensus.cc:2468] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
W20260709 13:04:20.657724  1563 consensus_meta.cc:287] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.002s
I20260709 13:04:20.649507  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:20.650017  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:20.650319  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:20.614490  1710 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:04:20.659848  1563 raft_consensus.cc:2468] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:04:20.655568  1709 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 19.
W20260709 13:04:20.625612  8545 consensus_meta.cc:287] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.004s
I20260709 13:04:20.694118  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:20.695266  1532 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:20.716863  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:20.718079  1710 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:20.620251  8901 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:20.643702  8977 raft_consensus.cc:697] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 22 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:20.623672  8917 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:20.632862  8899 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:20.663143  8640 raft_consensus.cc:2764] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:04:20.628050  8984 consensus_queue.cc:1048] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.100s
I20260709 13:04:20.618613  8918 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:20.741864  2303 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.007s	sys 0.016s
I20260709 13:04:20.746815  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:04:20.747594  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 250270us
I20260709 13:04:20.718502  8919 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:04:20.752219  2303 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.007s	sys 0.016s
I20260709 13:04:20.622831  8935 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:20.615041  8903 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.735082  8964 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:20.738526  8862 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:20.801741  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:20.719655  8925 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.015s	sys 0.053s
I20260709 13:04:20.731899  2268 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:20.833781  2301 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:04:20.745002  8901 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:20.941915  1639 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:04:20.667286  8902 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:20.911686  8946 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 13:04:20.755223  8919 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:04:20.936250  8925 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.015s	sys 0.053s
W20260709 13:04:20.953256  1683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.025s	sys 0.036s
W20260709 13:04:20.953644  1683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.025s	sys 0.036s
W20260709 13:04:20.965093  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 382378us
I20260709 13:04:20.966552  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:20.954043  8919 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:20.935173  8964 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.956013  8478 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 25 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:20.947460  8901 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:20.998653  1683 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
W20260709 13:04:20.937101  8870 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.009s	sys 0.017s
I20260709 13:04:20.940693  8862 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.001111  8861 consensus_queue.cc:1048] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:21.001386  8478 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:20.999462  8536 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:21.003078  8713 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:21.003429  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:21.004106  8935 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:21.004822  1607 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:04:21.005558  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 250405us
I20260709 13:04:20.969964  8984 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:21.009263  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:21.027493  1533 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:21.009958  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:20.977365  8953 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:21.035650  2302 raft_consensus.cc:1240] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:21.015672  8870 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.010s	sys 0.018s
I20260709 13:04:21.013742  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:21.037307  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:04:21.012658  1638 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.015s	sys 0.006s
I20260709 13:04:21.043429  1638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53222 (request call id 1738) took 1115 ms. Trace:
I20260709 13:04:21.051018  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:21.054673  8862 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.043901  1638 rpcz_store.cc:276] 0709 13:04:19.927759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.927883 (+   124us) service_pool.cc:224] Handling call
0709 13:04:19.928193 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.928433 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:04:19.987370 (+ 58937us) log.cc:844] Serialized 14 byte log entry
0709 13:04:20.441068 (+453698us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b480015da58. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a6bcaaa1 00007fd7a6bc1b37 00007fd7a6bc19ca 00007fd7a6cb926f 00007fd7a6cb8f37 00007fd7a6ccd55c 00007fd7a6ccd508 00007fd7a6ccd488 00007fd7a6ccd443 00007fd7a6ccc6ec 00007fd7a5bf6a96
0709 13:04:20.497551 (+ 56483us) log.cc:844] Serialized 14 byte log entry
0709 13:04:21.043051 (+545500us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.043111 (+    60us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:21.043170 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.043258 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.043369 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":67677,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":76,"raft.queue_time_us":142607,"raft.run_cpu_time_us":2407,"raft.run_wall_time_us":58881,"spinlock_wait_cycles":5526400,"thread_start_us":1113007,"threads_started":4,"wal-append.queue_time_us":406634}
I20260709 13:04:20.988454  8977 consensus_queue.cc:237] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.049048  8935 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:21.065644  8935 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:21.066869  8935 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.068924  1533 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:21.070212  8862 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:21.071619  8903 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.072305  8902 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.072693  8536 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:21.073550  8536 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.044077  8984 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.034749  8915 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:21.086611  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:21.043429  8960 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:21.044780  8951 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:21.043556  8991 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:21.054687  8901 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:21.094043  8981 log.cc:927] Time spent T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 13:04:21.132143  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 79315.8us
W20260709 13:04:21.135711  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 403687us
I20260709 13:04:21.155696  8713 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:21.159617  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:21.161986  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:21.164180  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:21.166371  8984 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:21.142215  8951 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:21.168018  8951 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.171650  8964 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.134493  8991 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.174461  8991 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.177100  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:21.179998  2302 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 19: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:21.182298  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:21.180176  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:21.185845  8756 consensus_queue.cc:1059] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.004s
I20260709 13:04:21.189289  8956 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:21.189862  8956 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:21.194574  8536 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.195554  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:21.190485  8713 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.198829  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:21.200698  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 67316.5us
I20260709 13:04:21.210501  2302 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:21.228924  8994 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:21.235388  1609 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.237883  8903 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:21.238688  8903 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:21.239805  8987 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:21.240365  8987 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:21.239977  8919 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.241410  8987 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:21.252874  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 72734.7us
W20260709 13:04:21.256681  1453 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3f22f7f5cede49bea2083e8905932fb7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6208c2fa49f643b5a50ab8f6d292fab2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:21.258404  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 61418.2us
I20260709 13:04:21.265236  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:21.267781  8919 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.269460  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:21.259091  8987 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:21.271646  8956 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.273913  8956 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.279778  8826 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.283632  8956 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:21.286584  8956 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.287426  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:21.289053  8956 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.290839  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:21.291272  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:21.312146  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:21.312354  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:21.313781  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:21.314354  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:21.315336  1716 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 22.
I20260709 13:04:21.316237  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:21.316669  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:21.291270  8961 log.cc:927] Time spent T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.762s	user 0.003s	sys 0.001s
I20260709 13:04:21.317420  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:21.319352  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 2061) took 1672 ms. Trace:
I20260709 13:04:21.319782  2303 rpcz_store.cc:276] 0709 13:04:19.646990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.647174 (+   184us) service_pool.cc:224] Handling call
0709 13:04:19.647453 (+   279us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:19.695429 (+ 47976us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.695478 (+    49us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:19.716938 (+ 21460us) log.cc:844] Serialized 78 byte log entry
0709 13:04:20.191738 (+474800us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:20.756019 (+564281us) log.cc:844] Serialized 14 byte log entry
0709 13:04:20.915258 (+159239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.915362 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:21.290660 (+375298us) spinlock_profiling.cc:243] Waited 6.23 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:04:21.318996 (+ 28336us) raft_consensus.cc:1685] finished
0709 13:04:21.319104 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.319266 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":158883,"prepare.queue_time_us":242402,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":189730,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":14010,"spinlock_wait_cycles":14322432,"thread_start_us":957803,"threads_started":2,"wal-append.queue_time_us":677751}
I20260709 13:04:21.323032  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:21.329245  8872 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:21.333158  8872 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.335675  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.337321  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:21.337766  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:21.341243  8991 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.349469  8991 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.342093  8478 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:21.336103  8872 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:21.361181  8991 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:21.362375  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 45769.8us
I20260709 13:04:21.363346  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.365796  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:21.380617  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 46260.2us
I20260709 13:04:21.383754  1534 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:21.384558  9012 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:04:21.384984  8956 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:21.391089  8987 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:21.393062  8956 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:21.395856  8993 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.325s	user 0.004s	sys 0.000s
I20260709 13:04:21.399138  8478 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:21.400365  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:21.401127  1716 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:21.401262  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:21.401866  1714 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:04:21.404596  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:21.407855  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:21.407855  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:21.382157  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:21.444130  1712 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 LEADER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:21.442418  8872 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.419054  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:21.471406  8872 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.411481  8991 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:21.425981  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:21.473021  1683 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.394502  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.483760  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:21.473245  8947 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:21.484454  8992 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:21.485107  8947 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.488776  8947 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.491880  8862 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:21.492903  2268 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:21.493628  8919 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:21.494024  8919 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.494364  8919 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:21.495106  8980 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:21.502767  8919 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:21.505136  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 93990.2us
I20260709 13:04:21.506187  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:21.506834  1710 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 20
W20260709 13:04:21.507457  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 143044us
W20260709 13:04:21.508246  9008 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 13:04:21.511466  8979 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:21.512072  8979 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.514276  8979 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.517177  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:21.516101  1609 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.522663  8987 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:21.525959  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:21.527433  9007 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.299s	user 0.002s	sys 0.000s
I20260709 13:04:21.529350  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60390 (request call id 3399) took 1410 ms. Trace:
I20260709 13:04:21.529898  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:21.530555  1638 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:04:21.531620  1643 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:21.529757  1639 rpcz_store.cc:276] 0709 13:04:20.119275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.140565 (+ 21290us) service_pool.cc:224] Handling call
0709 13:04:21.529142 (+1388577us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b5400247554. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:21.529295 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1839459712}
I20260709 13:04:21.539286  8902 raft_consensus.cc:3055] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:21.539746  8902 raft_consensus.cc:740] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:21.540593  8902 consensus_queue.cc:260] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.541960  8902 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:21.548095  8865 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:21.553356  8877 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:21.553944  8877 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.556058  8877 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.556635  1639 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:21.556895  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:21.558475  8245 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.559020  8245 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:21.562127  9003 log.cc:927] Time spent T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.335s	user 0.000s	sys 0.001s
W20260709 13:04:21.563546  9004 log.cc:927] Time spent T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.320s	user 0.002s	sys 0.000s
W20260709 13:04:21.564913  8999 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.339s	user 0.000s	sys 0.002s
I20260709 13:04:21.564829  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:21.565627  1562 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:21.567042  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:21.567797  1567 raft_consensus.cc:2368] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19.
I20260709 13:04:21.572567  8979 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:21.573304  8979 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:21.576071  8915 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:21.576702  8915 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.577381  1681 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.578696  1681 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:21.579643  8918 raft_consensus.cc:2804] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:21.475170  8872 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:21.441236  1716 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:21.580566  8918 raft_consensus.cc:697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:21.447475  8911 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.479020  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:21.581429  8911 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.581462  8918 consensus_queue.cc:237] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:21.482236  9006 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.251s	user 0.000s	sys 0.002s
I20260709 13:04:21.584677  8911 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:21.585067  1681 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:21.585546  1532 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:21.586292  8899 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:21.586777  8899 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:21.587749  8899 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.579703  8245 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:21.571838   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:21.589011  8928 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:21.590387  1681 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:21.593225  1681 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.596491  8245 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:21.597380  8245 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:21.599190  1560 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:21.606235  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:21.607061  1563 raft_consensus.cc:2368] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20.
W20260709 13:04:21.616282  2296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53464 (request call id 1794) took 1437 ms (client timeout 1844 ms). Trace:
W20260709 13:04:21.616756  2296 rpcz_store.cc:269] 0709 13:04:20.179310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.180278 (+   968us) service_pool.cc:224] Handling call
0709 13:04:21.616104 (+1435826us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54006414d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:21.616226 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3285684480}
I20260709 13:04:21.610277  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:21.618134  1643 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 25.
I20260709 13:04:21.619235  8918 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:21.619766  8918 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.622695  8918 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.581001  8915 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.584161  8805 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:21.623965  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:21.629370  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:21.606313  8979 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:21.632858  1712 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:21.633383  8805 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:21.634196  8805 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.636824  8805 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.637984  8987 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:21.638952  8987 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:21.641566  8987 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:21.644500  1710 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:21.645711  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:21.646245  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:21.647336  1643 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:04:21.648849  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:21.651722  1563 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:04:21.655469  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:21.655447  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 260866us
I20260709 13:04:21.691872  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.694980  1534 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.662348  1639 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:21.663347  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:21.667258  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:21.698657  1635 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:04:21.654891  9010 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 13:04:21.672652  2301 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:04:21.675930  1562 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:04:21.676107  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:21.676894  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:21.681137  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:21.675833  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:21.686080  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:21.417007  8976 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:21.689729  1641 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:21.672282  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:21.710790  1565 raft_consensus.cc:2393] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 19: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:21.726409  1532 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:21.728770  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 47699.5us
I20260709 13:04:21.742166  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.771714  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:21.696926  8947 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:21.731638  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.782330  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:21.786309  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:21.787066  2269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.031s	sys 0.043s
I20260709 13:04:21.725508  8976 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:21.793423  1569 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20. Ops: []
W20260709 13:04:21.787945  2269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.032s	sys 0.043s
W20260709 13:04:21.789299  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 57702.4us
I20260709 13:04:21.713007  8987 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:21.725132  8991 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:21.725215  8892 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:21.797052  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.769550  8713 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:21.768956  8826 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:04:21.797820  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 591666us
W20260709 13:04:21.800431  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 74058.3us
I20260709 13:04:21.803928  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:21.807193  1607 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.725404  8979 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.069s
I20260709 13:04:21.837082  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:21.837949  1563 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 20: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:21.838578  1534 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.839751  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.835932  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:21.841459  1569 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 17: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:21.792344  8245 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:21.799461  8976 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:21.842096  2269 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:21.800058  8960 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:21.801877  8877 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:21.864063  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:21.804757  8990 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:21.869747  8956 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:21.904036  1564 raft_consensus.cc:1275] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:04:21.855821  8899 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:04:21.804844  8868 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:21.799633  8919 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:21.798864  8887 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 13:04:21.856680  8245 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:21.884464  8983 consensus_queue.cc:1059] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:04:21.885224  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:21.887447  9010 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:21.897850  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:21.898173  1566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:21.837167  8892 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:21.903210  2302 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:04:21.804965  8987 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.834553  8862 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:21.810104  8713 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:21.913302  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 106127us
I20260709 13:04:21.945220  1607 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:21.791621  8536 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:04:21.884035  1681 leader_election.cc:400] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20.
I20260709 13:04:21.903364  8935 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:21.923832  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:21.937189  1563 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 23.
I20260709 13:04:21.949013  1681 leader_election.cc:403] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:21.844458  8826 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:21.954289  1532 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:21.939903  8245 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:21.988221  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 104280us
I20260709 13:04:21.904752  8960 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:21.994560  2269 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:21.947873  8713 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:04:21.999817  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 76046.1us
I20260709 13:04:22.000799  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:21.947675  8862 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:21.964624  8979 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:21.991256  8449 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20.
I20260709 13:04:22.008921  9011 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:22.041251  8912 consensus_queue.cc:1059] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:22.042063  8912 consensus_queue.cc:1243] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer 3f64b76180a54492bb015e405caeacec log is divergent from this leader: its last log entry 20.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:04:22.000169  9028 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:22.046965  8892 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:22.012116  8862 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:22.056392  8960 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:22.064718  8911 raft_consensus.cc:3055] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:22.065244  8911 raft_consensus.cc:740] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:22.081372  8987 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:22.083387  8826 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:22.084455  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:22.006978  8924 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:22.090338  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 250516us
I20260709 13:04:22.091214  1534 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:22.096892  1561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "408b4ec829c54266b41334aac8245405"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:22.020378  9001 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:04:22.066217  8911 consensus_queue.cc:260] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:22.102028  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:22.148049  8245 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:22.149847  1716 raft_consensus.cc:1240] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:04:22.151243  8245 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:22.152323  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:22.139838  8862 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:22.156052  1569 raft_consensus.cc:1240] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 20. Current term is 21. Ops: []
W20260709 13:04:22.186496  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 233361us
I20260709 13:04:22.188279  1607 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:22.189702  1607 leader_election.cc:400] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19.
I20260709 13:04:22.190171  9032 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:22.190789  9032 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:22.191326  9032 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:22.193276  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 202892us
I20260709 13:04:22.194674  2302 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:04:22.193334  1607 leader_election.cc:403] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:04:22.162659  8245 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:22.202232  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:22.179769  8545 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.205269  8862 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:22.201840  8960 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547) with higher term. Message: Invalid argument: T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 18. Current term is 19.
I20260709 13:04:22.162369  8976 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:22.198704  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 176597us
I20260709 13:04:22.203155  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:22.214414  8918 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:22.218869  8951 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:22.228638  8991 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:22.229998  1561 raft_consensus.cc:606] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:22.230396  1561 raft_consensus.cc:614] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:22.234382  1643 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:22.237792  1635 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:04:22.238967  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 147706us
I20260709 13:04:22.239125  8947 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:04:22.243113  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 140219us
I20260709 13:04:22.231022  9028 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:22.246425  9028 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:22.247447  8478 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.248008  8478 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:22.249996  8478 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:22.233816  8826 consensus_queue.cc:1048] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.017s
I20260709 13:04:22.221771  8545 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:22.259526  2302 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:22.267411  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:22.268105  1644 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
I20260709 13:04:22.272758  1560 raft_consensus.cc:1275] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:04:22.275565  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:22.274729  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 259310us
I20260709 13:04:22.276456  1560 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 22.
I20260709 13:04:22.254516  8545 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:22.292309  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:22.291046  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:22.296236  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:22.303113  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 67036.4us
I20260709 13:04:22.303949  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:22.304447  1569 raft_consensus.cc:1275] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:04:22.210736  8245 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:22.316143  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:22.314472  1683 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:22.318075  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:22.312052  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.319113  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:22.306011  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:22.231892  9034 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:22.320590  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:22.323509  1560 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 22: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:22.324955  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:22.325594  1715 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 22.
I20260709 13:04:22.332556  8990 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:22.341780  8976 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:22.320791  8449 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.027s
I20260709 13:04:22.346239  8245 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:22.266862  8942 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:04:22.311466  8960 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:22.330729  8984 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:22.333938  1713 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:04:22.355394  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:22.354305  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:22.357910  1635 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:04:22.361107  1713 raft_consensus.cc:1240] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:22.317760  9009 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:22.313225  8987 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:22.374023  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:22.374396  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:22.375504  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:22.376138  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:22.378209  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:22.388116  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:22.311697  9011 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 13:04:22.325291  9036 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:22.410952  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:22.413625  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:22.357625  8984 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.455677  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:22.423938  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:22.384768  8859 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:04:22.326514  8713 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:22.473074  1534 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:22.475639  1534 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:22.477249  1534 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:22.458081  8984 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:22.483175  8640 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:22.495227  8536 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:22.495774  8536 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.496196  8536 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:22.481392  8478 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:22.459379  8245 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:04:22.351820  9034 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:22.495401  8942 consensus_queue.cc:1048] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:22.396425  8953 consensus_queue.cc:1059] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.084s
I20260709 13:04:22.515027  8931 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:22.486667  8862 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:22.516247  8953 raft_consensus.cc:3055] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:04:22.516654  8953 raft_consensus.cc:740] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:22.517526  8953 consensus_queue.cc:260] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:22.519157  9037 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:22.519613  9037 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:22.520434  9037 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:22.522095  9037 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:22.522909  8911 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:22.535077  9034 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:22.498466  8713 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 26 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:22.537406  2302 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:04:22.540050  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:22.541514  8987 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:22.542934  9035 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:22.585866  1564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:22.586764  1564 raft_consensus.cc:606] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:22.590344  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 275847us
I20260709 13:04:22.585501  8953 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:22.589373  8976 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:22.569792  8536 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:22.610677  8976 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:22.615003  8976 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:22.588220  9032 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:22.716801  9032 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:22.719159  8984 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:22.604651  9049 consensus_queue.cc:1048] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.091s
I20260709 13:04:22.725625  1564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6711207c14044518a57e7c8914ac0091"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:22.726389  1564 raft_consensus.cc:606] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:22.726747  1564 raft_consensus.cc:614] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:22.732657  1635 raft_consensus.cc:1240] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:22.733916  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:22.734455  8536 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:22.734575  1563 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:22.735297  2304 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:04:22.737265  8870 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.741609  1562 raft_consensus.cc:1275] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:22.741442  1561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:22.753309  1561 raft_consensus.cc:606] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Received request to transfer leadership to TS 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:22.742049  9002 consensus_queue.cc:1059] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:04:22.742745  9033 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:22.743453  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:22.745734  9036 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:22.771278  2301 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 20.
I20260709 13:04:22.777480  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:22.777719  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:22.778333  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:22.778376  1717 raft_consensus.cc:2368] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 22. Current term is 23.
I20260709 13:04:22.778674  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:22.778373  1715 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:22.785666  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:22.786374  1717 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:04:22.781924  8536 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:22.804474  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:04:22.808758  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 349173us
I20260709 13:04:22.767978  8870 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:22.811695  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:22.814391  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:22.823673  1561 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:22.814839  1563 raft_consensus.cc:2468] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 24.
I20260709 13:04:22.816030  1567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:22.820458  1564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47"
dest_uuid: "3f64b76180a54492bb015e405caeacec"
mode: GRACEFUL
new_leader_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:57836
I20260709 13:04:22.790522  8984 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:22.832049  2268 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:22.835662  1561 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 23.
I20260709 13:04:22.788411  8935 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:22.844519  2269 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.831418  1564 raft_consensus.cc:606] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Received request to transfer leadership to TS 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:22.848795   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:22.852723  8870 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:22.806684  8536 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:22.861148  8536 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:22.870671  9009 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:22.881985  8990 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.882597  8990 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:22.899271  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:22.900885  8651 raft_consensus.cc:3055] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:22.901376  8651 raft_consensus.cc:740] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:22.906629  1717 raft_consensus.cc:1240] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 25. Current term is 26. Ops: []
I20260709 13:04:22.908416  1711 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:04:22.929857  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33280 (request call id 1541) took 1039 ms. Trace:
I20260709 13:04:22.930326  1716 rpcz_store.cc:276] 0709 13:04:21.890962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.891128 (+   166us) service_pool.cc:224] Handling call
0709 13:04:21.891526 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.150335 (+258809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.929597 (+779262us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b54003dc2d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:22.929791 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":593239808}
I20260709 13:04:22.955577  8868 raft_consensus.cc:3055] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:04:22.956276  8868 raft_consensus.cc:740] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:22.957482  8868 consensus_queue.cc:260] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:22.967456  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:22.972216  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:22.791338  8952 consensus_queue.cc:1048] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.057s
I20260709 13:04:22.958626  8868 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:22.902338  8651 consensus_queue.cc:260] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:22.978183  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 49525.1us
I20260709 13:04:22.989915  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:23.059094  2297 raft_consensus.cc:2393] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 21: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
W20260709 13:04:22.991597  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 159543us
I20260709 13:04:22.991819  8996 raft_consensus.cc:2804] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:23.062808  8996 raft_consensus.cc:697] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:23.063833  8996 consensus_queue.cc:237] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:23.072891  8574 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:23.078191  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:22.994660  9043 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.533s	user 0.003s	sys 0.000s
I20260709 13:04:23.005409  8919 raft_consensus.cc:993] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:04:23.083371  8919 raft_consensus.cc:1081] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:04:22.844573  8935 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:23.020195  2303 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:23.089388  8935 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:23.090646  1637 raft_consensus.cc:2410] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:04:23.096781  1462 catalog_manager.cc:5697] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 14 to 17, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:23.098856  1462 catalog_manager.cc:5697] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:23.051012  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 49174.2us
I20260709 13:04:23.099117   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:04:23.108235  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 287310us
I20260709 13:04:23.119181  8956 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:23.119997  8956 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:23.131904  8953 consensus_meta.cc:287] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.000s
I20260709 13:04:23.049631  8651 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:23.140707  8991 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:23.165040  9001 consensus_queue.cc:1059] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:23.167027  9037 raft_consensus.cc:3055] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:04:23.167656  9037 raft_consensus.cc:740] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:23.168642  9037 consensus_queue.cc:260] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:23.172215  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:23.172973  2303 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 22.
I20260709 13:04:23.109808  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:23.198287  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 57778.8us
I20260709 13:04:23.200634  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:23.214933  1609 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:23.122792  8956 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:23.235119  9016 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.236382  9016 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:23.238849  9016 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:23.241845  9037 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:23.244141  2271 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.244375  8478 consensus_queue.cc:1059] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:23.245093  8574 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:23.253417  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:23.260021  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 45165.5us
I20260709 13:04:23.303406  2297 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:04:23.261039  8987 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:23.265009  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 155190us
I20260709 13:04:23.265635  9056 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:23.267033  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 433339us
I20260709 13:04:23.261724  8935 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:04:23.273396  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.278358  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 64821us
I20260709 13:04:23.301167  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:23.319944  1681 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:23.324191  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:23.325945  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:23.332664  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:23.333566  1642 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:04:23.336431  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:23.348166  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:23.351068  1607 leader_election.cc:400] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 22. Current term is 23.
I20260709 13:04:23.351820  1607 leader_election.cc:403] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:23.354357  1643 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60378
I20260709 13:04:23.354702  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:23.355156  1643 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.355504  1643 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:23.343189  8979 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:23.275358  1462 catalog_manager.cc:5697] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 18 to 20, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 20 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:23.305454  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:23.452076  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 132117us
I20260709 13:04:23.315708  8935 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.453554  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:04:23.463035  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 149496us
I20260709 13:04:23.358206  9032 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:23.477977  9032 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:23.478382  9032 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:23.319967  8987 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:23.489338  8987 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:23.491406  8990 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:23.455422  8992 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:23.493826  8990 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:23.508980  8862 raft_consensus.cc:3055] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:04:23.509588  8862 raft_consensus.cc:740] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:23.510629  8862 consensus_queue.cc:260] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:23.513741  8862 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:23.535286  8991 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:23.536054  8991 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:23.466025  9016 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.539592  9016 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:23.553138  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:23.553910  1641 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:04:23.576210  2269 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:23.578263  1642 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 23.
I20260709 13:04:23.597117  8952 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:23.597697  8952 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.598099  8952 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:23.463371  8935 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:23.602267  1643 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:23.602738  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 51290.1us
I20260709 13:04:23.542629  9016 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:23.616278  1643 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:23.607748  2305 raft_consensus.cc:2468] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:04:23.580200  8979 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 22. Current term is 23.
W20260709 13:04:23.620433  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 166886us
I20260709 13:04:23.625538  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:23.636619  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:23.638863  1717 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:23.625095  8859 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:23.614516  8919 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 13:04:23.651120  8859 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:23.652132  8859 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:23.654229  1560 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.005s	sys 0.008s
W20260709 13:04:23.654706  1560 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.005s	sys 0.008s
W20260709 13:04:23.655794  2296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.005s	sys 0.009s
W20260709 13:04:23.656178  2296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.005s	sys 0.009s
I20260709 13:04:23.667651  1560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41282 (request call id 2893) took 1335 ms. Trace:
I20260709 13:04:23.668206  1560 rpcz_store.cc:276] 0709 13:04:22.332276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.333233 (+   957us) service_pool.cc:224] Handling call
0709 13:04:22.333577 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:22.364404 (+ 30827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:22.364433 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:22.364952 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:04:22.894708 (+529756us) spinlock_profiling.cc:243] Waited 51.4 ms on lock 0x7b1800198c40. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a18494bc 00007fd79eb9748b 00007fd7a79f1ab9 00007fd79ebb369b 00007fd79ebb193f 00007fd79ebb3c02 00007fd7a183b20b 00007fd7a1840d23 00007fd7a18410d1
0709 13:04:22.894893 (+   185us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:04:23.663335 (+768442us) log.cc:844] Serialized 14 byte log entry
0709 13:04:23.663679 (+   344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.663761 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:23.663801 (+    40us) raft_consensus.cc:1685] finished
0709 13:04:23.667423 (+  3622us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.667590 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30479,"raft.queue_time_us":32290,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":41913,"spinlock_wait_cycles":118227200,"thread_start_us":878944,"threads_started":2,"wal-append.queue_time_us":249384}
I20260709 13:04:23.664762  8931 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:23.667479  8991 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:23.669752  8924 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.000s	sys 0.014s
I20260709 13:04:23.682685  9010 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:23.691264  9009 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:23.708257  9009 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:23.709796  9032 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:23.710472  9009 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:23.725446  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:23.717176  9010 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:23.699630  8924 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.001s	sys 0.015s
I20260709 13:04:23.706305  8952 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:23.733243  8996 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.004s	sys 0.013s
I20260709 13:04:23.761368  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:23.762629  1711 raft_consensus.cc:3055] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:23.763134  1711 raft_consensus.cc:740] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:23.763887  1711 consensus_queue.cc:260] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:23.768432  1711 raft_consensus.cc:3060] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:23.783512  2268 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:23.785069  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:23.753063  8996 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.004s	sys 0.013s
W20260709 13:04:23.787735  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.757082  9032 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:23.792179  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 441084us
I20260709 13:04:23.792989  1607 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:23.798763  2304 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
W20260709 13:04:23.801381  1568 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.014s	sys 0.006s
W20260709 13:04:23.801975  1568 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.014s	sys 0.006s
I20260709 13:04:23.787488  9009 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:23.804901  8979 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:04:23.826718  1566 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.007s	sys 0.012s
W20260709 13:04:23.827343  1566 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.008s	sys 0.012s
I20260709 13:04:23.791718  8935 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:23.832965  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:23.838172  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:04:23.840052  1715 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.005s	sys 0.014s
W20260709 13:04:23.840492  1715 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.006s	sys 0.014s
I20260709 13:04:23.801117  1607 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:23.846453  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.866664  1715 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36078 (request call id 2828) took 1058 ms. Trace:
W20260709 13:04:23.892163  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 52176.1us
W20260709 13:04:23.897316  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 273767us
I20260709 13:04:23.903476  1640 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 22. Current term is 23. Ops: []
I20260709 13:04:23.826128  8979 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:23.867168  1715 rpcz_store.cc:276] 0709 13:04:22.783197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.783371 (+   174us) service_pool.cc:224] Handling call
0709 13:04:22.783674 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.783874 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:04:23.293754 (+509880us) log.cc:844] Serialized 14 byte log entry
0709 13:04:23.841351 (+547597us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:23.841425 (+    74us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:23.841487 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.841574 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.841729 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":832551,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":107,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":1036482,"threads_started":2}
I20260709 13:04:23.907744  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:23.842206  8826 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.858s	user 0.009s	sys 0.013s
W20260709 13:04:23.919441  8826 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.936s	user 0.009s	sys 0.013s
I20260709 13:04:23.937858  1715 pending_rounds.cc:85] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 16
I20260709 13:04:23.938277  1715 pending_rounds.cc:107] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 13:04:23.938529  1715 raft_consensus.cc:2889] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:23.940094  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:23.940719  1711 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:23.902209  8574 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:23.909104  8478 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:23.911880  8979 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:23.911118  8952 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:23.959875  2301 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.000s	sys 0.027s
W20260709 13:04:23.960609  2301 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.000s	sys 0.027s
I20260709 13:04:23.959112  1711 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 22.
I20260709 13:04:23.938788  9032 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:23.957175  9034 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:24.003546  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 377133us
W20260709 13:04:23.959244  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 489539us
I20260709 13:04:23.971745  9009 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:24.004599  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 400645us
I20260709 13:04:24.038545  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:24.045442  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:23.960143  8574 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:23.972564  8478 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.097633  2304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 2127) took 1073 ms. Trace:
I20260709 13:04:24.029619  8919 raft_consensus.cc:993] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:04:24.016113  9016 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:24.127418  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 82049.7us
I20260709 13:04:23.990665  9032 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:24.025255  9072 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:24.105099  2304 rpcz_store.cc:276] 0709 13:04:23.023992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.026523 (+  2531us) service_pool.cc:224] Handling call
0709 13:04:23.026878 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.799491 (+772613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.080589 (+281098us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b54001112d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:24.097533 (+ 16944us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1773583744}
W20260709 13:04:24.132956  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 331765us
I20260709 13:04:24.072052  8956 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:24.099745  9070 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:24.160063  1532 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.016s	sys 0.008s
W20260709 13:04:24.160460  1532 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.016s	sys 0.008s
W20260709 13:04:24.160770  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 853746us
I20260709 13:04:24.033205  9034 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.133936  8942 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:04:24.162243  8935 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:24.134888  9016 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:24.197265  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 206854us
I20260709 13:04:24.203575  1642 raft_consensus.cc:1240] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:24.141487  8826 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:24.233397  8826 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:24.235591  8826 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:24.235742  8651 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.206470  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:24.236550  1641 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:24.236337  8651 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.238489  8651 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:24.207234  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 49685.8us
I20260709 13:04:24.154737  8990 raft_consensus.cc:1081] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:04:24.208985  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:24.241669  2268 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:24.210882  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:24.242513  8931 raft_consensus.cc:2804] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:24.243281  8931 raft_consensus.cc:697] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:24.243343  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.244112  1709 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 22.
I20260709 13:04:24.214854  1562 raft_consensus.cc:1240] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:04:24.244501  8931 consensus_queue.cc:237] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.214890  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.215824  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:24.217180  1560 raft_consensus.cc:1240] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:04:24.217110  1609 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:24.251569  8925 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.252153  8925 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.256117  8925 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:24.150743  9072 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:24.131990  9069 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:24.154515  8862 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:24.154764  8956 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:24.269407  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:24.269372  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 52391.4us
I20260709 13:04:24.269356  8862 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
W20260709 13:04:24.272884  9061 log.cc:927] Time spent T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 13:04:24.273124  9063 log.cc:927] Time spent T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:04:24.273909  8956 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:24.274397  9068 log.cc:927] Time spent T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:04:24.275401  9067 log.cc:927] Time spent T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 13:04:24.276427  9060 log.cc:927] Time spent T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.260s	user 0.000s	sys 0.000s
W20260709 13:04:24.277587  9066 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.204s	user 0.000s	sys 0.000s
I20260709 13:04:24.277724  9072 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:24.280850  9016 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:24.281452  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:24.163244  9070 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:24.284401  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.283260  1641 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 24.
I20260709 13:04:24.210098  9079 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.260897  8935 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:24.262243  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 121993us
I20260709 13:04:24.307302  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:24.307652  1566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35222 (request call id 1401) took 1236 ms. Trace:
I20260709 13:04:24.308013  1566 rpcz_store.cc:276] 0709 13:04:23.071307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.075218 (+  3911us) service_pool.cc:224] Handling call
0709 13:04:23.075543 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.075957 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.075994 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:23.076616 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 13:04:23.827806 (+751190us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:04:23.827881 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.827954 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.307445 (+479491us) raft_consensus.cc:1685] finished
0709 13:04:24.307465 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.307598 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1501,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":750800,"threads_started":1,"wal-append.queue_time_us":1022536}
I20260709 13:04:24.311539  1563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:24.312871  1569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:24.309826  1568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41274 (request call id 3556) took 2035 ms. Trace:
I20260709 13:04:24.332652  1569 raft_consensus.cc:3055] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:24.164599   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:24.337487  1560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:24.333647  1569 raft_consensus.cc:740] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:24.340296  1565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
W20260709 13:04:24.308128  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 68144.8us
I20260709 13:04:24.287874  8992 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:24.339470  1564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:24.343021  1564 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 26.
I20260709 13:04:24.338284  1560 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:24.333110  1568 rpcz_store.cc:276] 0709 13:04:22.274505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.276836 (+  2331us) service_pool.cc:224] Handling call
0709 13:04:22.277261 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:22.306719 (+ 29458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:22.306752 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:22.698991 (+392239us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b18004b7780. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb822a6 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a
0709 13:04:22.951936 (+252945us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b2c00000658. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:04:22.952453 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 13:04:23.802474 (+850021us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:04:24.152276 (+349802us) log.cc:844] Serialized 14 byte log entry
0709 13:04:24.152716 (+   440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.152810 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.309570 (+156760us) raft_consensus.cc:1685] finished
0709 13:04:24.309606 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.309758 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47427,"prepare.queue_time_us":173061,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":38182,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":52493,"spinlock_wait_cycles":19463296,"thread_start_us":1168365,"threads_started":2,"wal-append.queue_time_us":1003000}
I20260709 13:04:24.339354  1462 catalog_manager.cc:5697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 18 to 19, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 19 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.340981  1565 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:24.341226  1569 consensus_queue.cc:260] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:24.352828  1569 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:24.306897  2303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 2130) took 1115 ms. Trace:
I20260709 13:04:24.312633  1562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec"
I20260709 13:04:24.291513  9052 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:24.314384  1561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3f64b76180a54492bb015e405caeacec" is_pre_election: true
I20260709 13:04:24.310118  9072 raft_consensus.cc:2804] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:24.357162  1561 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:24.354784  2303 rpcz_store.cc:276] 0709 13:04:23.191405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.194571 (+  3166us) service_pool.cc:224] Handling call
0709 13:04:23.197195 (+  2624us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.197652 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.197695 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:23.207492 (+  9797us) log.cc:844] Serialized 26 byte log entry
0709 13:04:23.664062 (+456570us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:23.664100 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.664163 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.306696 (+642533us) raft_consensus.cc:1685] finished
0709 13:04:24.306715 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.306843 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":597,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":456177,"threads_started":1,"wal-append.queue_time_us":821795}
W20260709 13:04:24.367280  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 120785us
I20260709 13:04:24.306443  9079 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.327916  8935 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:24.398939  1684 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.369496  9072 raft_consensus.cc:697] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 20 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:24.368750  9087 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:24.411314  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:24.404678  1569 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 22.
I20260709 13:04:24.352560  8992 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:24.406003  9074 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:04:24.406826  8449 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:24.407524  8935 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:24.370674  8574 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:24.420572  1641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60378
I20260709 13:04:24.421551  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:24.422380  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:24.399031  9032 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:24.444273  1641 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:24.399389  9080 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:24.444856  1641 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:24.389739  8478 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:24.447250  1533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.443984  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:24.365991  9052 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.415593  9034 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:24.489764  9070 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:24.442996  9087 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.508272  9087 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:24.448892  8449 consensus_queue.cc:1243] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 21.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
W20260709 13:04:24.511548  9077 log.cc:927] Time spent T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.361s	user 0.003s	sys 0.000s
I20260709 13:04:24.517881  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:24.518843  9009 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:24.519634  9009 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:24.520071  9009 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:24.521071  9010 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:24.532160  1716 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36066 (request call id 3078) took 1282 ms. Trace:
I20260709 13:04:24.532820  1716 rpcz_store.cc:276] 0709 13:04:23.249645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.251568 (+  1923us) service_pool.cc:224] Handling call
0709 13:04:23.251962 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.252366 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.252392 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:23.312329 (+ 59937us) spinlock_profiling.cc:243] Waited 45.5 ms on lock 0x7b2c0010a498. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:04:23.320232 (+  7903us) log.cc:844] Serialized 26 byte log entry
0709 13:04:23.960910 (+640678us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:23.960974 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.961079 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.531771 (+570692us) raft_consensus.cc:1685] finished
0709 13:04:24.531804 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.532076 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":510,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"spinlock_wait_cycles":104575872,"thread_start_us":487888,"threads_started":1,"wal-append.queue_time_us":709053}
I20260709 13:04:24.539095  2268 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:24.552305  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 102736us
I20260709 13:04:24.549358  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.556293  1641 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:24.558872  1533 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:24.578152  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 134170us
I20260709 13:04:24.581617  9108 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:04:24.584167   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:04:24.584381  8992 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:24.543305  9073 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:24.540349  9032 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:04:24.578395  9103 log.cc:927] Time spent T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:04:24.586892  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:24.593348  2303 raft_consensus.cc:2468] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
I20260709 13:04:24.551370  9010 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:24.589697  1641 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:24.590642  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:24.588503  1533 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:24.612526  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.600070  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.601362  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:24.448920  8574 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.563599  9037 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:24.614558  9037 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.616833  9037 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:24.611760  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 232839us
I20260709 13:04:24.623601  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:24.623847  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.624815  1681 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:24.590593  8919 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:24.458137  9080 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.626624  1533 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:24.611155  9010 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.590301  9070 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:24.600881  9108 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:24.638461  1462 catalog_manager.cc:5697] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.639151  2301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53464 (request call id 1824) took 1359 ms. Trace:
I20260709 13:04:24.639691  2301 rpcz_store.cc:276] 0709 13:04:23.279973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.280106 (+   133us) service_pool.cc:224] Handling call
0709 13:04:23.280426 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.294137 (+ 13711us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.294167 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:23.294599 (+   432us) log.cc:844] Serialized 26 byte log entry
0709 13:04:23.961020 (+666421us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:04:23.961066 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.961138 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.638955 (+677817us) raft_consensus.cc:1685] finished
0709 13:04:24.638974 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.639095 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13223,"raft.queue_time_us":13331,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":10503,"thread_start_us":502908,"threads_started":1,"wal-append.queue_time_us":1049876}
I20260709 13:04:24.636406  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:24.641687  1718 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:24.632989  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.585361  9073 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:24.664913  9073 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:24.680299  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 53940.4us
I20260709 13:04:24.643376  9108 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:24.644336  9070 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.644521  1462 catalog_manager.cc:5697] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.647133  9080 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:24.634490  9097 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.647367  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.632597  9089 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:24.639344  9010 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:24.631580  9096 raft_consensus.cc:2804] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:04:24.649468  8979 consensus_meta.cc:287] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: Time spent flushing consensus metadata: real 0.671s	user 0.010s	sys 0.002s
I20260709 13:04:24.749883  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.602764  9032 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:24.650213  1458 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45150 (request call id 329) took 29510 ms (client timeout 15000 ms). Trace:
I20260709 13:04:24.627377  8574 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.552012  9072 consensus_queue.cc:237] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.656414  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.657680  9098 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:24.657725  9052 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:24.658296  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.690306  1462 catalog_manager.cc:5697] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.745220  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:24.747442  1640 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 13:04:24.748087  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:24.761617  1641 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:04:24.650233  8919 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:24.750743  1714 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 26: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:24.762032  1462 catalog_manager.cc:5697] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:24.763243  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.764374  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:24.751264  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 124674us
I20260709 13:04:24.688067  8449 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:24.767243  8449 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:24.748960  1642 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:24.742290  9069 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:24.775322  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:24.775200  9069 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.776750  1682 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:24.777443  9069 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:24.743687  8478 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:24.777865  9052 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:24.778525  9052 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.779068  9052 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:24.780362  8574 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:24.759516  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:24.771077  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:24.750084  9096 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:24.778180  8478 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.753223  9108 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:24.753686  9089 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 24 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:24.789592  1718 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 22.
I20260709 13:04:24.786401  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:24.786929  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:24.816175  9096 raft_consensus.cc:3060] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:24.754623  8979 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.823403  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.789849  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:24.824208  1641 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:04:24.786532  8449 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:24.825352  1682 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:24.751957  9097 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.838258  1643 raft_consensus.cc:3060] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:24.833660  8545 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:24.817059  9076 consensus_queue.cc:1059] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 13:04:24.826113  9108 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:24.843951  8245 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:24.814442  9098 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:04:24.844503  8245 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.854113  1718 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:24.835139  9069 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:24.837884  8935 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:24.840883  8545 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:24.871100  8545 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:24.845125  9083 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:24.881336  1643 raft_consensus.cc:2468] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 22.
I20260709 13:04:24.841310  9094 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:24.854175  9076 consensus_queue.cc:1243] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 21.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:04:24.858798  8449 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:24.890153  8245 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:24.898391  9064 log.cc:927] Time spent T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.871s	user 0.004s	sys 0.010s
I20260709 13:04:24.899648  2296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 2125) took 1878 ms. Trace:
I20260709 13:04:24.900066  2296 rpcz_store.cc:276] 0709 13:04:23.021100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.021217 (+   117us) service_pool.cc:224] Handling call
0709 13:04:23.021532 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:23.021917 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:23.021948 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:23.147475 (+125527us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b2c00047a38. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a059aa11 00007fd7a05aa372 00007fd7a1cdca01 00007fd7a1ca143a 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2
0709 13:04:23.147991 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:04:23.656568 (+508577us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:04:23.656609 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.656677 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:24.897793 (+1241116us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
0709 13:04:24.899404 (+  1611us) raft_consensus.cc:1685] finished
0709 13:04:24.899432 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.899582 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2981,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":285493376,"thread_start_us":508181,"threads_started":1,"wal-append.queue_time_us":808440}
I20260709 13:04:24.902590  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:24.904508  9097 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:24.904858  1643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53222 (request call id 1798) took 1085 ms. Trace:
I20260709 13:04:24.905269  1643 rpcz_store.cc:276] 0709 13:04:23.818985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.819105 (+   120us) service_pool.cc:224] Handling call
0709 13:04:24.904696 (+1085591us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b54003250d4. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca52fa 00007fd7a6c66b17 00007fd7a01e16d4 00007fd7a01e1662 00007fd7a01e1591 00007fd7a01e150c 00007fd7a01e07c2 00007fd79ffdc48c 00007fd79ffdb8c6 00007fd79ffdb26f
0709 13:04:24.904802 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2297133440}
I20260709 13:04:24.910460  1532 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:24.912679  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:24.854336  9108 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.918597  1532 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:24.926380  1532 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:24.893203  8449 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:24.929821  1532 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:24.929529  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:24.924618  8870 raft_consensus.cc:493] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.932513  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:24.933019  8870 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.912053  9113 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:24.924508  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 88483.5us
I20260709 13:04:24.947404  1607 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:24.881641  9069 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.951591  9069 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:24.953097  9069 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:24.919339  8952 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.881006  8935 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.856783  9089 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:24.917824  8931 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:24.948858  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:24.885015  9009 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.921190  9099 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:24.923154  8651 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:24.933594  9108 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:24.924125  9081 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:24.967108  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:24.931180  8919 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:24.933580  8987 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.986476  8987 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:24.754297  1458 rpcz_store.cc:269] 0709 13:03:55.139963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:55.144045 (+  4082us) service_pool.cc:224] Handling call
0709 13:04:24.650101 (+29506056us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:17.828422 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:17.829367 (+   945us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:17.829439 (+    72us) write_op.cc:435] Acquired schema lock
0709 13:04:17.829468 (+    29us) tablet.cc:662] Decoding operations
0709 13:04:17.918980 (+ 89512us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:17.957595 (+ 38615us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:17.960048 (+  2453us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:04:18.028622 (+ 68574us) tablet.cc:701] Row locks acquired
0709 13:04:18.028690 (+    68us) write_op.cc:260] PREPARE: finished
0709 13:04:18.028939 (+   249us) write_op.cc:270] Start()
0709 13:04:18.029172 (+   233us) write_op.cc:276] Timestamp: P: 1783602258028852 usec, L: 0
0709 13:04:18.029227 (+    55us) op_driver.cc:348] REPLICATION: starting
0709 13:04:18.075805 (+ 46578us) log.cc:844] Serialized 5978 byte log entry
0709 13:04:19.168895 (+1093090us) op_driver.cc:464] REPLICATION: finished
0709 13:04:19.617653 (+448758us) write_op.cc:301] APPLY: starting
0709 13:04:19.617774 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:19.914625 (+296851us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:19.945250 (+ 30625us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:20.631930 (+686680us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:20.632302 (+   372us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=30,mrs_lookups=0
0709 13:04:20.632369 (+    67us) write_op.cc:312] APPLY: finished
0709 13:04:20.632818 (+   449us) log.cc:844] Serialized 139 byte log entry
0709 13:04:21.142988 (+510170us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:21.152523 (+  9535us) write_op.cc:454] Released schema lock
0709 13:04:21.152674 (+   151us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:21.153512 (+   838us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:21.153653 (+   141us) write_op.cc:454] Released schema lock
0709 13:04:21.564151 (+410498us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b4400022b4c. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eadac85 00007fd79eadb6a2 00007fd7a184d1db 00007fd7a1848aaf 00007fd7a183c86e 00007fd7a183bbb6 00007fd7a183b7d8 00007fd7a1845221 00007fd7a18451d9 00007fd7a1845169
Metrics: {"mutex_wait_us":335517,"reported_tablets":46,"spinlock_wait_cycles":1280,"child_traces":[["op",{"apply.queue_time_us":448357,"apply.run_cpu_time_us":52887,"apply.run_wall_time_us":1535768,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66857,"num_ops":15,"prepare.queue_time_us":87988,"prepare.run_cpu_time_us":25023,"prepare.run_wall_time_us":571775,"raft.queue_time_us":413208,"raft.run_cpu_time_us":44093,"raft.run_wall_time_us":462276,"replication_time_us":1139403,"spinlock_wait_cycles":42672000,"thread_start_us":1203630,"threads_started":5,"wal-append.queue_time_us":940554,"wal-append.run_cpu_time_us":41843,"wal-append.run_wall_time_us":2817953}]]}
I20260709 13:04:24.988829  8987 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:24.976263  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:24.989316  9115 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:04:24.989704  1640 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 23.
I20260709 13:04:24.886808  9083 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.922782  8996 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:24.954573  9112 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:24.966315  9095 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:24.893762  9094 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.978298  8478 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:24.958734  9032 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:24.973505  9069 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:24.980469  9099 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:24.978972  8651 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.995194  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:25.027743  1640 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
W20260709 13:04:25.021332  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 91815.5us
I20260709 13:04:25.030630  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:24.966325  8952 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.041683  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:24.999151  9037 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:25.052377  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:24.973186  8931 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.058210  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.012615  9069 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:25.033471  8651 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:25.061669  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:25.066985  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 99905.7us
I20260709 13:04:25.062947  1534 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:24.981070  9052 raft_consensus.cc:515] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.032998  8979 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.073078  2302 raft_consensus.cc:2393] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 20: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:25.070366  2296 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:25.070753  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:25.034859  9117 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:25.012651  8996 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.062268  8826 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:25.119249  8826 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:25.119711  8826 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:25.120676  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 50122.9us
I20260709 13:04:25.121745  1607 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:25.123752  8979 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:25.123225  1607 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:25.125849  8979 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:25.126296  8870 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.126381  8979 raft_consensus.cc:697] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 22 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:25.052582  9083 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.127395  9083 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:25.128026  9083 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:25.127565  8979 consensus_queue.cc:237] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:25.110762  8935 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.116640  8996 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.130138  8952 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.130301  9083 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.066296  9069 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:25.080348  9117 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:25.132851  9117 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.072970  9080 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:25.135104  9094 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.118671  9052 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.138782  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.139295  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.142660  8931 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.144038  8942 raft_consensus.cc:3055] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:04:25.144538  8942 raft_consensus.cc:740] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:25.145475  8942 consensus_queue.cc:260] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.152375  9009 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:24.993743  8449 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:25.157104  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.158326  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:24.994868  9081 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.161370  9081 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.163308  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:25.164028  1710 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:04:25.165730  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.166761  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.167397  2299 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:04:25.172576  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.173731  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.175127  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:25.175727  2298 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:25.173625  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:25.176707  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 107928us
I20260709 13:04:25.161864  9097 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:25.162364  1640 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 13.
I20260709 13:04:25.177855  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:25.178464  1644 raft_consensus.cc:3060] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:04:25.146695  8942 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:25.179895  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.180958  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:25.182077  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:25.185724  1709 raft_consensus.cc:2368] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22.
I20260709 13:04:25.178665  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.189729  9097 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:25.192202  9097 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:25.193017  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.187351  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.197705  1717 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 22.
W20260709 13:04:25.188781  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 232817us
I20260709 13:04:25.207060  9124 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:25.207406  9079 leader_election.cc:290] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.207729  9124 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.211095  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.218035  1644 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:04:25.123842  9118 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:25.233328  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.234462  1710 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:04:25.234658  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:25.229888  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.230051  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.235690  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:25.230578  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.118681  9099 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:25.228268  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 50841.7us
I20260709 13:04:25.235782  1715 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 17.
I20260709 13:04:25.196745  9081 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:25.254072  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.246908  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:25.251093  1716 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 16: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:25.255935  1641 raft_consensus.cc:1240] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 13:04:25.231873  1532 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:25.257426  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.258206  1714 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:25.264611  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.178727  8545 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:25.272228  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.275449  1683 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:25.277958  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:25.279346  1607 leader_election.cc:400] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22.
I20260709 13:04:25.279879  1607 leader_election.cc:403] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:04:25.277352  1683 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:25.291850  1642 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 19. Current term is 21. Ops: []
I20260709 13:04:25.249997  9120 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:25.254640  8931 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:25.210943  8651 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.250370  9126 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:25.209903  8826 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:25.329342  1712 raft_consensus.cc:2393] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 22: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:25.296645  1607 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:25.312628  8996 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:25.291785  9118 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:25.314636  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.320703  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:25.343974  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 47382.7us
I20260709 13:04:25.271596  9129 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:25.296552  9112 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:04:25.387271  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 109922us
I20260709 13:04:25.319813  8536 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:25.324936  9069 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:25.390904  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.261651  9081 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:25.402630  1607 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
I20260709 13:04:25.363067  9118 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.366120  9129 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.355491  8996 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.365517  9094 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:25.412457  9009 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.389719  8536 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:25.355430  9122 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997) with higher term. Message: Invalid argument: T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22.
I20260709 13:04:25.395371  9052 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:25.327291  8931 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.396361  8887 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:04:25.443740  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 196797us
I20260709 13:04:25.413949  8956 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:25.329180  9120 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.419780  9118 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:25.431881  8536 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:25.427145  8996 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:25.450263  9106 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:25.456661  8931 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.446201  8887 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.418335  9129 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:25.445261  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:25.440913  9010 consensus_meta.cc:287] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent flushing consensus metadata: real 0.686s	user 0.010s	sys 0.008s
W20260709 13:04:25.464996  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 203138us
W20260709 13:04:25.444774  9096 consensus_meta.cc:287] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: Time spent flushing consensus metadata: real 0.615s	user 0.000s	sys 0.021s
I20260709 13:04:25.468050  2298 raft_consensus.cc:2468] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
W20260709 13:04:25.469036  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 60349.9us
I20260709 13:04:25.443917  9094 raft_consensus.cc:697] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:25.443567  9124 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:25.473300  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 155705us
W20260709 13:04:25.497539  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 52316.7us
I20260709 13:04:25.397830  9112 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.500168  1683 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:25.498893  2269 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:25.441061  9009 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.503149  9010 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.519699  9127 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:04:25.498711  9056 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:25.537253  1637 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:04:25.537526  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:25.510703  9110 raft_consensus.cc:2804] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:25.508144  8952 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:25.543032  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250e1f9f055e4fda9987138306f2e0b7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:25.475809  9106 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.462097  8956 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.549686  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:04:25.550594  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 147896us
I20260709 13:04:25.552989  1643 raft_consensus.cc:2393] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 16: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:25.538337  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:25.504850  9096 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.555423  8826 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.558614  9135 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:25.560650  8887 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.564033  9009 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:25.564579  9081 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.552160  8952 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:04:25.568658  9131 log.cc:927] Time spent T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: Append to log took a long time: real 0.254s	user 0.000s	sys 0.002s
I20260709 13:04:25.553804  9142 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:25.550474  9087 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:04:25.571439  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 71293.1us
I20260709 13:04:25.572764  9112 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:25.576607  1743 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:37201 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:37201 timed out after 15.000s (SENT)
I20260709 13:04:25.579407  1639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60390 (request call id 3471) took 1330 ms. Trace:
I20260709 13:04:25.582729  1532 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:25.585291  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:25.585827  1717 raft_consensus.cc:1240] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 19. Current term is 21. Ops: []
I20260709 13:04:25.531356  9127 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:25.611922  1639 rpcz_store.cc:276] 0709 13:04:24.248948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.250802 (+  1854us) service_pool.cc:224] Handling call
0709 13:04:24.251112 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:24.281681 (+ 30569us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:04:24.305108 (+ 23427us) log.cc:844] Serialized 14 byte log entry
0709 13:04:24.399818 (+ 94710us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:24.399902 (+    84us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:04:24.399966 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.677721 (+277755us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b18003ea880. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79ebb4522 00007fd79ebb67c1 00007fd79ebb6779 00007fd79ebb6709 00007fd79ebb66d1 00007fd79ebb59cd 000000000038f354 000000000038f199 00007fd79eb9783a
0709 13:04:24.835005 (+157284us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b18003ea880. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb822a6 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419
0709 13:04:25.579120 (+744115us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b5400247554. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1ca1e01 00007fd7a1ca083e 00007fd7a6c65419 00007fd7a01dcbd4 00007fd7a01dcb62 00007fd7a01dca91 00007fd7a01dca0c 00007fd7a01dbcc2 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:25.579172 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.579337 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":88303,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":78,"raft.queue_time_us":121,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":2726,"spinlock_wait_cycles":75896576,"thread_start_us":116975,"threads_started":2,"wal-append.queue_time_us":199086,"wal-append.run_cpu_time_us":1072,"wal-append.run_wall_time_us":1001102}
I20260709 13:04:25.613831  9127 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:25.615442  9087 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.617786  9087 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.615512  9089 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:25.619738  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:25.620370  1643 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:25.620671  8956 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:25.623523  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.587188  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:25.544234  8545 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:25.526485  9094 consensus_queue.cc:237] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:25.582150  9110 raft_consensus.cc:697] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:25.591557  8651 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.591894  8953 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:25.636552  9120 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.636646  8953 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.638622  9110 consensus_queue.cc:237] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.625021  2269 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:25.605770  8952 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:25.646027  1643 raft_consensus.cc:2468] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 22.
I20260709 13:04:25.627514  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:25.624470  9069 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 26 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.599118  9135 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.632697  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:25.644150  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:25.620393  9129 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.651705  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:25.652148  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:25.666806  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.671691  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.675966  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:25.676994  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:25.679996  1711 raft_consensus.cc:2393] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 26: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:25.550014  9056 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.689774  1682 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:25.682372  1533 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:25.701807  9135 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:25.704564  9129 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.704746  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:25.705999  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.707240  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.708626  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.709432  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.710210  2298 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 24: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:25.711015  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.711786  9010 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:25.712956  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.713857  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:25.714033  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.714677  2303 raft_consensus.cc:2368] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22.
I20260709 13:04:25.717841  9155 raft_consensus.cc:2804] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:25.720753  1533 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:25.707325  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.722851  2302 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
I20260709 13:04:25.640519  8953 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:25.726346  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.730516  2297 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:04:25.657631  8912 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:25.743414  1609 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:25.745141  1609 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:25.746639  1609 leader_election.cc:400] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22.
I20260709 13:04:25.747241  1609 leader_election.cc:403] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:25.729743  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 113700us
I20260709 13:04:25.632822  9009 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.088s
I20260709 13:04:25.762171  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:25.762897  1636 raft_consensus.cc:3060] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:04:25.624907  9142 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.730223  8990 raft_consensus.cc:3060] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:25.767504  1609 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:25.737255  9112 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:25.750169  9122 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:04:25.803005  1609 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:25.750352  9081 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:25.806066  9099 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:25.806560  9099 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:25.807024  9099 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:25.805368  9126 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:25.808252  9126 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:25.808692  9126 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:25.809548  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.814149  9070 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.819950  8862 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:25.729532  9127 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.825640  9127 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.731452  8536 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.828424  8536 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.829854  8996 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.832273  8996 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.832937  9089 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:25.833493  9089 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.834496  9112 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:25.834750  2304 raft_consensus.cc:1240] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:25.835083  9112 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.835681  9089 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.836269  9089 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:25.837208  9112 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.836829  9089 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.839545  9056 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.840852  9096 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.841249  9089 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.758056  9087 consensus_queue.cc:1059] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:04:25.843551  8870 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:25.843735  9087 raft_consensus.cc:3055] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:25.844321  9087 raft_consensus.cc:740] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:25.844275  8870 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.847159  9142 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:25.847667  9074 log.cc:927] Time spent T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.430s	user 0.005s	sys 0.006s
I20260709 13:04:25.849735  9106 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.792081  9150 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:25.851341  9150 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.853543  9150 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:25.854027  8942 consensus_meta.cc:287] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: Time spent flushing consensus metadata: real 0.651s	user 0.010s	sys 0.000s
W20260709 13:04:25.856293  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 54920.1us
I20260709 13:04:25.860493  1714 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22. Ops: []
W20260709 13:04:25.861209  9133 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.320s	user 0.002s	sys 0.000s
I20260709 13:04:25.863646  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:25.867982  8956 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:25.868738  8956 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.868075  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:25.872352  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:25.877082  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.845786  9087 consensus_queue.cc:260] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:25.879176  1636 raft_consensus.cc:2468] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 22.
I20260709 13:04:25.882584  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:25.883255  1718 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:04:25.883591  2269 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:25.884135  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:25.884573  1716 raft_consensus.cc:1275] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
W20260709 13:04:25.885270  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.870954  8956 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.880134  1643 raft_consensus.cc:2393] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 23: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:25.889276  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:25.887966  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:25.893849  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:25.898056  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 95148.6us
I20260709 13:04:25.899001  1609 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:25.801762  9113 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:25.903616  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.903702  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.803958  9079 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:25.905234  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.907537  1718 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 24.
I20260709 13:04:25.911590  9099 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.915010  9099 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.847179  8870 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.928887  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:25.930004  2296 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:25.886546  9094 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) with higher term. Message: Invalid argument: T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22.
I20260709 13:04:25.902999  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.937849  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:25.902879  9069 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:25.902406  8996 raft_consensus.cc:2804] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:25.933110  9087 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:04:25.927122  9113 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.953821  9113 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:25.960731  9144 log.cc:927] Time spent T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:04:25.961970  8990 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:25.932476  9079 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.967605  9079 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.942123  8952 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.975440  8449 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:25.977094  8952 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:25.979161  2299 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:25.946514  8996 raft_consensus.cc:697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:25.936230  9118 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:25.983367  9155 raft_consensus.cc:697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:26.000847  9155 consensus_queue.cc:237] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:25.997689  9118 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.985144  9126 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.011286  9126 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:25.994704  8996 consensus_queue.cc:237] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.105811  1718 raft_consensus.cc:1275] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:04:26.122243  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 221462us
I20260709 13:04:26.124972  9126 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.134581  9126 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:26.139680  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:26.140920  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:26.141578  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:26.141891  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:26.142876  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:26.143558  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:26.144635  2301 raft_consensus.cc:2468] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:26.144745  1709 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 LEADER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:26.154901  9162 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.155961  9162 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:26.165450  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:26.166057  9118 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:26.166599  2297 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 22: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:26.167343  9162 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:26.167002  9118 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:26.167750  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:26.170050  9118 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:26.171121  1638 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 24: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:26.187368  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:26.191782  9155 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.193913  9155 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:26.203299  9126 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:26.193789  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:26.205049  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:26.206163  1639 raft_consensus.cc:1240] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:26.206303  1718 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 24: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:26.208670  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:26.190717  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 427258us
I20260709 13:04:26.225955  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:26.232193  9087 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:26.232344  9151 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:26.233265  9113 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:26.233687  9151 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:26.237082  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:26.239490  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:26.241271  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:26.233459  9087 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.226250  1532 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:26.247839  9162 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:26.250842  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:26.253880  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:26.259999  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:26.260990  2300 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:26.234589  9113 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.212648  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.263154  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:26.265805  9126 consensus_queue.cc:1059] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:26.278080  8951 consensus_queue.cc:1059] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:26.281415  9113 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:26.283524  9078 raft_consensus.cc:3055] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:26.284162  1710 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:26.283711  9052 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.284224  9078 raft_consensus.cc:740] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:26.285136  9052 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.283586  9124 consensus_queue.cc:1048] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:04:26.288043  9052 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:26.286121  9078 consensus_queue.cc:260] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.289921  2299 raft_consensus.cc:1240] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:04:26.290810  9078 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:26.284435  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:26.266422  9087 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:26.264230  9151 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:26.311827  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:26.313189  2304 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 24: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:26.313774  1683 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:26.316438  9124 raft_consensus.cc:3055] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:26.320518  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:26.330176  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:26.331463  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.331763  1638 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:04:26.332311  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.333603  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.318571  9155 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:26.309019  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:26.347208  2300 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:04:26.387648  1710 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:04:26.395252  2297 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:26.405360  1644 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 26.
I20260709 13:04:26.384317  9124 raft_consensus.cc:740] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:26.414134  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.439092  1713 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 26.
I20260709 13:04:26.444182  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.446550  9124 consensus_queue.cc:260] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.447873  9124 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:26.441771  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:26.456281  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 56427.8us
W20260709 13:04:26.462100  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 156737us
I20260709 13:04:26.464856  8887 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:26.465827  8887 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.467429  8545 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:26.474133  9113 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:26.474897  9113 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:26.478459  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.484927  1609 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:26.486287  9122 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:26.486977  9122 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 20 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:26.488561  8887 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:26.488175  9122 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.482141  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.491948  9087 consensus_queue.cc:1059] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:26.492867  9087 consensus_queue.cc:1243] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer 28bc59d37f664ce0a6f5784f2619f0b5 log is divergent from this leader: its last log entry 22.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:04:26.472563  9163 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:26.502735  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:26.499333  9162 raft_consensus.cc:3055] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:04:26.508441  8887 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.511641  2302 raft_consensus.cc:1275] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:04:26.507572  9162 raft_consensus.cc:740] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:26.516003  9162 consensus_queue.cc:260] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:26.517539  9162 raft_consensus.cc:3060] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:26.521428  9163 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.524192  9163 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:26.513065  1532 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:26.541625  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:26.541997  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 263084us
I20260709 13:04:26.542860  1683 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:26.543190  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:26.543888  1714 raft_consensus.cc:2393] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 18: Already voted for candidate 3f64b76180a54492bb015e405caeacec in this term.
I20260709 13:04:26.544253  8859 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:26.544562  1711 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 13. (term mismatch)
I20260709 13:04:26.544945  1711 pending_rounds.cc:85] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 12
I20260709 13:04:26.545207  1711 pending_rounds.cc:107] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 13:04:26.545490  1711 raft_consensus.cc:2889] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:26.547716  9071 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:26.548435  9071 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:26.549715  9071 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:26.551105  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 51234.8us
I20260709 13:04:26.556249  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.558813  1715 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 23.
I20260709 13:04:26.566848  8449 consensus_queue.cc:1059] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:04:26.570852  9113 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:26.571004  8449 raft_consensus.cc:3055] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:04:26.557693  8887 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:26.571590  8449 raft_consensus.cc:740] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:26.572899  8449 consensus_queue.cc:260] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.573657  8887 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:26.574581  8449 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:04:26.579759  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:26.583029  8924 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:26.583747  8924 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.584897  8449 consensus_queue.cc:1059] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:26.585984  8449 raft_consensus.cc:3055] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:26.586424  8449 raft_consensus.cc:740] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:26.586890  2301 raft_consensus.cc:1275] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:04:26.587497  8449 consensus_queue.cc:260] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.588570  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:26.589570  2301 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 22.
I20260709 13:04:26.591295  9119 consensus_queue.cc:1048] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 13:04:26.592279  1532 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:26.588796  8449 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:26.595402  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 54847.2us
I20260709 13:04:26.566818  9157 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.586005  8924 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:26.597710  8887 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 13:04:26.605723  1681 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.609965  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:26.611698  1644 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:04:26.614571  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:26.604887  9113 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:26.603627  9078 raft_consensus.cc:493] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.611495  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:26.622481  1683 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:26.609740  8887 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.625046  8887 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:26.630867  8545 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:26.638456  8545 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.638854  8545 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:26.604729  9157 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:26.635169  9162 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:04:26.623644  9155 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:04:26.654739  1532 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:26.663794  9162 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:26.671667  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.675246  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.676812  1713 raft_consensus.cc:2468] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 22.
I20260709 13:04:26.676468  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.655557  9078 raft_consensus.cc:515] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.681140  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:26.668299  9119 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:26.696264  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:26.699659  1532 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:26.705778  8956 raft_consensus.cc:2804] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:26.706316  8956 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.706758  8956 raft_consensus.cc:3060] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:26.707731  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 60517.4us
I20260709 13:04:26.694216  9087 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:26.711081  9087 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:26.711877  9168 log.cc:927] Time spent T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:04:26.713554  9087 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:26.714223  9087 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:26.715966  9162 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:26.721400  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:26.699090  9119 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:26.730768  1453 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3f22f7f5cede49bea2083e8905932fb7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6208c2fa49f643b5a50ab8f6d292fab2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:26.743651  1681 leader_election.cc:336] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:26.744367  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:26.745808  1681 consensus_peers.cc:597] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:26.749450  1681 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.749228  9159 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:26.749706  9162 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:26.763499  9167 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.443s	user 0.000s	sys 0.006s
W20260709 13:04:26.764079  1607 leader_election.cc:336] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:26.763581  8887 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:26.767501  1681 consensus_peers.cc:597] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.765843  1607 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:26.765503  8994 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:26.770792  8994 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.772791  9126 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:26.756424  8924 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:26.774592  9151 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:26.779582  8924 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.779815  9151 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:26.781054  9162 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:26.779218  1681 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.791240  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.795809  2268 leader_election.cc:304] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:26.799750  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 46974us
I20260709 13:04:26.773182  9078 leader_election.cc:290] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:26.801667  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.781752  8924 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:26.762811  8956 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:26.781790  9151 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:26.808773  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:26.774564  8994 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:26.801172  9126 raft_consensus.cc:493] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:26.809577  1681 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.818132  8956 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:26.821928  1607 leader_election.cc:336] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:26.834856  1711 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 22: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:04:26.835115  1681 leader_election.cc:336] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:26.821192  9151 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:04:26.857512  8996 raft_consensus.cc:2749] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:26.863777  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 51719.8us
W20260709 13:04:26.864848  1607 leader_election.cc:336] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:26.866168  1607 leader_election.cc:336] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:26.867215  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 101130us
I20260709 13:04:26.867722  9162 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:26.868762  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 45553.9us
W20260709 13:04:26.869736  1681 leader_election.cc:336] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:26.870121  1681 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:26.871081  9121 consensus_queue.cc:1048] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:04:26.872570  8887 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 27 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:26.874575  9159 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:26.845224  9126 raft_consensus.cc:515] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.875241  1639 raft_consensus.cc:1275] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:04:26.866595  1607 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:26.877081  9126 leader_election.cc:290] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:26.877682  1607 leader_election.cc:336] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:26.878460  1682 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:26.879443  9052 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:26.881095  9052 consensus_queue.cc:1048] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:04:26.879025  1681 leader_election.cc:336] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:26.883476  1681 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:26.878226  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:26.891398  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:26.887784  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 178986us
I20260709 13:04:26.877085  9124 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:26.897420  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:26.888190  9157 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:26.900030  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:26.894493  9126 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:26.907042  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:26.896108  9159 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:26.889048  8545 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:26.919175  1638 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 23.
I20260709 13:04:26.928218  1640 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: []
W20260709 13:04:26.930902  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 126664us
I20260709 13:04:26.929514  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:26.922034  1534 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:26.939054  1532 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:26.951407  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:26.952304  1715 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:26.953147  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:04:26.954560  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 147292us
I20260709 13:04:26.954454  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:26.951098  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:26.962767  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 93263.6us
I20260709 13:04:26.959491  1712 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 24.
W20260709 13:04:26.965183  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 86247.2us
I20260709 13:04:26.929348  8994 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:26.947661  9162 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:26.971127  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 56457.6us
I20260709 13:04:27.021441  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:04:27.022099  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 144414us
I20260709 13:04:27.022337  1637 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:04:27.023041  1607 leader_election.cc:336] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.023485  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:27.024124  9151 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:27.027549  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 125162us
I20260709 13:04:27.027877  9177 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:27.028716  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.029799  1533 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:27.031024  9087 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:27.031164  9163 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:27.041567  1681 consensus_peers.cc:597] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.045285  1718 raft_consensus.cc:1275] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:04:27.057621  1641 raft_consensus.cc:1240] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 20 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:27.029822  8990 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:27.067289  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:27.070048  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:27.071444  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:27.071652  1638 raft_consensus.cc:1240] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:04:27.073078  1644 raft_consensus.cc:1275] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:04:27.076296  9119 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:27.067358  9162 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:27.073649  9181 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:27.052389  8994 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:27.098454  1534 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.111145  1637 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 27.
W20260709 13:04:27.102458  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 54855.7us
I20260709 13:04:27.095413  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:27.116807  9155 consensus_queue.cc:1059] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:04:27.119308  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 96257.6us
W20260709 13:04:27.135361  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 80680.8us
W20260709 13:04:27.138340  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 82960.4us
I20260709 13:04:27.139886  2268 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:27.136191  1533 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:27.113873  8990 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:27.153106  8951 consensus_queue.cc:1059] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:27.154441  8951 consensus_queue.cc:1048] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:27.156097  8951 raft_consensus.cc:3055] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:04:27.156504  8951 raft_consensus.cc:740] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:27.157258  9162 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:27.157567  8951 consensus_queue.cc:260] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:27.159406  8951 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:27.163733  8990 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:27.167034  1607 leader_election.cc:336] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.167654  1607 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.168596  9122 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:27.169337  1607 leader_election.cc:336] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.169821  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.170701  9122 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:27.176397  8994 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:27.177174  9179 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:27.188416  1715 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 27.
W20260709 13:04:27.194955  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 58793.6us
W20260709 13:04:27.197630  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 59401.6us
I20260709 13:04:27.203178  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:27.203866  1637 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:27.209373  1607 leader_election.cc:336] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:27.212832  1681 consensus_peers.cc:597] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.201550  9087 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:04:27.214752  9162 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:27.222429  9162 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:27.229017  9162 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:27.233558  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 93911us
I20260709 13:04:27.239634  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:27.244225  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:27.246913  1608 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:27.248471  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 188353us
W20260709 13:04:27.243009  1681 consensus_peers.cc:597] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.253652  1637 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 24.
I20260709 13:04:27.201597  9179 consensus_queue.cc:1048] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:04:27.255297  1682 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:27.243549  9087 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 27 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:27.261934  9173 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:27.275640  9193 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:04:27.277431  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfadedb9a5e4ca0abc25384a524c546" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:27.255380  9162 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:27.280117  9162 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:27.282837  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 73491.5us
W20260709 13:04:27.283705  1607 consensus_peers.cc:597] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:27.284343  1607 leader_election.cc:336] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.284758  1607 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:27.261293  8449 raft_consensus.cc:2804] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:27.287352  9124 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:27.287441  8449 raft_consensus.cc:697] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:27.288750  8449 consensus_queue.cc:237] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:27.290112  1717 raft_consensus.cc:1240] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:04:27.291460  9162 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:27.295953  1683 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:27.265161  1681 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:27.321703  1681 leader_election.cc:336] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.325546  1681 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:27.329316  9078 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:27.335594  9188 log.cc:927] Time spent T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:04:27.337201  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 68448.7us
I20260709 13:04:27.338189  1534 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.339745  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.340128  8990 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:27.341090  9173 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:27.341758  9173 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:27.344761  9194 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:04:27.346520  1607 leader_election.cc:336] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.347062  1607 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:27.348524  1607 leader_election.cc:336] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:27.350071  1607 consensus_peers.cc:597] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:27.365594  1682 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:27.350685  1607 leader_election.cc:336] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.367609  1607 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:27.372053  9124 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:27.392735  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 53080.5us
I20260709 13:04:27.374501  9173 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:27.407797  9124 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:04:27.388643  8956 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.385046  9194 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:27.411474  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.377199  8990 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:27.428016  1683 leader_election.cc:304] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:27.454687  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 214808us
I20260709 13:04:27.463649  9194 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:27.511097  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 99678.1us
I20260709 13:04:27.407424  9087 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:27.423390  8956 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:27.524672  8956 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:27.525753  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 175054us
I20260709 13:04:27.526181  9174 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:27.526873  9174 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:27.526552  1607 leader_election.cc:336] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.528148  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:27.529263  9174 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:27.508987  9173 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:27.522779  9190 log.cc:927] Time spent T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:04:27.522311  2268 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:27.515105  9167 log.cc:927] Time spent T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:04:27.532123  9212 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:27.532511  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 216873us
I20260709 13:04:27.533380  9155 raft_consensus.cc:2749] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:27.511152  8990 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:27.539119  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 108686us
I20260709 13:04:27.537731  9122 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:04:27.550261  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:27.551201  1681 consensus_peers.cc:597] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.555305  9201 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:27.549207  8990 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:27.548920  9212 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:27.571568  8990 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:27.572598  8545 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:27.574674  1681 consensus_peers.cc:597] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.583428  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:27.599536  2302 raft_consensus.cc:3060] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Advancing to term 24
W20260709 13:04:27.589179  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 48242.5us
W20260709 13:04:27.592505  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 71605us
W20260709 13:04:27.601660  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 72278.8us
I20260709 13:04:27.602100  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:27.577076  9212 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:27.607337  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:27.591195  8990 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:27.584734  9196 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:27.626546  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
W20260709 13:04:27.619144  1681 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.636626  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.639179  9151 raft_consensus.cc:2804] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:27.639638  9151 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:27.640329  8990 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:27.640024  9196 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:27.640048  9151 raft_consensus.cc:3060] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:27.645200  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:27.646108  1607 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.658958  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:27.660921  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:27.660959  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:27.665474  1643 raft_consensus.cc:2393] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 20: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
W20260709 13:04:27.667843  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 65841.2us
I20260709 13:04:27.668641  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:27.669394  1710 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 13:04:27.673355  1681 leader_election.cc:336] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:27.674561  1681 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.674809  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:27.675396  9213 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:27.675987  9213 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:27.678239  9213 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:27.678762  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:27.683365  9142 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:27.685884  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:27.642867  9196 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:27.696177  1532 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:27.714833  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a8e85db5b4380bf845a82736ec5fd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:27.715500  1532 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:27.699734  1644 raft_consensus.cc:1240] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:04:27.709038  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:27.698664  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:27.697927  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:27.720909  1681 consensus_peers.cc:597] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.721510  2302 raft_consensus.cc:2468] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 24.
W20260709 13:04:27.721524  1681 leader_election.cc:336] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.723112  9177 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:27.723861  8924 raft_consensus.cc:2764] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:04:27.724192  9113 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:04:27.726331  9157 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:27.726040  9151 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:27.675451  9178 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:27.726851  9157 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:27.727372  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:27.728075  1709 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 22.
I20260709 13:04:27.729041  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:27.729210  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:27.730149  9151 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:27.734719  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:27.684123  9142 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:27.738138  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:27.738859  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:27.739507  1715 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 17.
I20260709 13:04:27.740017  9052 raft_consensus.cc:3055] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:04:27.740424  9052 raft_consensus.cc:740] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:27.741447  9052 consensus_queue.cc:260] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:27.742316  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:27.743029  1709 raft_consensus.cc:2468] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
I20260709 13:04:27.743844  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:27.745076  1709 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:27.745051  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:27.746593  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:27.758596  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.752142  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:27.748621  1451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 58655.8us
I20260709 13:04:27.753901  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:27.770712  1636 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:04:27.772130  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:27.754910  1532 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:27.775681  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:27.753492  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:27.752511  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:27.749078  2303 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 22: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:27.755914  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:27.780164  1534 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.781833  8990 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:27.784080  9113 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:27.784577  9113 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:27.784996  9113 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:27.780508  1532 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:27.786005  2300 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:04:27.784137  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:27.788841  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 67305us
I20260709 13:04:27.789572  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:04:27.790971  1681 leader_election.cc:336] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.791095  8956 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:04:27.791486  1681 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.791574  8956 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:27.792191  8956 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:27.792961  9164 raft_consensus.cc:2804] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:04:27.793344  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 147251us
I20260709 13:04:27.794143  1607 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.747159  9213 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:04:27.795825  9220 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:27.795996  9163 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:27.799780  9181 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:04:27.793378  9164 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.800501  9164 raft_consensus.cc:3060] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:27.804396  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.805987  8887 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:27.804909  9087 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:27.809535  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.810110  9052 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:04:27.792752  1681 leader_election.cc:336] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.818527  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:27.825574  1681 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:27.827168  1451 catalog_manager.cc:4755] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): ChangeConfig:REMOVE_PEER RPC failed for tablet ca83cd5e36eb4e22b75011b87bfcd49d: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:27.828900  1681 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.830772  1534 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:27.838876  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 51103.7us
I20260709 13:04:27.836760  9151 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:27.852707  9214 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:04:27.856782  2268 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:27.858603  2268 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.841190  8887 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:27.865396  2268 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:27.871034  2268 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:27.872520  2268 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:27.877023  1681 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.865695  9087 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.877686  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:27.878146  2268 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.878334  9203 consensus_queue.cc:1048] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:04:27.880913  1458 catalog_manager.cc:5697] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:27.878705  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 45470.3us
W20260709 13:04:27.879770  2268 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.890794  9122 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:27.877859  9220 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
W20260709 13:04:27.900934  1681 leader_election.cc:336] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.908289  1681 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:27.879099  9078 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:27.898334  2268 leader_election.cc:336] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.910486  1607 leader_election.cc:304] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.912745  9203 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.019s
I20260709 13:04:27.882741  9087 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:27.934660  1607 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:27.935316  1607 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.935613  1533 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:27.935953  1607 leader_election.cc:336] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.936273  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:27.941805  1607 consensus_peers.cc:597] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.926263  9228 raft_consensus.cc:2749] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:27.946394  1607 consensus_peers.cc:597] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.947675  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:27.949764  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:27.926177  8449 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:27.917554  9142 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:27.951406  1533 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:27.948237  1607 consensus_peers.cc:597] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:27.951305  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 59207.3us
I20260709 13:04:27.960961  9122 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:27.963600  2297 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:04:27.958209  9177 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:27.955673  9163 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:28.012322  2268 leader_election.cc:336] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:27.910046  9220 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:27.933539  8956 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:27.955756  8990 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:27.967593  9157 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:27.972733  9211 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:28.021761  9227 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:28.021939  9211 consensus_queue.cc:1243] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 5184c101be204393adf3ac8b31f25ad6 log is divergent from this leader: its last log entry 21.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:04:27.978088  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:28.019773  8449 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.978796  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:28.022403  9227 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.024573  1683 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:27.990067  9113 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:27.994699  9087 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:28.000252  9159 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.945575  9203 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:28.012708  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 64566.3us
I20260709 13:04:28.020097  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:28.021181  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:27.945996  9174 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:28.028492  9157 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:28.029745  9227 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.017683  9122 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:28.036450  1607 leader_election.cc:336] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.038676  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:28.035984  9087 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.040620  9193 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:28.041150  9087 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.042971  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:28.045619  9227 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:28.048094  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:28.050810  1532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 84567.5us
I20260709 13:04:28.023079  8449 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.055945  1714 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 21.
I20260709 13:04:28.061769  9087 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.061825  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:28.035616  9119 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
W20260709 13:04:28.071373  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 59046.8us
I20260709 13:04:28.085325  8956 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.085951  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:28.086611  1715 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:28.040122  9159 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.089956  9220 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.051674  9174 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.091451  9159 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.093235  9174 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.094033  1715 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 18.
I20260709 13:04:28.096788  1532 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
W20260709 13:04:28.094882  1607 leader_election.cc:336] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.097472  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:28.098579  1607 leader_election.cc:336] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.095492  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:28.096876  2297 raft_consensus.cc:1275] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:04:28.098912  9113 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.031965  9163 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } }
I20260709 13:04:28.100136  9126 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:28.101344  9163 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:28.101513  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:28.101924  9163 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:28.102429  9163 raft_consensus.cc:697] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:28.103109  9159 consensus_queue.cc:1048] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:28.103277  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:28.103382  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:28.106364  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 74178.5us
I20260709 13:04:28.107311  1533 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:28.108506  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:28.111449  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:28.114243  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:28.115509  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:28.115756  1718 raft_consensus.cc:1240] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:04:28.117049  1609 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:28.117372  1534 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:28.123683  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:28.125772  1533 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:28.126832  1710 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 22.
I20260709 13:04:28.127465  1534 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:28.130034  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:28.130658  1638 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 24.
I20260709 13:04:28.124825  8956 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:28.133466  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:28.133899  8990 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:04:28.134235  1638 pending_rounds.cc:85] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Aborting all ops after (but not including) 16
I20260709 13:04:28.134413  9174 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:28.135030  9151 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:28.135026  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:28.134512  1638 pending_rounds.cc:107] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: Aborting uncommitted NO_OP operation due to leader change: 21.17
I20260709 13:04:28.138898  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:28.141598  1534 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:28.144433  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:28.139122  1638 raft_consensus.cc:2889] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:28.149115  2304 raft_consensus.cc:2393] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 22: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:28.149690  8449 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.152293  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:28.152590  9087 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:28.154664  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:28.155827  9211 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:28.156270  9211 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.156530  9163 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.120354  9119 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:28.159648  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:28.161782  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:28.169867  1714 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 27 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:04:28.156616  9211 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:28.174252  1532 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:28.175644  8990 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:28.176261  8990 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 22 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:28.177213  8990 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.184964  1683 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:28.150451  9231 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:28.196847  1532 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:28.204262  9214 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:28.211357  9233 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.212204  9233 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.214000  8924 consensus_queue.cc:1059] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:04:28.215121  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 55780.8us
I20260709 13:04:28.215379  9206 raft_consensus.cc:3055] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:28.215958  9206 raft_consensus.cc:740] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:28.183923  9159 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:28.216954  9206 consensus_queue.cc:260] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:28.218813  2268 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:28.219532  2268 leader_election.cc:336] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.220041  2268 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:28.221419  2268 leader_election.cc:336] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.221925  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:28.222517  9233 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.224175  1639 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:04:28.227969  9173 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:28.232980  9206 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:28.240600  1639 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:04:28.245575  9175 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:04:28.247553  9164 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:28.250773  9164 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:28.257184  1533 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:28.260052  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 44013.8us
I20260709 13:04:28.263456  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:28.266538  1681 leader_election.cc:336] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.268775  1681 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:28.269521  9162 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:28.261222  1458 catalog_manager.cc:5697] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 18 to 20, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 20 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:04:28.267236  2301 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 22.
I20260709 13:04:28.270061  9173 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:28.270728  9121 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:28.270907  1458 catalog_manager.cc:5697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 19 to 20, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 20 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:28.271836  1609 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:28.272451  1458 catalog_manager.cc:5697] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:28.273103  9213 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:28.273821  9213 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.274289  9213 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:28.260984  9178 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:28.260776  9194 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:28.279281  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 51995.2us
I20260709 13:04:28.280612  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:28.283867  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:28.290058  1532 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:28.323519  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 55290.6us
I20260709 13:04:28.309085  1458 catalog_manager.cc:5697] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:28.319591  9162 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:04:28.322106  9155 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:28.343621  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:28.344329  1715 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 22.
I20260709 13:04:28.325276  8887 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:28.287668  9122 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:28.348362  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 126901us
W20260709 13:04:28.349120  2268 leader_election.cc:336] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.380549  1640 raft_consensus.cc:1240] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Rejecting Update request from peer 3f64b76180a54492bb015e405caeacec for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:28.416401  9213 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:28.416712  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 67577.3us
W20260709 13:04:28.417721  2268 leader_election.cc:336] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.418834  2268 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:28.401108  9220 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.350190  8887 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:28.483716  8990 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:28.486111  8990 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:28.489632  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 71898.1us
I20260709 13:04:28.490211  9178 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:28.490396  2268 leader_election.cc:336] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.490864  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:28.491953  9183 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:28.492185  2268 leader_election.cc:336] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:28.493456  2268 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:28.494105  2268 leader_election.cc:336] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.494460  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:28.495893  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:28.496552  1715 raft_consensus.cc:2468] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 21.
I20260709 13:04:28.496805  8990 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.497646  9194 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:04:28.498171  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:28.499066  8887 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:28.499663  9240 log.cc:927] Time spent T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:04:28.501170  9213 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.502075  9177 raft_consensus.cc:2749] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:28.484082  9220 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.507493  9211 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:28.511907  2268 leader_election.cc:336] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.512454  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:28.513581  2268 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:28.514138  2268 leader_election.cc:336] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.514227  1713 raft_consensus.cc:1240] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Rejecting Update request from peer 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:04:28.514461  9194 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:28.514633  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:28.515515  9194 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:04:28.515908  2268 leader_election.cc:336] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.516381  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:28.517433  2268 leader_election.cc:336] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.517836  2268 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:28.518630  8924 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:28.518303  9194 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:28.521248  2268 leader_election.cc:336] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.521770  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:28.523136  9201 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:28.527294  9211 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.538286  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:28.540125  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:28.541560  2299 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:28.542779  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:28.543540  1642 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
I20260709 13:04:28.546093  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:28.555308  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:28.556063  2300 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 14.
I20260709 13:04:28.559636  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:28.559921  9220 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.560900  2299 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 23.
I20260709 13:04:28.562189  1609 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:28.562690  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:28.563174  9228 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:28.563771  9228 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 23 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:28.564693  9228 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.577551  9183 consensus_queue.cc:1059] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:28.579007  9121 raft_consensus.cc:3055] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:28.579522  9121 raft_consensus.cc:740] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:28.579857  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:28.580569  9121 consensus_queue.cc:260] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.581396  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:28.584125  1533 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:28.609714  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 49261.7us
I20260709 13:04:28.611140  1683 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:28.628593  9121 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:28.634093  1533 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:28.636305  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:28.609877  1458 catalog_manager.cc:5697] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 14 to 17, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 17 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:28.639511  9173 consensus_queue.cc:1059] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:28.641971  9173 raft_consensus.cc:3055] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:28.642473  9173 raft_consensus.cc:740] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:28.645560  9173 consensus_queue.cc:260] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:28.650309  1532 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.631606  8951 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:04:28.631969  9177 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:28.676935  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:28.677587  1709 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:28.654762  9175 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:28.654853  9211 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:04:28.701618  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 67573.7us
W20260709 13:04:28.713290  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 92528us
I20260709 13:04:28.674019  9173 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:28.671200  8951 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.751628  1715 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:04:28.709447  9087 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:04:28.756441  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 90061.9us
I20260709 13:04:28.744711  1458 catalog_manager.cc:5697] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 12 to 15, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 15 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:28.771483  9122 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 25 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:28.774967  9087 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.777390  9087 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.746078  8951 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:04:28.785112  1607 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.785940  9071 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:28.786576  9071 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.788985  9071 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.789985  9201 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:28.790627  9201 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.793246  9201 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:28.808339  1709 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 23.
I20260709 13:04:28.812105  8951 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.815564  8951 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.833727  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
W20260709 13:04:28.834267  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 82211.4us
I20260709 13:04:28.834558  1644 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:04:28.843705  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:28.846400  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:28.847056  2299 raft_consensus.cc:3060] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:04:28.849896  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:28.853107  1715 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:04:28.855674  9203 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:28.857885  8449 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.858367  8449 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.860704  1637 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:28.861579  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:28.862221  1715 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 16.
I20260709 13:04:28.864295  1637 raft_consensus.cc:2393] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 16: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:28.865136  1638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:28.873385  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:28.880281  9174 raft_consensus.cc:493] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:28.881029  9174 raft_consensus.cc:515] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:28.863041  9213 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:04:28.884353  1458 catalog_manager.cc:5697] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 17 to 21, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 21 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:28.909128  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 92846.8us
I20260709 13:04:28.915055  1458 catalog_manager.cc:5697] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 18 to 23, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 23 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:28.919720  2299 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:28.865494  9151 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 13:04:28.925028  1458 catalog_manager.cc:5697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 18 to 21, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 21 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:28.927191  1458 catalog_manager.cc:5697] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 18 to 19, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 19 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:28.929299  1458 catalog_manager.cc:5697] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 13 to 22. New cstate: current_term: 22 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:28.865929  9203 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:28.937520  9203 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:28.927682  8956 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:04:28.955592  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:28.959192  8449 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:28.959455  1462 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60118 (request call id 234) took 23093 ms (client timeout 15000 ms). Trace:
I20260709 13:04:28.960543  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:28.960920  9174 leader_election.cc:290] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:28.961223  1636 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
I20260709 13:04:28.930769  1458 catalog_manager.cc:5697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:28.963524  1682 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:04:28.970382  1607 leader_election.cc:336] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:28.971078  1607 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:28.971756  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:28.982744  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:28.981171  1683 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:28.986454  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:28.988291  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:28.992326  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:28.992956  2302 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:28.993480  2268 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:28.991110  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 44538.9us
I20260709 13:04:28.981722  9212 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:04:28.999722  9252 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:29.000393  9252 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:28.959941  1462 rpcz_store.cc:269] 0709 13:04:05.829112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.829551 (+   439us) service_pool.cc:224] Handling call
0709 13:04:28.914657 (+23085106us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:24.958079 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:24.986408 (+ 28329us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:24.986441 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:04:24.986458 (+    17us) tablet.cc:662] Decoding operations
0709 13:04:24.987419 (+   961us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:24.987505 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:24.987546 (+    41us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:04:24.988244 (+   698us) tablet.cc:701] Row locks acquired
0709 13:04:24.988265 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:04:24.988428 (+   163us) write_op.cc:270] Start()
0709 13:04:24.988554 (+   126us) write_op.cc:276] Timestamp: P: 1783602264988375 usec, L: 0
0709 13:04:24.988578 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:04:25.159204 (+170626us) log.cc:844] Serialized 3218 byte log entry
0709 13:04:25.817323 (+658119us) spinlock_profiling.cc:243] Waited 9.16 ms on lock 0x7b540013f954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c9ce28 00007fd7a56d7581 00007fd7a56d6d58 00007fd7a56db9f1 00007fd7a56db9a9 00007fd7a56db939 00007fd7a56db901 00007fd7a56dab2d 000000000038f354 000000000038f199
0709 13:04:25.998296 (+180973us) op_driver.cc:464] REPLICATION: finished
0709 13:04:26.000066 (+  1770us) write_op.cc:301] APPLY: starting
0709 13:04:26.000193 (+   127us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:26.011557 (+ 11364us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:26.011607 (+    50us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:26.197777 (+186170us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:26.198342 (+   565us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 13:04:26.198465 (+   123us) write_op.cc:312] APPLY: finished
0709 13:04:26.199230 (+   765us) log.cc:844] Serialized 81 byte log entry
0709 13:04:26.199539 (+   309us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:26.199998 (+   459us) write_op.cc:454] Released schema lock
0709 13:04:26.200106 (+   108us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:26.229251 (+ 29145us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:26.229290 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2561774,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":1402,"apply.run_cpu_time_us":18895,"apply.run_wall_time_us":228982,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37583,"num_ops":8,"prepare.queue_time_us":151453,"prepare.run_cpu_time_us":18295,"prepare.run_wall_time_us":886639,"raft.queue_time_us":246496,"raft.run_cpu_time_us":1400,"raft.run_wall_time_us":22059,"replication_time_us":1004671,"spinlock_wait_cycles":21060992,"thread_start_us":313378,"threads_started":4,"wal-append.queue_time_us":98553,"wal-append.run_cpu_time_us":31924,"wal-append.run_wall_time_us":1968253}]]}
I20260709 13:04:29.004142  1458 catalog_manager.cc:5697] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 17 to 18, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 18 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:28.986006  9126 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:29.002655  9252 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:29.008433  9174 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:29.016067  9174 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.017261  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6711207c14044518a57e7c8914ac0091" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:29.018491  9174 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:29.009009  9121 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:29.034965  9121 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:29.007262  1458 catalog_manager.cc:5697] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:29.035686  9121 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:29.022964  9211 raft_consensus.cc:2804] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:04:29.036784  9211 raft_consensus.cc:697] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:29.036422  1458 catalog_manager.cc:5697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 11 to 17, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 17 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:29.037762  9211 consensus_queue.cc:237] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:29.028946  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 47278.2us
I20260709 13:04:29.003557  9212 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.042963  1644 raft_consensus.cc:2468] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 15.
I20260709 13:04:29.050599  9212 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:29.051682  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:29.038741  1458 catalog_manager.cc:5697] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 20 to 22, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 22 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:29.058364  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:29.053081  1458 catalog_manager.cc:5697] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:29.061056  1458 catalog_manager.cc:5697] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 8 to 18. New cstate: current_term: 18 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:04:29.062429  1458 catalog_manager.cc:5697] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 13 to 15, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 15 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:04:29.064666  1458 catalog_manager.cc:5697] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
W20260709 13:04:29.073606  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 65615.9us
I20260709 13:04:29.082162  1715 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:04:29.087208  9214 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.089291  9214 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:29.091840  2296 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:04:29.050865  9174 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:29.096371  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:29.096990  2301 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Refusing update from remote peer 3f64b76180a54492bb015e405caeacec: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:04:29.100032  9174 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.101289  9212 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.102485  9174 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:29.099273  1534 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:29.104578  1534 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:29.095582  1458 catalog_manager.cc:5697] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec reported cstate change: config changed from index 14 to 17, VOTER 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) evicted. New cstate: current_term: 23 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:29.107945   431 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 1594)
I20260709 13:04:29.050028  1533 leader_election.cc:304] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:29.106113  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:29.119679  1458 catalog_manager.cc:5697] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec reported cstate change: term changed from 19 to 23, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 23 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:29.126994  9162 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:29.130112  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 80123.3us
I20260709 13:04:29.133173  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:29.133872  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:29.140406  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:29.141125  1644 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:29.153553  1644 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:04:29.112838  9203 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 13:04:29.137333  9251 raft_consensus.cc:2749] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:29.113826  9252 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:29.164369  1682 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:29.167196  9205 raft_consensus.cc:2804] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:04:29.167848  9205 raft_consensus.cc:697] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:29.168794  9205 consensus_queue.cc:237] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.178562  9162 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:29.181524  2301 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:04:29.182166  9214 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:29.183691  9126 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:29.186278  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:29.187026  1642 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:29.188843  1682 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:29.189952  9214 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:29.190402  9214 raft_consensus.cc:493] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.190802  9214 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:29.197906  1534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 61738.8us
I20260709 13:04:29.201722  1534 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:29.202761  9164 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:04:29.204393  9164 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:29.205751  9121 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.208791  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:29.209452  2296 raft_consensus.cc:3060] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:04:29.209671  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:29.210446  1641 raft_consensus.cc:2368] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17.
I20260709 13:04:29.212121  2269 leader_election.cc:400] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17.
I20260709 13:04:29.212477  2269 leader_election.cc:403] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:04:29.227862  2296 raft_consensus.cc:2468] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 17.
I20260709 13:04:29.229727  9183 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023) with higher term. Message: Invalid argument: T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 for earlier term 16. Current term is 17.
I20260709 13:04:29.235183  9212 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:29.225653  1534 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:29.255110  9071 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:29.254438  1534 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:29.256687  9252 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:29.347302  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:29.318652  9121 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:29.362561  9162 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.403456  9251 raft_consensus.cc:493] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:29.406427  9251 raft_consensus.cc:515] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.410646  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:29.404598  9231 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:29.387773  9162 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:29.392253  9214 raft_consensus.cc:515] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:29.441479  9162 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:29.444654  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6815b361554c338159f274bd78c0e9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:29.445519  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:29.444657  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:29.454687  9214 leader_election.cc:290] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:29.438781  9231 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.442699  9251 leader_election.cc:290] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:29.464936  1533 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:29.466042  9174 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:29.468551  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:29.471673  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520f39ecc282462782487a3651e28b44" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:29.472231  1642 raft_consensus.cc:3060] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:04:29.475662  1681 leader_election.cc:336] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:29.470597  9231 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:29.493587  9254 log.cc:927] Time spent T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: Append to log took a long time: real 0.103s	user 0.002s	sys 0.001s
I20260709 13:04:29.514676  1642 raft_consensus.cc:2468] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 21.
I20260709 13:04:29.515923  1682 leader_election.cc:304] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:29.520366  1534 leader_election.cc:304] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:29.524727  9162 raft_consensus.cc:2749] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:29.526042  9174 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.526707  9174 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.528874  9174 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:29.519984  9175 raft_consensus.cc:493] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:29.530628  9175 raft_consensus.cc:515] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.532974  9175 leader_election.cc:290] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:29.523815  9159 raft_consensus.cc:2804] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:04:29.534765  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:29.536516  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2792e3e29993414a8d6f55078ae13994" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:29.536676  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:29.540557  1534 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:29.544265  9159 raft_consensus.cc:697] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:29.545267  9159 consensus_queue.cc:237] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:29.546133  9175 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:29.551328  2302 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:04:29.554021  9214 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:29.561074  1641 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:29.565229  9214 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:04:29.568404  1681 leader_election.cc:336] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:29.573849  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:29.576290  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a8ba0cc212d41e5b0ad586b885d07a7" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:29.586714  1534 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:29.592732  9175 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:29.594818  1534 leader_election.cc:304] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:29.595690  1642 raft_consensus.cc:1275] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:04:29.596081  9175 raft_consensus.cc:2749] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:29.597084  9214 consensus_queue.cc:1048] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:29.621119  2302 raft_consensus.cc:1275] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:29.623818  9052 consensus_queue.cc:1048] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:04:29.629962  1681 consensus_peers.cc:597] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:29.656409  9175 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:29.657282  9175 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.660609  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:29.665184  9175 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:29.676609  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:29.679407  1534 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:29.682793  1681 consensus_peers.cc:597] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:29.684995  9251 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:29.716508  1681 consensus_peers.cc:597] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:29.717262  1681 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:29.714619  9251 raft_consensus.cc:493] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.719996  9251 raft_consensus.cc:515] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:29.722424  9251 leader_election.cc:290] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:29.723109  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:29.723433  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9310eb7a2c8447e28c9096c983ddf5b5" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:29.724920  1532 leader_election.cc:304] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:29.726140  9164 raft_consensus.cc:2749] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:29.768847  9164 raft_consensus.cc:493] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:29.769507  9164 raft_consensus.cc:515] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.772089  9164 leader_election.cc:290] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:29.773762  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:29.775331  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4594cfed297144b39e1a29473c9707d9" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:29.779287  1534 leader_election.cc:304] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:29.781657  9164 raft_consensus.cc:2749] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:29.820497  9262 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:29.821107  9262 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.823570  9262 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:29.825341  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:29.825549  9214 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:29.826292  1709 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 25: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:29.828118  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:29.828715  2299 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 25.
I20260709 13:04:29.826287  9214 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:29.830606  1609 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:29.832660  9262 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:29.833237  9262 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:29.833714  9262 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:29.836400  1681 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:04:29.839193  9214 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:29.840914  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:29.841784  1641 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 25: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:29.843390  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:29.846469  9159 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:29.847146  9262 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.864017  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:29.865716  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:29.866570  1715 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 25: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
W20260709 13:04:29.863934  1681 consensus_peers.cc:597] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:29.872589  2299 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 25.
I20260709 13:04:29.873901  1609 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:29.874786  9203 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:29.875347  9203 raft_consensus.cc:697] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 25 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:29.876214  9203 consensus_queue.cc:237] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.880834  9262 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 25 election: Requested vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:29.911234  9251 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:29.911840  9251 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.913980  9174 raft_consensus.cc:493] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:29.914552  9174 raft_consensus.cc:515] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:29.914896  9251 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:29.915673  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:29.916762  9174 leader_election.cc:290] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:29.920674  9174 raft_consensus.cc:493] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:29.922693  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:29.924057  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:29.924070  1534 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:29.920737  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c129438e9d58493a93890208b25765cf" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:29.878692  9159 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:29.925948  9175 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:04:29.921415  9174 raft_consensus.cc:515] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:29.930335  1532 leader_election.cc:304] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:29.936558  9164 raft_consensus.cc:2749] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:04:29.948478  1681 consensus_peers.cc:597] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:29.962512  9164 raft_consensus.cc:493] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:29.963164  9164 raft_consensus.cc:515] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:29.965693  9164 leader_election.cc:290] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:29.967852  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:29.969405  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:29.975888  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f86a4ba55242bd8d2459c6b9df56d2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:29.979538  1532 leader_election.cc:304] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:29.980834  9164 raft_consensus.cc:2749] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:29.991506  1681 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:29.993002  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb58673aa80747e58aa935f4fed2212d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:29.994823  1532 leader_election.cc:304] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:29.996021  9164 raft_consensus.cc:2749] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:29.988664  9174 leader_election.cc:290] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:30.009541  1681 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.031127  1607 consensus_peers.cc:597] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.049933  1644 raft_consensus.cc:1275] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:04:30.056655  9159 consensus_queue.cc:1048] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:04:30.069581  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57312
I20260709 13:04:30.070395  1709 raft_consensus.cc:606] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Received request to transfer leadership
I20260709 13:04:30.072861  9164 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:30.073720  9164 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:30.076843  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:30.077055  9164 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:30.079697  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:30.080895  1458 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45142 (request call id 354) took 8560 ms. Trace:
I20260709 13:04:30.081293  1534 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:30.082849  9164 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:30.086069  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57312
I20260709 13:04:30.086724  1709 raft_consensus.cc:606] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Received request to transfer leadership
W20260709 13:04:30.107215  1681 consensus_peers.cc:597] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.108415   431 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 1594)
I20260709 13:04:30.111506  1709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d"
dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57312
I20260709 13:04:30.112193  1709 raft_consensus.cc:606] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Received request to transfer leadership
I20260709 13:04:30.081429  1458 rpcz_store.cc:276] 0709 13:04:21.520056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.987896 (+3467840us) service_pool.cc:224] Handling call
0709 13:04:26.044205 (+1056309us) spinlock_profiling.cc:243] Waited 63.4 ms on lock 0x7b540013f954. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0b090 00007fd7a1ca9f3a 00007fd7a7a601cc 00007fd7a7b5e37e 00007fd7a601f2a4 00007fd7a601f232 00007fd7a601f161 00007fd7a601f0dc 00007fd7a601e392 00007fd79ffdc48c 00007fd79ffdb8c6
0709 13:04:30.080796 (+4036591us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:29.406295 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:29.406971 (+   676us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:29.407024 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:04:29.407043 (+    19us) tablet.cc:662] Decoding operations
0709 13:04:29.478820 (+ 71777us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:29.488897 (+ 10077us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:29.489382 (+   485us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:04:29.559607 (+ 70225us) tablet.cc:701] Row locks acquired
0709 13:04:29.559656 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:04:29.559810 (+   154us) write_op.cc:270] Start()
0709 13:04:29.562778 (+  2968us) write_op.cc:276] Timestamp: P: 1783602269559745 usec, L: 0
0709 13:04:29.562805 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:04:29.564995 (+  2190us) log.cc:844] Serialized 9197 byte log entry
0709 13:04:29.652810 (+ 87815us) op_driver.cc:464] REPLICATION: finished
0709 13:04:29.758157 (+105347us) write_op.cc:301] APPLY: starting
0709 13:04:29.758302 (+   145us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:29.842070 (+ 83768us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:29.842196 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:29.902323 (+ 60127us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:29.902862 (+   539us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 13:04:29.919190 (+ 16328us) write_op.cc:312] APPLY: finished
0709 13:04:29.983150 (+ 63960us) log.cc:844] Serialized 204 byte log entry
0709 13:04:29.988088 (+  4938us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:29.991592 (+  3504us) write_op.cc:454] Released schema lock
0709 13:04:29.991657 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:29.994978 (+  3321us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:29.995009 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1649309,"reported_tablets":38,"spinlock_wait_cycles":145753088,"child_traces":[["op",{"apply.queue_time_us":105024,"apply.run_cpu_time_us":32113,"apply.run_wall_time_us":236787,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100313,"num_ops":23,"prepare.queue_time_us":132087,"prepare.run_cpu_time_us":11555,"prepare.run_wall_time_us":182568,"raft.queue_time_us":15704,"raft.run_cpu_time_us":8631,"raft.run_wall_time_us":46071,"replication_time_us":89828,"spinlock_wait_cycles":146304,"thread_start_us":194502,"threads_started":4,"wal-append.queue_time_us":23764}]]}
W20260709 13:04:30.154814  1681 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.158890  2268 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.159704  9174 raft_consensus.cc:493] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:04:30.162873  1681 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:30.161721  9174 raft_consensus.cc:515] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:30.180514  9174 leader_election.cc:290] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:30.183281  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:30.183281  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3fdbd88a9d440e59ddd4e49c79ab2f2" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:30.189721   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:30.195382   431 tablet_replica.cc:333] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.196578   431 raft_consensus.cc:2243] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.197414   431 raft_consensus.cc:2272] T 4594cfed297144b39e1a29473c9707d9 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.202616   431 tablet_replica.cc:333] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.210690   431 raft_consensus.cc:2243] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.212020  1533 leader_election.cc:304] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:30.213912  1607 consensus_peers.cc:597] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.216185  1681 consensus_peers.cc:597] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.217901  1681 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.219753   431 pending_rounds.cc:70] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Trying to abort 1 pending ops.
I20260709 13:04:30.218313  9173 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.230208  9173 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:30.232700  9173 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:30.235999  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:30.235625  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:30.237377  1709 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 25.
I20260709 13:04:30.217830  9174 raft_consensus.cc:2749] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:30.240262  1532 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:30.236706  1644 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 25.
I20260709 13:04:30.242307  9174 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:04:30.242982  9174 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.244128  9174 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:30.219986   431 pending_rounds.cc:77] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec: Aborting op as it isn't in flight: id { term: 24 index: 17 } timestamp: 7305634869979926528 op_type: NO_OP noop_request { }
I20260709 13:04:30.246240   431 raft_consensus.cc:2889] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:30.246622   431 raft_consensus.cc:2272] T b7808a4bebf84e598198ed51ea156981 P 3f64b76180a54492bb015e405caeacec [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.249828   431 tablet_replica.cc:333] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.250664   431 raft_consensus.cc:2243] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.256709  9173 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:30.265625  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet fb294ac919b44e56960745f79e83dfc4 with delete_type TABLET_DATA_TOMBSTONED (TS 5184c101be204393adf3ac8b31f25ad6 not found in new config with opid_index 17) from {username='slave'} at 127.0.0.1:40842
I20260709 13:04:30.270463  9173 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:30.275339  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:30.287685  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:30.305052  1532 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:30.305550   431 raft_consensus.cc:2272] T bb58673aa80747e58aa935f4fed2212d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.306710  9164 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:30.303639  9175 raft_consensus.cc:493] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.303797  9173 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:30.382263  8449 raft_consensus.cc:993] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: : Instructing follower 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:04:30.383049  8449 raft_consensus.cc:1081] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Signalling peer 5184c101be204393adf3ac8b31f25ad6 to start an election
I20260709 13:04:30.378613  9164 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.384826  9164 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:30.388489  9164 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:30.372857  9175 raft_consensus.cc:515] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:30.417454  9284 tablet_replica.cc:333] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:30.419130  9284 raft_consensus.cc:2243] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.420373  9284 raft_consensus.cc:2272] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.437311  9175 leader_election.cc:290] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:30.438799  9174 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:30.440016   431 tablet_replica.cc:333] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.454064   431 raft_consensus.cc:2243] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.455117   431 raft_consensus.cc:2272] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:30.439898  1607 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 225 similar messages]
I20260709 13:04:30.461097   431 tablet_replica.cc:333] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.462029  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:30.466658  9284 ts_tablet_manager.cc:1916] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:30.471215  9174 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 26 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:30.472816  2268 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.480414  1607 consensus_peers.cc:597] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.482111  1533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 108357us
I20260709 13:04:30.484803  2302 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:04:30.486296  9282 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:30.487036   431 raft_consensus.cc:2243] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.492709  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:30.493834   431 raft_consensus.cc:2272] T 408b4ec829c54266b41334aac8245405 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.497159   431 tablet_replica.cc:333] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.498324  1642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d"
dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
 from {username='slave'} at 127.0.0.1:60390
I20260709 13:04:30.526356  1642 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:30.526736  1642 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:04:30.527814  1515 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:04:30.528259  1515 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for ca83cd5e36eb4e22b75011b87bfcd49d already in progress
W20260709 13:04:30.528529  1515 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for ca83cd5e36eb4e22b75011b87bfcd49d already in progress
I20260709 13:04:30.501010  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe2946ff23d498f98c4627ee968a4e3" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:30.502677  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:30.506165  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:30.537412  1712 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:04:30.507246  1607 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.508745  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:30.538991  1641 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:04:30.519691  1710 raft_consensus.cc:1275] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:04:30.544813  1712 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 26.
I20260709 13:04:30.546324  1641 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f64b76180a54492bb015e405caeacec in term 26.
I20260709 13:04:30.547253  1532 leader_election.cc:304] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:30.548213  9164 raft_consensus.cc:2749] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:30.549185  1532 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec; no voters: 
I20260709 13:04:30.549080  9283 consensus_queue.cc:1048] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:30.550207   431 raft_consensus.cc:2243] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.550149  9164 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 26 FOLLOWER]: Leader election won for term 26
W20260709 13:04:30.549207  1607 consensus_peers.cc:597] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.550961  9164 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 26 LEADER]: Becoming Leader. State: Replica: 3f64b76180a54492bb015e405caeacec, State: Running, Role: LEADER
I20260709 13:04:30.552040  9164 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:30.562611  2268 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.556171   431 raft_consensus.cc:2272] T 9310eb7a2c8447e28c9096c983ddf5b5 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.564467  1533 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:30.566689  2268 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.567680  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.549141  1642 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:30.573105  9293 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:30.579828  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:30.580066  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:30.580827  1709 raft_consensus.cc:3055] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:04:30.580899  2299 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:30.581310  1709 raft_consensus.cc:740] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:30.582238  1709 consensus_queue.cc:260] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:30.584101  1709 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:30.590883   431 tablet_replica.cc:333] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.591969   431 raft_consensus.cc:2243] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.592739   431 raft_consensus.cc:2272] T d1256ff04335423286d7c081ec776061 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.599077  1642 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:30.606492  2299 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:04:30.619561  1709 raft_consensus.cc:2468] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
W20260709 13:04:30.629458  1607 consensus_peers.cc:597] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.631507  1607 leader_election.cc:336] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:30.634034  2268 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.636549  1681 consensus_peers.cc:597] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.641549  1681 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.645346  2268 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:30.649915  1461 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45164 (request call id 365) took 77921 ms (client timeout 15000 ms). Trace:
I20260709 13:04:30.649755  1607 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:30.651284  9283 raft_consensus.cc:2804] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:04:30.651800  9283 raft_consensus.cc:697] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:30.652773  9283 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:30.656101  1461 rpcz_store.cc:269] 0709 13:03:12.728784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.729114 (+   330us) service_pool.cc:224] Handling call
0709 13:04:30.649803 (+77920689us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18195,"reported_tablets":47}
I20260709 13:04:30.629204   431 tablet_replica.cc:333] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.659821   431 raft_consensus.cc:2243] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:04:30.661980   431 raft_consensus.cc:2272] T 3905c76f9588440c8a836e5786f6667e P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.665179   431 tablet_replica.cc:333] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.668212   431 raft_consensus.cc:2243] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.668919   431 raft_consensus.cc:2272] T d9ba36df4f014c3a916b0393305ea44a P 3f64b76180a54492bb015e405caeacec [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:30.676615  2268 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.696414  9284 ts_tablet_manager.cc:1929] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 23.16
I20260709 13:04:30.697118  9284 log.cc:1199] T fb294ac919b44e56960745f79e83dfc4 P 5184c101be204393adf3ac8b31f25ad6: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-1-root/wals/fb294ac919b44e56960745f79e83dfc4
I20260709 13:04:30.699635  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet fb294ac919b44e56960745f79e83dfc4 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:04:30.708968   431 tablet_replica.cc:333] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.709956   431 raft_consensus.cc:2243] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:30.716921   431 raft_consensus.cc:2272] T 33657ac7590d4739b9bda3f7bb345c7b P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:30.719085  1463 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45150 (request call id 328) took 57540 ms (client timeout 15000 ms). Trace:
W20260709 13:04:30.719674  1463 rpcz_store.cc:269] 0709 13:03:33.144032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:33.144378 (+   346us) service_pool.cc:224] Handling call
0709 13:04:30.682237 (+57537859us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":336657,"reported_tablets":46}
I20260709 13:04:30.735981   431 tablet_replica.cc:333] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.736960   431 raft_consensus.cc:2243] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.737735   431 raft_consensus.cc:2272] T 2792e3e29993414a8d6f55078ae13994 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:30.745977  2268 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:30.748152   431 tablet_replica.cc:333] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.750782   431 raft_consensus.cc:2243] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:04:30.756724  1464 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45164 (request call id 364) took 105497 ms (client timeout 15000 ms). Trace:
W20260709 13:04:30.757194  1464 rpcz_store.cc:269] 0709 13:02:45.259507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.287760 (+ 28253us) service_pool.cc:224] Handling call
0709 13:04:30.756426 (+105468666us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33250,"reported_tablets":47}
I20260709 13:04:30.760317   431 raft_consensus.cc:2272] T 71b0b66754d049cfbccb38fbb16ef3cd P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.763049   431 tablet_replica.cc:333] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.763950   431 raft_consensus.cc:2243] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:04:30.764885   431 raft_consensus.cc:2272] T 213d9f9af66a4ebd8c9a7b0163196a57 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.767491   431 tablet_replica.cc:333] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.768381   431 raft_consensus.cc:2243] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:04:30.769521   431 raft_consensus.cc:2272] T 4c0f61db23f34217bd5089a459cbb914 P 3f64b76180a54492bb015e405caeacec [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.772575   431 tablet_replica.cc:333] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.773504   431 raft_consensus.cc:2243] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.774132   431 raft_consensus.cc:2272] T c129438e9d58493a93890208b25765cf P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.779294   431 tablet_replica.cc:333] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.780189   431 raft_consensus.cc:2243] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:04:30.781458   431 raft_consensus.cc:2272] T 768b897f18a644e8b1e8f8ed1c8b616f P 3f64b76180a54492bb015e405caeacec [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.793423  9293 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:30.793969   431 tablet_replica.cc:333] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.794090  9293 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:30.795253   431 raft_consensus.cc:2243] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:30.796375   431 raft_consensus.cc:2272] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.797433  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:30.799285   431 tablet_replica.cc:333] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.800203   431 raft_consensus.cc:2243] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.800864   431 raft_consensus.cc:2272] T f712f9c188e44e588e739161cac39330 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.803637  9293 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:30.806006  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:30.806615  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:30.808192  1532 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:30.810463  9293 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:30.812505   431 tablet_replica.cc:333] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.813545   431 raft_consensus.cc:2243] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.814294   431 raft_consensus.cc:2272] T 3f22f7f5cede49bea2083e8905932fb7 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.825508   431 tablet_replica.cc:333] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.826421   431 raft_consensus.cc:2243] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:30.827663   431 raft_consensus.cc:2272] T c46cf090bf9b4084b648d00f5f0053e2 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.830675   431 tablet_replica.cc:333] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.831540   431 raft_consensus.cc:2243] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.832165   431 pending_rounds.cc:70] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Trying to abort 1 pending ops.
I20260709 13:04:30.832408   431 pending_rounds.cc:77] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec: Aborting op as it isn't in flight: id { term: 21 index: 16 } timestamp: 7305634863311605760 op_type: NO_OP noop_request { }
I20260709 13:04:30.832811   431 raft_consensus.cc:2889] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:30.833308   431 raft_consensus.cc:2272] T d4bc96896acb4fcf8b5ec5b7206248ca P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.842561   431 tablet_replica.cc:333] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.847528   431 raft_consensus.cc:2243] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.848275   431 raft_consensus.cc:2272] T 84f86a4ba55242bd8d2459c6b9df56d2 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.852001   431 tablet_replica.cc:333] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.855754   431 raft_consensus.cc:2243] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.856405   431 raft_consensus.cc:2272] T e62e41b6e9a54f26a1c7c1538bdf5334 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.867249   431 tablet_replica.cc:333] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.868191   431 raft_consensus.cc:2243] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:04:30.869051   431 pending_rounds.cc:70] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Trying to abort 1 pending ops.
I20260709 13:04:30.869299   431 pending_rounds.cc:77] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec: Aborting op as it isn't in flight: id { term: 26 index: 14 } timestamp: 7305634900190507008 op_type: NO_OP noop_request { }
I20260709 13:04:30.869673   431 raft_consensus.cc:2889] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 26 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:30.870036   431 raft_consensus.cc:2272] T 2b9b7b078d0d47d3880d3dec3230b779 P 3f64b76180a54492bb015e405caeacec [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.873670   431 tablet_replica.cc:333] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.874560   431 raft_consensus.cc:2243] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.875200   431 raft_consensus.cc:2272] T 5a08948b1fac416d8d27be0561a00db4 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.878034   431 tablet_replica.cc:333] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.878858   431 raft_consensus.cc:2243] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.887732   431 raft_consensus.cc:2272] T 403bcba338a042c1b5fecfe8ac430922 P 3f64b76180a54492bb015e405caeacec [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.890645   431 tablet_replica.cc:333] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.891798   431 raft_consensus.cc:2243] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:30.893085   431 raft_consensus.cc:2272] T fb294ac919b44e56960745f79e83dfc4 P 3f64b76180a54492bb015e405caeacec [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.893312  9293 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.908703   431 tablet_replica.cc:333] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.908442  9293 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:30.909794   431 raft_consensus.cc:2243] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:04:30.910897  9293 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:30.915225  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:30.915980  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:30.926606  1533 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:30.927917  9293 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:30.915042   431 raft_consensus.cc:2272] T c2832827f30046328715699fa6ee0c52 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.932178   431 tablet_replica.cc:333] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.933257   431 raft_consensus.cc:2243] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.934317   431 raft_consensus.cc:2272] T 3fe2946ff23d498f98c4627ee968a4e3 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.960278   431 tablet_replica.cc:333] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.961285   431 raft_consensus.cc:2243] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.961699  9293 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:30.962191   431 raft_consensus.cc:2272] T 1b7e7c3881a44475827c9cbca08a2fe1 P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.962365  9293 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:30.965692  9293 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:30.965978   431 tablet_replica.cc:333] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.967331   431 raft_consensus.cc:2243] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.968477  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:30.969437  1715 raft_consensus.cc:2393] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f64b76180a54492bb015e405caeacec in current term 18: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:30.969295  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:30.971318   431 raft_consensus.cc:2272] T c5eae102421c4bc4b19781bf4470a0b0 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.971841  1533 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3f64b76180a54492bb015e405caeacec; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:30.970628  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "3f64b76180a54492bb015e405caeacec" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:30.973634  9293 raft_consensus.cc:3060] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:04:30.976614   431 tablet_replica.cc:333] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.977797   431 raft_consensus.cc:2243] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.978685   431 raft_consensus.cc:2272] T 70a0a5f8de344870be97d83cec17c047 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.982425   431 tablet_replica.cc:333] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:30.983794   431 raft_consensus.cc:2243] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:30.984732   431 raft_consensus.cc:2272] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:30.988386   431 tablet_replica.cc:333] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:31.000355  9293 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:31.007427   431 raft_consensus.cc:2243] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:31.011876  1465 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45164 (request call id 366) took 48569 ms (client timeout 15000 ms). Trace:
W20260709 13:04:31.023473  1465 rpcz_store.cc:269] 0709 13:03:42.442753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:42.459136 (+ 16383us) service_pool.cc:224] Handling call
0709 13:03:43.092234 (+633098us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b540029ff90. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd79ea3cdb1 00007fd79eb7cb71 00007fd79eb7d0b2 00007fd7a7af6a89 00007fd7a7a8eda6 00007fd7a7a60088 00007fd7a7b5e37e 00007fd7a601f2a4 00007fd7a601f232 00007fd7a601f161 00007fd7a601f0dc
0709 13:04:31.011667 (+47919433us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":666471,"reported_tablets":47,"spinlock_wait_cycles":70047616}
I20260709 13:04:31.025530   431 raft_consensus.cc:2272] T ca83cd5e36eb4e22b75011b87bfcd49d P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.028831   431 tablet_replica.cc:333] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:31.029887   431 raft_consensus.cc:2243] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.030521   431 raft_consensus.cc:2272] T 6711207c14044518a57e7c8914ac0091 P 3f64b76180a54492bb015e405caeacec [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.040899   431 tablet_replica.cc:333] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:31.041793   431 raft_consensus.cc:2243] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.050665   431 raft_consensus.cc:2272] T 8c85446f87564ae6865060bb988ee1b8 P 3f64b76180a54492bb015e405caeacec [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.053656   431 tablet_replica.cc:333] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:31.054443   431 raft_consensus.cc:2243] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.055562   431 raft_consensus.cc:2272] T 6bdefe70d276420ca0283414ab16fab9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.059114   431 tablet_replica.cc:333] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:31.096835   431 raft_consensus.cc:2243] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.098151   431 raft_consensus.cc:2272] T cdf54b70d44944ef9013c4adb0970687 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.116307  2302 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:31.119428  9283 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:31.131980   431 tablet_replica.cc:333] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:31.132972   431 raft_consensus.cc:2243] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.133648   431 raft_consensus.cc:2272] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.136526   431 tablet_replica.cc:333] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:31.137341   431 raft_consensus.cc:2243] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.140268   431 raft_consensus.cc:2272] T 9f6815b361554c338159f274bd78c0e9 P 3f64b76180a54492bb015e405caeacec [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:31.150305  1466 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45150 (request call id 327) took 83536 ms (client timeout 15000 ms). Trace:
W20260709 13:04:31.152029  1466 rpcz_store.cc:269] 0709 13:03:07.613712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:07.614018 (+   306us) service_pool.cc:224] Handling call
0709 13:04:31.150090 (+83536072us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100587,"reported_tablets":46}
I20260709 13:04:31.159629   431 tablet_replica.cc:333] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:31.166258   431 raft_consensus.cc:2243] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.166997   431 raft_consensus.cc:2272] T 8a4760a45328493e87523c2ee8292599 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.170049  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet fb294ac919b44e56960745f79e83dfc4 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 17)) from {username='slave'} at 127.0.0.1:40842
I20260709 13:04:31.170364   431 tablet_replica.cc:333] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:31.179570   431 raft_consensus.cc:2243] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.179615  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet fb294ac919b44e56960745f79e83dfc4 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:04:31.180433   431 raft_consensus.cc:2272] T d29864f650d0477e8dd2e7f82eea9e47 P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.183893   431 tablet_replica.cc:333] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:31.184708   431 raft_consensus.cc:2243] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.185439   431 pending_rounds.cc:70] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: Trying to abort 1 pending ops.
I20260709 13:04:31.185703   431 pending_rounds.cc:77] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec: Aborting op as it isn't in flight: id { term: 19 index: 12 } timestamp: 7305634861214507008 op_type: NO_OP noop_request { }
I20260709 13:04:31.186087   431 raft_consensus.cc:2889] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:31.186496   431 raft_consensus.cc:2272] T 520f39ecc282462782487a3651e28b44 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.207942   431 tablet_replica.cc:333] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:31.208765   431 raft_consensus.cc:2243] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.210150   431 raft_consensus.cc:2272] T 82743dc3081f4edba18591ca4b0d9c09 P 3f64b76180a54492bb015e405caeacec [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.213351   431 tablet_replica.cc:333] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:31.214219   431 raft_consensus.cc:2243] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.235018   431 raft_consensus.cc:2272] T 37b565cd27a64e96addb09389c2c7d5e P 3f64b76180a54492bb015e405caeacec [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.237964   431 tablet_replica.cc:333] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec: stopping tablet replica
I20260709 13:04:31.238668   431 raft_consensus.cc:2243] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.239328   431 raft_consensus.cc:2272] T d59c4d21b6cb46edbfc2b3d50b51812a P 3f64b76180a54492bb015e405caeacec [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:31.251333  1607 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:31.259157  1715 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:04:31.261482  9262 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:04:31.366636  2268 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:31.407582  1459 catalog_manager.cc:5697] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 16 to 19, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 19 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:31.411123  1459 catalog_manager.cc:5697] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 11 to 12, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 12 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:31.434044  1467 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60118 (request call id 230) took 115763 ms (client timeout 15000 ms). Trace:
W20260709 13:04:31.441807  1467 rpcz_store.cc:269] 0709 13:02:35.686560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:35.728681 (+ 42121us) service_pool.cc:224] Handling call
0709 13:04:31.433803 (+115705122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56564,"reported_tablets":42}
W20260709 13:04:31.618178  1681 connection.cc:573] client connection to 127.0.107.193:36547 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:04:31.621829  1607 connection.cc:573] client connection to 127.0.107.193:36547 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:04:31.622557  2268 connection.cc:573] client connection to 127.0.107.193:36547 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 13:04:31.694228  1460 catalog_manager.cc:5697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } }
I20260709 13:04:31.696503  1460 catalog_manager.cc:5697] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 18 to 19, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 19 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:31.698278  1459 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45164 (request call id 367) took 21126 ms (client timeout 15000 ms). Trace:
I20260709 13:04:31.707556  1460 catalog_manager.cc:5697] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:31.709100  1460 catalog_manager.cc:5697] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:31.722433  1460 catalog_manager.cc:5697] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:04:31.730396  1460 catalog_manager.cc:5697] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 22 to 23, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 23 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:31.732820  1460 catalog_manager.cc:5697] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 18 to 20, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 20 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
W20260709 13:04:31.704691  1459 rpcz_store.cc:269] 0709 13:04:10.571467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.379091 (+807624us) service_pool.cc:224] Handling call
0709 13:04:31.698191 (+20319100us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:31.441262 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:31.443275 (+  2013us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:31.443319 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:04:31.443336 (+    17us) tablet.cc:662] Decoding operations
0709 13:04:31.443753 (+   417us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:31.443833 (+    80us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:31.443858 (+    25us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:04:31.444157 (+   299us) tablet.cc:701] Row locks acquired
0709 13:04:31.444174 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:04:31.444310 (+   136us) write_op.cc:270] Start()
0709 13:04:31.444444 (+   134us) write_op.cc:276] Timestamp: P: 1783602271444256 usec, L: 0
0709 13:04:31.444466 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:04:31.447087 (+  2621us) log.cc:844] Serialized 919 byte log entry
0709 13:04:31.491650 (+ 44563us) op_driver.cc:464] REPLICATION: finished
0709 13:04:31.557323 (+ 65673us) write_op.cc:301] APPLY: starting
0709 13:04:31.557410 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:31.559199 (+  1789us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:31.615576 (+ 56377us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:31.646354 (+ 30778us) spinlock_profiling.cc:243] Waited 886 us on lock 0x7b180034ab00. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd7a19e00ec 00007fd7a1a2cc0a 00007fd7a1a2c4f7 00007fd7a1a2c490 00007fd7a19e3f48 00007fd7a56f81bb 00007fd7a573f5d4 00007fd7a55eb8da 00007fd7a55ee3cb
0709 13:04:31.647936 (+  1582us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:31.648129 (+   193us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:04:31.648176 (+    47us) write_op.cc:312] APPLY: finished
0709 13:04:31.648392 (+   216us) log.cc:844] Serialized 33 byte log entry
0709 13:04:31.649219 (+   827us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:31.649313 (+    94us) write_op.cc:454] Released schema lock
0709 13:04:31.649359 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:31.690972 (+ 41613us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:31.691016 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12761,"reported_tablets":47,"spinlock_wait_cycles":61312,"child_traces":[["op",{"apply.queue_time_us":65413,"apply.run_cpu_time_us":8953,"apply.run_wall_time_us":111984,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12491,"num_ops":2,"prepare.queue_time_us":25072,"prepare.run_cpu_time_us":15329,"prepare.run_wall_time_us":35790,"raft.queue_time_us":12570,"raft.run_cpu_time_us":20491,"raft.run_wall_time_us":115649,"replication_time_us":47019,"spinlock_wait_cycles":2066560,"thread_start_us":129901,"threads_started":4,"wal-append.queue_time_us":29786}]]}
I20260709 13:04:31.734495  1460 catalog_manager.cc:5697] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 19 to 20, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 20 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:31.767833  1460 catalog_manager.cc:5697] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:31.770165  9282 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:31.769984  1460 catalog_manager.cc:5697] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:31.770797  9282 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:31.773322  9282 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:31.777120  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:31.782660  9262 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:31.783633  9262 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:31.785916  9285 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:04:31.788400  1515 auto_rebalancer.cc:629] Failed to schedule move for tablet 3905c76f9588440c8a836e5786f6667e: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:31.788630  9285 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:31.791257  1607 leader_election.cc:336] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:31.791827  1607 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:31.787544  9262 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:31.793556  9285 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:31.799175  9282 raft_consensus.cc:2749] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:31.826534  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:31.836133  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:31.842407  1607 leader_election.cc:336] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:31.859056  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:31.862466  2268 leader_election.cc:336] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:31.863430  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:31.864329  9291 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:04:31.865996  9282 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:31.873198  9282 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:31.873890  9282 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:31.876571  9282 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:31.882752  1607 leader_election.cc:336] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:31.882440  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:31.885255  1607 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:31.886225  9282 raft_consensus.cc:2749] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:04:31.915040  1462 catalog_manager.cc:5697] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:31.920373  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet fb294ac919b44e56960745f79e83dfc4 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 14 (current committed config index is 17)) from {username='slave'} at 127.0.0.1:40842
I20260709 13:04:31.927567  1462 catalog_manager.cc:5697] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 21 to 22, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 22 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:31.931824  9291 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:31.933292  9291 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:31.935745  9291 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:31.938172  1515 auto_rebalancer.cc:629] Failed to schedule move for tablet 2b9b7b078d0d47d3880d3dec3230b779: Illegal state: Replica 5184c101be204393adf3ac8b31f25ad6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 26 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:04:31.940182  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:31.932521  1460 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45150 (request call id 330) took 18301 ms (client timeout 15000 ms). Trace:
I20260709 13:04:31.941941  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet fb294ac919b44e56960745f79e83dfc4 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:04:31.942129  1462 catalog_manager.cc:5697] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 10 to 19, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:31.947149  1462 catalog_manager.cc:5697] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 17 to 21, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 21 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:31.956903  9285 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:04:31.941457  1460 rpcz_store.cc:269] 0709 13:04:13.631428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.257169 (+3625741us) service_pool.cc:224] Handling call
0709 13:04:31.932424 (+14675255us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:31.802266 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:31.802874 (+   608us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:31.802906 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:04:31.802969 (+    63us) tablet.cc:662] Decoding operations
0709 13:04:31.804021 (+  1052us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:31.804076 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:31.804112 (+    36us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:04:31.804945 (+   833us) tablet.cc:701] Row locks acquired
0709 13:04:31.804961 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:04:31.805115 (+   154us) write_op.cc:270] Start()
0709 13:04:31.805226 (+   111us) write_op.cc:276] Timestamp: P: 1783602271805065 usec, L: 0
0709 13:04:31.805245 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:04:31.805889 (+   644us) log.cc:844] Serialized 4000 byte log entry
0709 13:04:31.808778 (+  2889us) op_driver.cc:464] REPLICATION: finished
0709 13:04:31.809285 (+   507us) write_op.cc:301] APPLY: starting
0709 13:04:31.809372 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:31.815803 (+  6431us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:31.815908 (+   105us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:31.822104 (+  6196us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:31.822346 (+   242us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:04:31.822424 (+    78us) write_op.cc:312] APPLY: finished
0709 13:04:31.822774 (+   350us) log.cc:844] Serialized 97 byte log entry
0709 13:04:31.843165 (+ 20391us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:31.843297 (+   132us) write_op.cc:454] Released schema lock
0709 13:04:31.843356 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:31.851702 (+  8346us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:31.851750 (+    48us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6355,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":279,"apply.run_cpu_time_us":14084,"apply.run_wall_time_us":42346,"cfile_cache_hit":40,"cfile_cache_hit_bytes":41820,"num_ops":10,"prepare.queue_time_us":111,"prepare.run_cpu_time_us":3933,"prepare.run_wall_time_us":12810,"replication_time_us":3370}]]}
I20260709 13:04:31.957752  9285 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:31.960314  9285 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:31.961704  1462 catalog_manager.cc:5697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:31.962386  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:31.964851  1462 catalog_manager.cc:5697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:31.946058  1715 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:04:31.972069  1462 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 15 to 19, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:31.994032  9289 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:31.995007  9289 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:31.997565  9289 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:32.001986  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:32.012984  8449 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.013783  8449 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.017407  8449 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:32.018373  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:32.019219  2302 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 12.
I20260709 13:04:32.020924  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:32.028779  9292 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:04:32.029336  9292 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.029778  9292 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:04:32.031467  9052 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.032181  9052 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.041414  9052 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:32.048416  9262 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.049206  9262 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.054003  9262 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:32.059263  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:32.060097  1641 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 27.
I20260709 13:04:32.060829  1461 catalog_manager.cc:5697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } }
I20260709 13:04:32.062793  1461 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 19 to 20, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.065539  1461 catalog_manager.cc:5697] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 19 to 21, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 21 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.071781  9052 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.059454  1462 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60118 (request call id 235) took 8573 ms. Trace:
I20260709 13:04:32.115419  9289 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.113229  1462 rpcz_store.cc:276] 0709 13:04:23.485910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.004883 (+5518973us) service_pool.cc:224] Handling call
0709 13:04:32.059348 (+3054465us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:31.987760 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:31.988303 (+   543us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:31.988343 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:04:31.988357 (+    14us) tablet.cc:662] Decoding operations
0709 13:04:31.989145 (+   788us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:31.989203 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:31.989229 (+    26us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:04:31.989801 (+   572us) tablet.cc:701] Row locks acquired
0709 13:04:31.989815 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:04:31.989954 (+   139us) write_op.cc:270] Start()
0709 13:04:31.990081 (+   127us) write_op.cc:276] Timestamp: P: 1783602271989913 usec, L: 0
0709 13:04:31.990100 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:04:31.990704 (+   604us) log.cc:844] Serialized 2835 byte log entry
0709 13:04:31.993071 (+  2367us) op_driver.cc:464] REPLICATION: finished
0709 13:04:31.993610 (+   539us) write_op.cc:301] APPLY: starting
0709 13:04:31.993704 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:31.998071 (+  4367us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:31.998179 (+   108us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:32.003688 (+  5509us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:32.003894 (+   206us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:04:32.003940 (+    46us) write_op.cc:312] APPLY: finished
0709 13:04:32.004232 (+   292us) log.cc:844] Serialized 73 byte log entry
0709 13:04:32.005143 (+   911us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:32.005339 (+   196us) write_op.cc:454] Released schema lock
0709 13:04:32.005392 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:32.005803 (+   411us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:32.005836 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":305,"apply.run_cpu_time_us":10941,"apply.run_wall_time_us":12125,"cfile_cache_hit":28,"cfile_cache_hit_bytes":29274,"num_ops":7,"prepare.queue_time_us":9115,"prepare.run_cpu_time_us":3282,"prepare.run_wall_time_us":11468,"replication_time_us":2810}]]}
I20260709 13:04:32.075865  1461 catalog_manager.cc:5697] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 21 to 22, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 22 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:04:32.116184  9289 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.118198  1461 catalog_manager.cc:5697] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 19 to 20, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.120842  1461 catalog_manager.cc:5697] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 20 to 23, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 23 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:32.124054  1461 catalog_manager.cc:5697] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 23 to 27, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 27 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:04:32.127630  1461 catalog_manager.cc:5697] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 20 to 24, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 24 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.113276  9052 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:32.143435  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 160077us
I20260709 13:04:32.145257  9052 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:32.140552  9285 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.149308  1461 catalog_manager.cc:5697] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.151189  9285 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.152441  1461 catalog_manager.cc:5697] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 14 to 16, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 16 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.153754  9285 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:32.154511  1461 catalog_manager.cc:5697] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.156955  1461 catalog_manager.cc:5697] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 21 to 22, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 22 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.172588  9262 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.172032  1461 catalog_manager.cc:5697] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.173872  9262 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:32.085757  1681 leader_election.cc:336] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.178220  1681 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:32.179451  9052 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:32.181211  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 95446.4us
W20260709 13:04:32.181901  1681 leader_election.cc:336] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:32.182981  1681 leader_election.cc:336] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.186640  9292 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.189738  9292 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:32.198549  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 141042us
I20260709 13:04:32.203789  9320 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.238777  9320 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.204231  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:32.205987  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:32.207979  9262 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:32.211130  9291 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.201157  9282 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
W20260709 13:04:32.212893  2268 leader_election.cc:336] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.246282  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:32.213346  9289 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:32.215225  9319 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.246587  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:32.214783  2299 consensus_queue.cc:237] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.220625  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:32.248231  2304 raft_consensus.cc:3060] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:04:32.231351  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 46703.8us
I20260709 13:04:32.234364  8449 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.244680  9320 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:32.245306  1607 leader_election.cc:336] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.245234  9291 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.249359  8449 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.260568  2296 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:32.248417  9282 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.273756  8449 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:32.274528  8449 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:04:32.274943  8449 raft_consensus.cc:493] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.275352  8449 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:32.280444  9291 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:32.251204  9319 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.270740  9282 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:32.287281  9319 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:32.289732  9319 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.290270  9319 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:32.296000  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 53049.6us
I20260709 13:04:32.299695  8449 raft_consensus.cc:515] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.298764  2304 raft_consensus.cc:2468] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 13.
W20260709 13:04:32.306551  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 67644us
I20260709 13:04:32.318188  8449 leader_election.cc:290] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 28 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:32.302613  9291 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:32.321471  1683 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:32.325690  8449 raft_consensus.cc:2804] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:04:32.326479  8449 raft_consensus.cc:697] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:32.333806  8449 consensus_queue.cc:237] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.340771  9319 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:32.343462  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 86262.2us
I20260709 13:04:32.371192  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:32.372292  2304 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
W20260709 13:04:32.373942  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 127690us
I20260709 13:04:32.382040  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:32.382975  1636 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:04:32.408597  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:32.409545  1636 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 22.
I20260709 13:04:32.421929  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0f61db23f34217bd5089a459cbb914" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:32.422611  1641 raft_consensus.cc:3060] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:32.428730  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:32.433727  1636 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 22.
W20260709 13:04:32.438129  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 116686us
I20260709 13:04:32.444504  1641 raft_consensus.cc:2468] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 28.
W20260709 13:04:32.453027  2268 leader_election.cc:336] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.453555  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:32.454391  1715 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 18.
I20260709 13:04:32.455524  1609 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:32.456390  9322 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:32.457121  9322 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.457502  9322 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:32.456377  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:32.460016  1715 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
W20260709 13:04:32.454622  2268 leader_election.cc:336] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.460906  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:32.465801  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 214881us
I20260709 13:04:32.441519  9317 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.473971  9338 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.483023  9317 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.487805  9317 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:32.488379  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:32.489089  1709 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 18.
I20260709 13:04:32.490881  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:32.491760  1709 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 15.
I20260709 13:04:32.496766  9291 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.500324  9333 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:32.500473  9291 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.507499  1636 raft_consensus.cc:1275] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:04:32.515616  9285 consensus_queue.cc:1048] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:32.483141  9338 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.521003  9338 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:32.521847  9291 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:32.525696  9322 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.539834  9322 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:32.560279  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 105622us
I20260709 13:04:32.561424  1682 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:32.563488  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:32.567724  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:32.568347  2296 raft_consensus.cc:3060] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:32.568974  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:32.569299  1683 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:32.563990  1458 catalog_manager.cc:5697] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 19 to 20, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 20 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.571485  1682 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:32.572007  1458 catalog_manager.cc:5697] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.576079  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:32.576671  1458 catalog_manager.cc:5697] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.578539  1458 catalog_manager.cc:5697] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
I20260709 13:04:32.579792  1682 leader_election.cc:304] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:32.582386  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:32.582878  1681 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:32.583284  1709 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 23.
I20260709 13:04:32.583774  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:32.581843  1458 catalog_manager.cc:5697] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 17 to 19, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.587067  1458 catalog_manager.cc:5697] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.598879  1461 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45150 (request call id 331) took 3661 ms. Trace:
I20260709 13:04:32.575500  9341 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:04:32.601635  9341 raft_consensus.cc:493] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.602056  9341 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:32.599375  1461 rpcz_store.cc:276] 0709 13:04:28.937400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.658522 (+1721122us) service_pool.cc:224] Handling call
0709 13:04:32.598787 (+1940265us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:32.180172 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:32.181097 (+   925us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:32.181136 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:04:32.181149 (+    13us) tablet.cc:662] Decoding operations
0709 13:04:32.182394 (+  1245us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:32.182464 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:32.182494 (+    30us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:04:32.183879 (+  1385us) tablet.cc:701] Row locks acquired
0709 13:04:32.183901 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:04:32.184062 (+   161us) write_op.cc:270] Start()
0709 13:04:32.184177 (+   115us) write_op.cc:276] Timestamp: P: 1783602272184013 usec, L: 0
0709 13:04:32.184199 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:04:32.184952 (+   753us) log.cc:844] Serialized 5154 byte log entry
0709 13:04:32.188322 (+  3370us) op_driver.cc:464] REPLICATION: finished
0709 13:04:32.188798 (+   476us) write_op.cc:301] APPLY: starting
0709 13:04:32.188882 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:32.209918 (+ 21036us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:32.210023 (+   105us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:32.227512 (+ 17489us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:32.227739 (+   227us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 13:04:32.227791 (+    52us) write_op.cc:312] APPLY: finished
0709 13:04:32.228155 (+   364us) log.cc:844] Serialized 121 byte log entry
0709 13:04:32.237986 (+  9831us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:32.247315 (+  9329us) write_op.cc:454] Released schema lock
0709 13:04:32.250470 (+  3155us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:32.266721 (+ 16251us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:32.270985 (+  4264us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10666,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":254,"apply.run_cpu_time_us":18873,"apply.run_wall_time_us":72450,"cfile_cache_hit":52,"cfile_cache_hit_bytes":54366,"num_ops":13,"prepare.queue_time_us":238,"prepare.run_cpu_time_us":5128,"prepare.run_wall_time_us":6143,"replication_time_us":3999}]]}
I20260709 13:04:32.603240  2300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:32.603890  2300 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 18.
I20260709 13:04:32.596733  2268 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:32.578850  9338 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:32.607199  9338 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.607611  9338 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:32.612870  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:32.613169  2296 raft_consensus.cc:2468] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 20.
I20260709 13:04:32.615000  1609 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:32.617004  1625 tablet_service.cc:1558] Processing DeleteTablet for tablet fb294ac919b44e56960745f79e83dfc4 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 14 (current committed config index is 17)) from {username='slave'} at 127.0.0.1:40842
W20260709 13:04:32.617875  1683 consensus_peers.cc:597] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b7e7c3881a44475827c9cbca08a2fe1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:32.619128  9321 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:32.619549  9321 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.587888  9337 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:32.620303  9337 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.620656  9337 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:32.580793  9331 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:32.581017  9329 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:32.596530  9332 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:32.619899  9321 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:32.621649  9052 raft_consensus.cc:2804] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:04:32.626668  9331 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.617847  9345 raft_consensus.cc:2804] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:32.628461  9329 raft_consensus.cc:493] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.615339  9346 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.647790  9346 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.650444  9346 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:32.639343  2268 leader_election.cc:336] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.634868  9342 raft_consensus.cc:2955] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Committing config change with OpId 21.16: config changed from index 6 to 16, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.636582  9332 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.647116  9329 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:32.641613  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:32.639008  1641 raft_consensus.cc:2955] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index 6 to 16, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.662150  1643 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:04:32.665182  9322 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:04:32.665740  9322 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.666158  9322 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:32.668082  9052 raft_consensus.cc:697] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:32.669360  9052 consensus_queue.cc:237] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.674436  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:32.676476  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 47799.5us
I20260709 13:04:32.677203  1683 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:32.678886  9336 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:04:32.680289  9336 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.680684  9336 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:32.641455  9340 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:04:32.704134   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:32.678371  1607 leader_election.cc:336] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:32.679126  2268 leader_election.cc:336] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:32.725188  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 46062.8us
W20260709 13:04:32.725788  2268 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:32.731104  9321 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.703476  9341 raft_consensus.cc:515] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.752585  1709 raft_consensus.cc:2393] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 20: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:32.697144  9335 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:04:32.687590  9285 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.754523  9285 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.756039  9337 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.757378  9285 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:32.758211  9337 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:32.712030  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 51557.3us
W20260709 13:04:32.759039  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 80643.9us
I20260709 13:04:32.659415  9332 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:32.660169  9345 raft_consensus.cc:697] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 20 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:32.687263  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:32.764319  1715 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 22.
I20260709 13:04:32.697587  9322 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.766660  9322 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:32.692425  9336 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.769057  9336 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:32.723694  9340 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.771504  9340 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:32.682848  9338 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.775249  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:32.775960  2296 raft_consensus.cc:3060] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:32.777046  9338 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:32.753770  9341 leader_election.cc:290] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:32.655035  9331 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:32.778218  9345 consensus_queue.cc:237] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.780444  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:32.781131  1642 raft_consensus.cc:3060] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:04:32.762378  2268 leader_election.cc:336] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.770743  9336 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.782366  1453 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d59c4d21b6cb46edbfc2b3d50b51812a with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:32.785611  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:32.786710  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:32.787989  9285 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:04:32.789740  9285 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.790397  9285 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:32.789676  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:32.795236  1709 raft_consensus.cc:3060] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:32.797919  9329 raft_consensus.cc:515] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.799399  1452 catalog_manager.cc:5028] TS 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): tablet fb294ac919b44e56960745f79e83dfc4 (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:04:32.785890  9336 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.804059  1715 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:04:32.808749  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:32.785228  1462 catalog_manager.cc:5697] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 21 to 22, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 22 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.809499  2299 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:32.811141  1683 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:32.811753  1709 raft_consensus.cc:2468] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 19.
I20260709 13:04:32.812538  9341 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:32.814724  9341 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.815181  9341 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:32.815924  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b0b66754d049cfbccb38fbb16ef3cd" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:32.816673  1644 raft_consensus.cc:3060] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:32.813259  9329 leader_election.cc:290] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:32.821153  2296 raft_consensus.cc:2468] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 19.
I20260709 13:04:32.794767  9331 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.801079  9332 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.824302  9341 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:32.826242  1607 leader_election.cc:336] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.815924  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:32.842675  1462 catalog_manager.cc:5697] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.806602  9336 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:32.833092  1644 raft_consensus.cc:2468] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 23.
I20260709 13:04:32.833595  9321 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:32.761270  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c46cf090bf9b4084b648d00f5f0053e2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:32.848657  9331 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:32.848984  1641 raft_consensus.cc:3060] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:04:32.849474  1715 raft_consensus.cc:3060] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:04:32.804138  1642 raft_consensus.cc:2468] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 23.
W20260709 13:04:32.837154  1681 leader_election.cc:336] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.851692  9341 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:32.852022  1682 leader_election.cc:304] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:04:32.852241  1681 consensus_peers.cc:597] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:32.842738  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:04:32.847438  1607 leader_election.cc:336] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:32.856374  1681 consensus_peers.cc:597] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:32.856769  1607 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:32.853000  1682 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
W20260709 13:04:32.857620  1681 leader_election.cc:336] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.857990  9331 raft_consensus.cc:2804] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:32.858225  1681 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:32.858899  1715 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 16.
I20260709 13:04:32.853598  9341 raft_consensus.cc:2804] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:32.860347  9341 raft_consensus.cc:697] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
W20260709 13:04:32.860457  1681 leader_election.cc:336] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.861308  9341 consensus_queue.cc:237] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.861531  9340 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:32.862119  1681 leader_election.cc:336] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.838474  1683 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:32.864360  1458 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45164 (request call id 368) took 4582 ms. Trace:
I20260709 13:04:32.848167  1462 catalog_manager.cc:5697] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 17 to 18, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.866596  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:32.866819  2268 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:32.867426  2296 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 21: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:32.859637  9335 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:32.859653  1607 leader_election.cc:336] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.869465  9337 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:04:32.858844  9331 raft_consensus.cc:697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:32.870108  9337 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3f64b76180a54492bb015e405caeacec)
I20260709 13:04:32.870529  9337 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:04:32.865334  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:32.870886  9331 consensus_queue.cc:237] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.853871  9332 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:32.874032  1607 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:32.862655  1681 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:32.867190  1462 catalog_manager.cc:5697] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.864828  9329 raft_consensus.cc:2804] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:04:32.877056  1607 leader_election.cc:336] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.877734  9329 raft_consensus.cc:697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:32.878399  9282 raft_consensus.cc:2804] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:04:32.878146  1462 catalog_manager.cc:5697] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 24 to 25, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 25 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.879014  9282 raft_consensus.cc:697] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:32.878634  9329 consensus_queue.cc:237] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:32.879949  9282 consensus_queue.cc:237] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:32.880756  1607 leader_election.cc:336] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:32.881350  1607 consensus_peers.cc:597] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:32.868958  9342 raft_consensus.cc:2804] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:32.882365  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:32.880654  1462 catalog_manager.cc:5697] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 17 to 18, leader changed from 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 18 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:32.883762  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:32.883195  1715 raft_consensus.cc:2393] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 21: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:32.884366  1709 raft_consensus.cc:3060] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:04:32.884465  1462 catalog_manager.cc:5697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 16 to 18, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 3f64b76180a54492bb015e405caeacec (127.0.107.193). New cstate: current_term: 18 leader_uuid: "3f64b76180a54492bb015e405caeacec" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:32.885500  1681 leader_election.cc:336] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.868408  1458 rpcz_store.cc:276] 0709 13:04:28.282359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.131967 (+1849608us) service_pool.cc:224] Handling call
0709 13:04:32.864299 (+2732332us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:32.620114 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:32.620633 (+   519us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:32.620666 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:04:32.620687 (+    21us) tablet.cc:662] Decoding operations
0709 13:04:32.659989 (+ 39302us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:32.663204 (+  3215us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:32.663247 (+    43us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:04:32.663846 (+   599us) tablet.cc:701] Row locks acquired
0709 13:04:32.663867 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:04:32.664039 (+   172us) write_op.cc:270] Start()
0709 13:04:32.664156 (+   117us) write_op.cc:276] Timestamp: P: 1783602272663990 usec, L: 0
0709 13:04:32.664182 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:04:32.664954 (+   772us) log.cc:844] Serialized 2462 byte log entry
0709 13:04:32.667177 (+  2223us) op_driver.cc:464] REPLICATION: finished
0709 13:04:32.667631 (+   454us) write_op.cc:301] APPLY: starting
0709 13:04:32.667715 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:32.672384 (+  4669us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:32.672489 (+   105us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:32.737540 (+ 65051us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:32.737778 (+   238us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:04:32.737836 (+    58us) write_op.cc:312] APPLY: finished
0709 13:04:32.738137 (+   301us) log.cc:844] Serialized 65 byte log entry
0709 13:04:32.739102 (+   965us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:32.739236 (+   134us) write_op.cc:454] Released schema lock
0709 13:04:32.739285 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:32.758944 (+ 19659us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:32.758979 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2147,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":253,"apply.run_cpu_time_us":10000,"apply.run_wall_time_us":91255,"cfile_cache_hit":24,"cfile_cache_hit_bytes":25092,"num_ops":6,"prepare.queue_time_us":2807,"prepare.run_cpu_time_us":3788,"prepare.run_wall_time_us":45122,"replication_time_us":2859}]]}
W20260709 13:04:32.887290  1681 leader_election.cc:336] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.863067  9285 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:32.889734  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:32.884006  9318 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:32.886684  9342 raft_consensus.cc:697] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:32.896911  1709 raft_consensus.cc:2468] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 19.
W20260709 13:04:32.897219  2268 leader_election.cc:336] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:32.899520  2268 leader_election.cc:336] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.902680  1607 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:32.900532  1641 raft_consensus.cc:2468] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 20.
I20260709 13:04:32.907657  1682 leader_election.cc:304] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:32.911681  9350 raft_consensus.cc:2749] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:32.913009  9318 raft_consensus.cc:2804] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:04:32.913625  9318 raft_consensus.cc:697] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:32.914489  9318 consensus_queue.cc:237] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:32.916383  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:32.917061  1715 raft_consensus.cc:3060] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:04:32.927565  9340 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:32.900321  9342 consensus_queue.cc:237] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:32.907349  1681 leader_election.cc:336] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:32.932963  2268 leader_election.cc:336] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.943410  1715 raft_consensus.cc:2468] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 23.
I20260709 13:04:32.939465  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:32.897426  9285 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:32.943899  9350 raft_consensus.cc:2804] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:04:32.969339  2268 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:32.960664  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:04:32.970129  1681 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:32.970232  1709 raft_consensus.cc:3060] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:32.970781  1681 consensus_peers.cc:597] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:32.972138  1681 consensus_peers.cc:597] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:32.972833  1681 consensus_peers.cc:597] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:32.973887  1681 leader_election.cc:336] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.976197  9332 raft_consensus.cc:2804] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:04:32.979038  9332 raft_consensus.cc:697] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:32.980085  9332 consensus_queue.cc:237] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:32.981316  2268 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:32.951269  9285 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:04:32.989357  2268 leader_election.cc:336] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:32.989488  1681 leader_election.cc:336] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:32.991057  2268 leader_election.cc:336] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:32.989820  1709 raft_consensus.cc:2468] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 24.
I20260709 13:04:32.982319  1643 raft_consensus.cc:2955] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:04:33.001677  9337 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:33.003022  2299 raft_consensus.cc:2393] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 21: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:32.988263  9350 raft_consensus.cc:697] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:33.008366  9350 consensus_queue.cc:237] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:33.023655  9338 raft_consensus.cc:2955] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:04:33.027496  1461 catalog_manager.cc:5697] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } }
W20260709 13:04:33.032292  1681 leader_election.cc:336] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:33.036099  1607 consensus_peers.cc:597] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:33.036676  1607 consensus_peers.cc:597] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:33.057118  1681 leader_election.cc:336] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:33.058028  1681 leader_election.cc:336] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:33.048947  9282 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.061728  9282 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:33.058768  1681 leader_election.cc:336] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:33.072973  1681 leader_election.cc:336] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:33.073535  1681 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:33.075258  2300 raft_consensus.cc:1275] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:04:33.078409  1681 leader_election.cc:336] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:33.078977  1681 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:33.088181  9330 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:33.081741  9341 raft_consensus.cc:2749] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:33.082176  9318 consensus_queue.cc:1048] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:04:33.161791  1636 raft_consensus.cc:1275] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 28 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
W20260709 13:04:33.166502  1681 consensus_peers.cc:597] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:33.167311  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 83674.2us
I20260709 13:04:33.178582  9338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:57326 (request call id 1) took 1235 ms. Trace:
W20260709 13:04:33.178967  1607 consensus_peers.cc:597] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:33.191656  9289 consensus_queue.cc:1048] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:33.227816  9361 ts_tablet_manager.cc:933] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:33.179077  9338 rpcz_store.cc:276] 0709 13:04:31.943628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.943845 (+   217us) service_pool.cc:224] Handling call
0709 13:04:31.982113 (+ 38268us) log.cc:844] Serialized 506 byte log entry
0709 13:04:32.243622 (+261509us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b180040f660. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb2ec0a 00007fd7a826e219 00007fd79ebb6a40 00007fd79ebb14e4 00007fd79ebb1455 00007fd7a1bfab0b 00007fd7a1c7fb63 00007fd7a1ca7752 00007fd7a6c67d29
0709 13:04:32.341531 (+ 97909us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b54002551a0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c1a142 00007fd7a1bfaeaa 00007fd7a1c01d53 00007fd7a1c01cd9 00007fd7a1c01c69 00007fd7a1c01c31 00007fd7a1c00dfd 000000000038f354 000000000038f199 00007fd79ebb4475
0709 13:04:32.344507 (+  2976us) spinlock_profiling.cc:243] Waited 888 us on lock 0x7b54002551a0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c16dbe 00007fd7a1c21930 00007fd7a1c218d8 00007fd7a1c21858 00007fd7a1c21813 00007fd7a1c20a9c 00007fd7a7b0c626 00007fd7a7b06dd8 00007fd7a1c5d7cb 00007fd7a1c602c0
0709 13:04:32.486712 (+142205us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b180040f660. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a7a7cde1 00007fd7a7a895b1 00007fd79eb822a6 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c21b26 00007fd7a1c1b49a 00007fd7a1c1a714
0709 13:04:32.487264 (+   552us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b54002551a0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c1a142 00007fd7a1bfaeaa 00007fd7a1c01d53 00007fd7a1c01cd9 00007fd7a1c01c69 00007fd7a1c01c31 00007fd7a1c00dfd 000000000038f354 000000000038f199 00007fd79ebb4475
0709 13:04:32.499149 (+ 11885us) spinlock_profiling.cc:243] Waited 5.42 ms on lock 0x7b54002551a0. stack: 00007fd79eb82343 00007fd79eb82240 00007fd79e5111d4 00007fd7a824f304 00007fd7a824f299 00007fd7a1c0a571 00007fd7a1c21b26 00007fd7a1c1b49a 00007fd7a1c1a714 00007fd7a1bfaeaa 00007fd7a1c01d53 00007fd7a1c01cd9 00007fd7a1c01c69 00007fd7a1c01c31 00007fd7a1c00dfd 000000000038f354
0709 13:04:33.178483 (+679334us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2384,"raft.queue_time_us":12585,"raft.run_cpu_time_us":25572,"raft.run_wall_time_us":606989,"spinlock_wait_cycles":39755136,"thread_start_us":251117,"threads_started":1,"wal-append.queue_time_us":256376}
I20260709 13:04:33.238780  9282 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:33.242815  9337 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:33.243351  9330 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.243866  9330 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:33.245890  9330 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:33.252882  1607 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:33.258153  1607 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:33.275153  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:33.265585  1461 catalog_manager.cc:5697] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 22 to 23, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 23 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:33.286338  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:33.287381  1643 raft_consensus.cc:1275] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:04:33.287307  1709 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 26.
I20260709 13:04:33.276069  1636 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 26.
W20260709 13:04:33.291745  1607 consensus_peers.cc:597] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:33.283952  1461 catalog_manager.cc:5697] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 6 to 16, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New cstate: current_term: 21 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:33.301056  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:33.301885  1709 raft_consensus.cc:2393] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 21: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:33.302763  2268 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:33.302007  1461 catalog_manager.cc:5697] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 18 to 19, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 19 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:33.305727  1681 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:33.306452  1681 leader_election.cc:336] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:33.308575  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 
I20260709 13:04:33.313158  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:33.314812  9321 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:33.315466  9321 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.315858  9321 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:04:33.311297  9292 consensus_queue.cc:1048] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:04:33.329777  2299 raft_consensus.cc:1275] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:04:33.307922  9340 raft_consensus.cc:2804] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:33.328949  9329 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:04:33.333298  9340 raft_consensus.cc:697] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:33.334386  9340 consensus_queue.cc:237] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:33.333300  9329 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.337309  9329 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:04:33.325318  1681 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:33.319665  9361 tablet_copy_client.cc:323] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33997
I20260709 13:04:33.364893  1643 raft_consensus.cc:1275] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:04:33.373441  1727 tablet_copy_service.cc:140] P 28bc59d37f664ce0a6f5784f2619f0b5: Received BeginTabletCopySession request for tablet 1b7e7c3881a44475827c9cbca08a2fe1 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:33280)
W20260709 13:04:33.374035  2268 leader_election.cc:336] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:33.374527  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:33.361946  1607 consensus_peers.cc:597] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:33.376749  1681 consensus_peers.cc:597] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:33.379637  1727 tablet_copy_service.cc:161] P 28bc59d37f664ce0a6f5784f2619f0b5: Beginning new tablet copy session on tablet 1b7e7c3881a44475827c9cbca08a2fe1 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:33280: session id = 6208c2fa49f643b5a50ab8f6d292fab2-1b7e7c3881a44475827c9cbca08a2fe1
W20260709 13:04:33.380950  2268 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:33.381639  2268 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:33.382046  2268 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:33.383095  9337 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:04:33.370492  9327 consensus_queue.cc:1048] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:04:33.386171  1643 raft_consensus.cc:1275] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:04:33.389133  9330 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:33.389704  9330 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:33.390735  9327 consensus_queue.cc:1048] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:33.392391  9330 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:33.395659  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:33.396540  2268 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:33.403498  1462 catalog_manager.cc:5697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index -1 to 12, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 17 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:04:33.406090  1462 catalog_manager.cc:5697] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 19 to 20, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.410686  1462 catalog_manager.cc:5697] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 15 to 16, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 16 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.413177  1462 catalog_manager.cc:5697] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 18 to 19, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 19 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:33.415506  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:33.447249  1682 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:33.325511  9321 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:33.459285  9329 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:33.463317  9371 consensus_queue.cc:1048] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:04:33.468393  1727 tablet_copy_source_session.cc:215] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:33.460230  9321 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 27 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:33.477631  1699 tablet_service.cc:1558] Processing DeleteTablet for tablet d59c4d21b6cb46edbfc2b3d50b51812a with delete_type TABLET_DATA_TOMBSTONED (TS 28bc59d37f664ce0a6f5784f2619f0b5 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:57312
I20260709 13:04:33.479288  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:33.479673  2300 raft_consensus.cc:1275] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:04:33.480062  1709 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 5184c101be204393adf3ac8b31f25ad6 in current term 27: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:04:33.489728  1714 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:04:33.454242  9335 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:33.511201  9345 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:04:33.544564  1715 raft_consensus.cc:1275] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:04:33.547765  1461 catalog_manager.cc:5697] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 12 to 13, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 13 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:33.511636  9331 consensus_queue.cc:1048] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 13:04:33.551067  1461 catalog_manager.cc:5697] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 23 to 24, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 24 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:33.558565  1681 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:33.565054  9374 log.cc:927] Time spent T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:04:33.568082  9340 consensus_queue.cc:1048] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:04:33.596383  9361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b7e7c3881a44475827c9cbca08a2fe1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:33.601766  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 156049us
W20260709 13:04:33.665086  1681 consensus_peers.cc:597] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:33.665863  1681 consensus_peers.cc:597] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:33.666344  1681 consensus_peers.cc:597] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:33.666749  1681 leader_election.cc:336] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:33.667538  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 53922.8us
W20260709 13:04:33.665937  1607 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:33.672554  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:33.686721  1715 raft_consensus.cc:1275] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:04:33.681739  9373 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:04:33.690428  9377 tablet_replica.cc:333] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
W20260709 13:04:33.723881   431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 1590)
I20260709 13:04:33.725121  9377 raft_consensus.cc:2243] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:33.726142  9377 raft_consensus.cc:2272] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:33.727121  9329 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 27 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:33.711391  9323 consensus_queue.cc:1048] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.031s
I20260709 13:04:33.742607  9377 ts_tablet_manager.cc:1916] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:33.759346  1709 raft_consensus.cc:1275] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:04:33.761830  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 72294us
W20260709 13:04:33.768079  2268 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:33.768492  2268 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:33.769093  2268 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:33.780541  2268 consensus_peers.cc:597] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:33.791669  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 116200us
I20260709 13:04:33.796245  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:33.797451  1643 raft_consensus.cc:2393] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 27: Already voted for candidate 5184c101be204393adf3ac8b31f25ad6 in this term.
I20260709 13:04:33.797889  1715 raft_consensus.cc:1275] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:04:33.825656  9372 consensus_queue.cc:1048] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:04:33.821177  9323 consensus_queue.cc:1048] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.011s
W20260709 13:04:33.834986  1607 consensus_peers.cc:597] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:33.835357  2268 consensus_peers.cc:597] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:33.853659  1681 consensus_peers.cc:597] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:33.854310  1681 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:33.854708  1681 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:33.855126  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:33.858502  1681 consensus_peers.cc:597] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:33.859050  1681 consensus_peers.cc:597] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:33.860100  9387 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 13:04:33.871546  1607 consensus_peers.cc:597] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:33.871748  1681 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:33.881605  1458 catalog_manager.cc:5697] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 22 to 23, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 23 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:33.896570  1458 catalog_manager.cc:5697] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 22 to 23, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 23 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:33.903473  1458 catalog_manager.cc:5697] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 27 to 28, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 28 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:33.915668  1458 catalog_manager.cc:5697] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 19 to 20, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 20 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:33.917841  1607 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:33.919597  1458 catalog_manager.cc:5697] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 18 to 19, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 19 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:33.970638  9382 log.cc:927] Time spent T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:04:34.063313  9361 tablet_copy_client.cc:806] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:34.064039  9361 tablet_copy_client.cc:670] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:34.067256  9377 ts_tablet_manager.cc:1929] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 21.15
I20260709 13:04:34.068946  9377 log.cc:1199] T d59c4d21b6cb46edbfc2b3d50b51812a P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/wals/d59c4d21b6cb46edbfc2b3d50b51812a
I20260709 13:04:34.071637  1451 catalog_manager.cc:5028] TS 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): tablet d59c4d21b6cb46edbfc2b3d50b51812a (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
I20260709 13:04:34.071549  9361 tablet_copy_client.cc:538] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:34.213441  9361 tablet_bootstrap.cc:492] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
W20260709 13:04:34.393432  2268 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:34.427731  1681 consensus_peers.cc:597] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:34.574568  9361 tablet_bootstrap.cc:492] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=1 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:34.602959  9361 tablet_bootstrap.cc:492] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:04:34.615712  9361 ts_tablet_manager.cc:1403] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.406s	user 0.059s	sys 0.008s
I20260709 13:04:34.643420  9361 raft_consensus.cc:359] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:04:34.649799  9361 raft_consensus.cc:740] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:04:34.650606  9361 consensus_queue.cc:260] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:04:34.662271  9361 ts_tablet_manager.cc:1434] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.000s
I20260709 13:04:34.681239  1727 tablet_copy_service.cc:342] P 28bc59d37f664ce0a6f5784f2619f0b5: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-1b7e7c3881a44475827c9cbca08a2fe1 received from {username='slave'} at 127.0.0.1:33280
I20260709 13:04:34.681826  1727 tablet_copy_service.cc:434] P 28bc59d37f664ce0a6f5784f2619f0b5: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-1b7e7c3881a44475827c9cbca08a2fe1 on tablet 1b7e7c3881a44475827c9cbca08a2fe1 with peer 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:34.684057  2296 raft_consensus.cc:1217] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEARNER]: Deduplicated request from leader. Original: 17.11->[17.12-17.12]   Dedup: 17.12->[]
W20260709 13:04:34.724427   431 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:34.790583  1681 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:34.908905  2268 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:34.917661  9337 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.918179  9337 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:34.920324  9337 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:34.922565  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:34.923926  1709 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:04:34.928520  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:34.936308  9337 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:34.936831  9337 raft_consensus.cc:493] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.937256  9337 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:34.956864  2268 leader_election.cc:336] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:34.971683  1709 consensus_queue.cc:237] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:35.046396  1607 consensus_peers.cc:597] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:35.057987  1681 consensus_peers.cc:597] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:35.071183  9337 raft_consensus.cc:515] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:35.074080  1681 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:35.074143  9337 leader_election.cc:290] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:35.073463  2296 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:35.078294  1643 raft_consensus.cc:1275] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:04:35.080214  9385 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:35.083627  9289 consensus_queue.cc:1048] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:04:35.094910  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdf54b70d44944ef9013c4adb0970687" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:35.095876  1709 raft_consensus.cc:3060] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:04:35.112088  9292 raft_consensus.cc:2955] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 5 to 11, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:35.115232  2268 leader_election.cc:336] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:35.121297  2296 raft_consensus.cc:2955] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 5 to 11, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:35.125586  1709 raft_consensus.cc:2468] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 22.
I20260709 13:04:35.127365  2268 leader_election.cc:304] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:35.128219  9337 raft_consensus.cc:2804] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:35.128770  9337 raft_consensus.cc:697] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:35.129586  9337 consensus_queue.cc:237] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:35.151865  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3f22f7f5cede49bea2083e8905932fb7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 11)
I20260709 13:04:35.176141  1462 catalog_manager.cc:5697] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 5 to 11, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 15 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:35.231947  1461 catalog_manager.cc:5697] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 20 to 22, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 22 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:35.224123  9342 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.269033  9385 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.248667  9342 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:35.275416  1681 consensus_peers.cc:597] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:35.274014  9385 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:35.277462  9342 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:35.278359  9385 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:35.276202  9403 raft_consensus.cc:1064] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:04:35.302132  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:35.302791  1709 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
W20260709 13:04:35.303920  1681 leader_election.cc:336] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:35.305528  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:04:35.270737  1636 raft_consensus.cc:2955] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 5 to 11, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:35.332072  9404 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:04:35.332594  9404 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.333004  9404 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:35.340947  2268 leader_election.cc:336] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:35.344172  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 72027.4us
I20260709 13:04:35.357522  9403 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:35.358680  1607 consensus_peers.cc:597] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:35.368294  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 98298.4us
I20260709 13:04:35.370674  2299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:35.372155  1681 consensus_peers.cc:597] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:35.395126  9404 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:35.396489  2299 raft_consensus.cc:2393] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 22: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:35.397627  9404 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:35.406796  1636 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:04:35.408097  1683 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:35.408782  1681 consensus_peers.cc:597] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:35.411746  2300 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 LEARNER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:04:35.413095  2268 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:35.419466  9292 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:04:35.436192  1607 consensus_peers.cc:597] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:35.437373  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:35.437636  9403 raft_consensus.cc:3060] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:04:35.437517  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:35.447461  2268 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111) [suppressed 390 similar messages]
W20260709 13:04:35.448836  2268 leader_election.cc:336] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:35.455195  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 47166.9us
I20260709 13:04:35.466142  9292 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:04:35.468859  1681 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:35.556277  1709 raft_consensus.cc:1275] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:04:35.569697  9403 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:35.579998  9342 consensus_queue.cc:1048] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:04:35.581272  1712 raft_consensus.cc:2468] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 22.
I20260709 13:04:35.583523  2268 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:35.584290  9342 raft_consensus.cc:2804] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:04:35.584807  9342 raft_consensus.cc:697] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:35.585682  9342 consensus_queue.cc:237] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:35.591204  9317 raft_consensus.cc:2955] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index 12 to 13, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:35.597448  2300 raft_consensus.cc:2955] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:35.605871  1681 consensus_peers.cc:597] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:35.606436  1681 consensus_peers.cc:597] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:35.624280  1607 consensus_peers.cc:597] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:35.637856  2268 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:35.640058  1451 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 63023.5us
I20260709 13:04:35.662596  1461 catalog_manager.cc:5697] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 20 to 22, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 22 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:35.666114  1636 raft_consensus.cc:2955] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:35.687127  2268 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:35.699523  1451 catalog_manager.cc:4755] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3f22f7f5cede49bea2083e8905932fb7: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:35.707454  1461 catalog_manager.cc:5697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index 12 to 13, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:35.727415   431 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:35.740515  1681 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:35.755642  2268 consensus_peers.cc:597] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:35.759923  1712 consensus_queue.cc:237] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:35.783465  1681 consensus_peers.cc:597] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:35.808162  2268 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:35.812300  2300 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:35.814239  9385 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:04:35.824080  9403 raft_consensus.cc:2955] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:35.838598  2300 raft_consensus.cc:2955] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:35.849203  1636 raft_consensus.cc:1275] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:04:35.851244  9385 consensus_queue.cc:1048] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:04:35.855605  1641 raft_consensus.cc:2955] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:35.861141  1461 catalog_manager.cc:5697] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: config changed from index 13 to 14, VOTER 3f64b76180a54492bb015e405caeacec (127.0.107.193) evicted. New cstate: current_term: 17 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:35.884559  1451 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1b7e7c3881a44475827c9cbca08a2fe1 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:04:35.897974  1681 consensus_peers.cc:597] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:35.925323  1607 consensus_peers.cc:597] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:35.937001  1607 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:35.942778  1681 consensus_peers.cc:597] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:35.943424  2268 consensus_peers.cc:597] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:35.949898  2268 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:35.973855  2268 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:36.015841  1681 consensus_peers.cc:597] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:36.048600  1607 consensus_peers.cc:597] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:36.086251  1712 raft_consensus.cc:1275] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
W20260709 13:04:36.088020  1681 consensus_peers.cc:597] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:36.088653  1681 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:36.124405  1681 consensus_peers.cc:597] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:36.139076  9342 consensus_queue.cc:1048] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:04:36.177569  9282 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.178268  9282 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:36.201996  1607 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:36.221477  1681 consensus_peers.cc:597] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:36.223407  2268 consensus_peers.cc:597] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:36.228735  9282 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:36.237349  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:36.238202  1709 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 27.
W20260709 13:04:36.244772  2268 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:36.369230  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 123247us
W20260709 13:04:36.418963  1681 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:36.450196  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 190151us
I20260709 13:04:36.476172  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:36.503985  9424 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:04:36.504657  9424 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.505097  9424 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:04:36.516695  2268 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:36.537500  2268 consensus_peers.cc:597] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:36.541695  1607 consensus_peers.cc:597] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:36.545682  1607 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:36.547415  1607 consensus_peers.cc:597] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:36.554320  9424 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:36.567840  9424 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 28 election: Requested vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:36.590588  1607 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:36.608150  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:36.609287  1710 raft_consensus.cc:3060] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:04:36.621207  1710 raft_consensus.cc:2468] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5184c101be204393adf3ac8b31f25ad6 in term 28.
I20260709 13:04:36.670421  1607 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec
I20260709 13:04:36.671329  9424 raft_consensus.cc:2804] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:04:36.671916  9424 raft_consensus.cc:697] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 28 LEADER]: Becoming Leader. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:36.673092  9424 consensus_queue.cc:237] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:36.711585  1451 catalog_manager.cc:4755] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1b7e7c3881a44475827c9cbca08a2fe1: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:36.715783  1458 catalog_manager.cc:5697] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: term changed from 24 to 28, leader changed from 3f64b76180a54492bb015e405caeacec (127.0.107.193) to 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194). New cstate: current_term: 28 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:36.728089   431 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:36.968775  2268 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:37.082495  1607 consensus_peers.cc:597] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:37.105589  1636 consensus_queue.cc:237] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
W20260709 13:04:37.141789  1681 consensus_peers.cc:597] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:37.150941  1609 consensus_peers.cc:597] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715): Couldn't send request to peer 6208c2fa49f643b5a50ab8f6d292fab2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c2832827f30046328715699fa6ee0c52. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:37.148578  1714 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:04:37.154629  9424 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
W20260709 13:04:37.161206  1607 consensus_peers.cc:597] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:37.168771  2268 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:37.194768  9282 raft_consensus.cc:2955] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:04:37.197130  1710 raft_consensus.cc:2955] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:04:37.223047  1458 catalog_manager.cc:5697] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: config changed from index -1 to 11, NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) added. New cstate: current_term: 16 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } } }
I20260709 13:04:37.288715  1710 raft_consensus.cc:1275] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 28 index: 14. (index mismatch)
I20260709 13:04:37.290483  9282 consensus_queue.cc:1048] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:04:37.507059  1607 consensus_peers.cc:597] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:37.619954  1681 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:37.643082  9442 ts_tablet_manager.cc:933] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2: Initiating tablet copy from peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:37.666051  1681 consensus_peers.cc:597] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:37.685335  2268 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:37.686266  1681 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:37.729146   431 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 1590)
I20260709 13:04:37.752465  9442 tablet_copy_client.cc:323] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:40023
W20260709 13:04:37.761781  1607 consensus_peers.cc:597] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:37.766358  1654 tablet_copy_service.cc:140] P 5184c101be204393adf3ac8b31f25ad6: Received BeginTabletCopySession request for tablet c2832827f30046328715699fa6ee0c52 from peer 6208c2fa49f643b5a50ab8f6d292fab2 ({username='slave'} at 127.0.0.1:53222)
I20260709 13:04:37.767196  1654 tablet_copy_service.cc:161] P 5184c101be204393adf3ac8b31f25ad6: Beginning new tablet copy session on tablet c2832827f30046328715699fa6ee0c52 from peer 6208c2fa49f643b5a50ab8f6d292fab2 at {username='slave'} at 127.0.0.1:53222: session id = 6208c2fa49f643b5a50ab8f6d292fab2-c2832827f30046328715699fa6ee0c52
I20260709 13:04:37.776988  1654 tablet_copy_source_session.cc:215] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:37.787606  9442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2832827f30046328715699fa6ee0c52. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:37.817122  1681 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:37.836469  9442 tablet_copy_client.cc:806] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:37.837347  9442 tablet_copy_client.cc:670] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:37.849788  9442 tablet_copy_client.cc:538] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:04:37.864517  1607 consensus_peers.cc:597] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:37.875818  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:37.889503  9442 tablet_bootstrap.cc:492] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap starting.
W20260709 13:04:37.934811  2268 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:38.083242  1451 catalog_manager.cc:4755] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3f22f7f5cede49bea2083e8905932fb7: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:38.100045  1681 consensus_peers.cc:597] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:38.125624  9442 tablet_bootstrap.cc:492] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=2 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:38.143839  9442 tablet_bootstrap.cc:492] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2: Bootstrap complete.
I20260709 13:04:38.144735  9442 ts_tablet_manager.cc:1403] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent bootstrapping tablet: real 0.256s	user 0.042s	sys 0.011s
I20260709 13:04:38.148411  9442 raft_consensus.cc:359] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:04:38.149436  9442 raft_consensus.cc:740] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Initialized, Role: LEARNER
I20260709 13:04:38.150249  9442 consensus_queue.cc:260] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: true } }
I20260709 13:04:38.153496  9442 ts_tablet_manager.cc:1434] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:38.156041  1654 tablet_copy_service.cc:342] P 5184c101be204393adf3ac8b31f25ad6: Request end of tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-c2832827f30046328715699fa6ee0c52 received from {username='slave'} at 127.0.0.1:53222
I20260709 13:04:38.156611  1654 tablet_copy_service.cc:434] P 5184c101be204393adf3ac8b31f25ad6: ending tablet copy session 6208c2fa49f643b5a50ab8f6d292fab2-c2832827f30046328715699fa6ee0c52 on tablet c2832827f30046328715699fa6ee0c52 with peer 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:38.169277  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 46485.1us
W20260709 13:04:38.261722  1681 consensus_peers.cc:597] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:38.315979  1681 consensus_peers.cc:597] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:38.316644  1607 consensus_peers.cc:597] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:38.321265  2268 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:38.321888  1681 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:38.366786  2268 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:38.385255  1681 consensus_peers.cc:597] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:38.396571  2268 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:38.398808  2268 consensus_peers.cc:597] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:38.442225  1607 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:38.492056  1681 consensus_peers.cc:597] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:38.496116  1681 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:38.526165  2268 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:38.533771  2302 raft_consensus.cc:1217] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.10->[16.11-16.11]   Dedup: 16.11->[]
W20260709 13:04:38.559438  1681 consensus_peers.cc:597] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:38.567160  1681 consensus_peers.cc:597] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:38.574849  1681 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:38.584872  2268 consensus_peers.cc:597] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:38.717257  1681 consensus_peers.cc:597] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:38.729653   431 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:38.743641  1607 consensus_peers.cc:597] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:38.767035  2268 consensus_peers.cc:597] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:38.768327  1607 consensus_peers.cc:597] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:38.777520  1681 consensus_peers.cc:597] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:38.777333  2268 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:38.859004  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 48436.2us
W20260709 13:04:38.879514  2268 consensus_peers.cc:597] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:38.913929  1681 consensus_peers.cc:597] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:39.013571  1607 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:39.063966  9282 raft_consensus.cc:1064] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: attempting to promote NON_VOTER 6208c2fa49f643b5a50ab8f6d292fab2 to VOTER
I20260709 13:04:39.066497  9282 consensus_queue.cc:237] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:39.100034  1607 consensus_peers.cc:597] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:39.116070  1710 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:39.117870  2300 raft_consensus.cc:1275] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 LEARNER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:04:39.120442  9424 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:04:39.122087  9448 consensus_queue.cc:1048] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:04:39.144073  1607 consensus_peers.cc:597] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:39.203015  2268 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:04:39.221158  9450 raft_consensus.cc:2955] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:39.260097  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 59123.3us
I20260709 13:04:39.272177  2300 raft_consensus.cc:2955] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:39.285998  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 48521.7us
I20260709 13:04:39.316882  1458 catalog_manager.cc:5697] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 11 to 12, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:39.332324  1710 raft_consensus.cc:2955] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
W20260709 13:04:39.343374  9452 log.cc:927] Time spent T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2: Append to log took a long time: real 0.062s	user 0.007s	sys 0.004s
W20260709 13:04:39.461555  2268 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:39.557461  1607 consensus_peers.cc:597] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:39.677366  1681 consensus_peers.cc:597] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:39.730233   431 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:39.944149  2268 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:40.098842  2268 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.118078  1681 consensus_peers.cc:597] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.135450  1681 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.201380  1451 catalog_manager.cc:4755] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3f22f7f5cede49bea2083e8905932fb7: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:40.204588  1607 consensus_peers.cc:597] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.246764  1681 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.321187  1681 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:40.326717  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.405654  1607 consensus_peers.cc:597] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.443948  1607 consensus_peers.cc:597] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.451458  1681 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111) [suppressed 454 similar messages]
W20260709 13:04:40.490244  1681 consensus_peers.cc:597] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:40.523377  2268 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.556345  2268 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.705641  1681 consensus_peers.cc:597] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.712523  1681 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.730736   431 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:40.737875  2268 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.792829  1681 consensus_peers.cc:597] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.872251  1681 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.896391  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 74722.9us
W20260709 13:04:40.904901  2268 consensus_peers.cc:597] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:40.931340  1681 consensus_peers.cc:597] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:40.960307  1607 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.970067  1607 consensus_peers.cc:597] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.985440  1681 consensus_peers.cc:597] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:40.991554  2268 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:41.035369  1607 consensus_peers.cc:597] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:41.039570  1681 consensus_peers.cc:597] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:41.123538  2268 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:41.133539  2268 consensus_peers.cc:597] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:41.157311  2268 consensus_peers.cc:597] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:41.190001  2268 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:41.218546  1681 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:41.267267  1681 consensus_peers.cc:597] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:41.275372  1681 consensus_peers.cc:597] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:41.277015  1607 consensus_peers.cc:597] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:41.340874  1681 consensus_peers.cc:597] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:41.341822  2268 consensus_peers.cc:597] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:41.419579  1681 consensus_peers.cc:597] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:41.436662  1607 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:41.437367  1607 consensus_peers.cc:597] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:41.654110  2268 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:41.721129  1607 consensus_peers.cc:597] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:41.731258   431 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:41.814213  1739 debug-util.cc:398] Leaking SignalData structure 0x7b0800231f00 after lost signal to thread 434
W20260709 13:04:41.815536  1739 debug-util.cc:398] Leaking SignalData structure 0x7b08002077e0 after lost signal to thread 1501
W20260709 13:04:41.816988  1739 debug-util.cc:398] Leaking SignalData structure 0x7b08001cf3c0 after lost signal to thread 1668
W20260709 13:04:41.818138  1739 debug-util.cc:398] Leaking SignalData structure 0x7b080010d2c0 after lost signal to thread 1742
W20260709 13:04:41.827752  1739 debug-util.cc:398] Leaking SignalData structure 0x7b08003bb420 after lost signal to thread 2329
W20260709 13:04:42.011418  1607 consensus_peers.cc:597] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:42.029588  2268 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:42.262820  1681 consensus_peers.cc:597] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:42.556368  2268 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:42.686956  1681 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:42.731765   431 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:42.736006  2268 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:42.742586  1681 consensus_peers.cc:597] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:42.770872  1607 consensus_peers.cc:597] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:42.997255  2268 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.001454  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.068496  1681 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.095090  2268 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.110174  1607 consensus_peers.cc:597] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.113790  1681 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.175487  1607 consensus_peers.cc:597] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.265466  1681 consensus_peers.cc:597] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:43.277058  1681 consensus_peers.cc:597] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.285262  1681 consensus_peers.cc:597] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.336086  1607 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.339812  2268 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.369426  2268 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.372231  2268 consensus_peers.cc:597] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:43.387334  1498 debug-util.cc:398] Leaking SignalData structure 0x7b0800198b20 after lost signal to thread 434
W20260709 13:04:43.388482  1498 debug-util.cc:398] Leaking SignalData structure 0x7b0800366280 after lost signal to thread 1501
W20260709 13:04:43.389825  1498 debug-util.cc:398] Leaking SignalData structure 0x7b08001ed340 after lost signal to thread 1668
W20260709 13:04:43.391307  1498 debug-util.cc:398] Leaking SignalData structure 0x7b080018f8e0 after lost signal to thread 1742
W20260709 13:04:43.392534  1498 debug-util.cc:398] Leaking SignalData structure 0x7b080023d240 after lost signal to thread 2329
W20260709 13:04:43.422259  1681 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.496963  2268 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:43.554380  1681 consensus_peers.cc:597] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:43.559355  1607 consensus_peers.cc:597] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.606148  1681 consensus_peers.cc:597] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:43.633294  1681 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.633896  1681 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.651122  1681 consensus_peers.cc:597] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.664835  1607 consensus_peers.cc:597] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:43.675851  2268 consensus_peers.cc:597] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:43.690171  2268 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:43.718050  1681 consensus_peers.cc:597] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:43.732447   431 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:43.786715  2268 consensus_peers.cc:597] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:43.836200  1607 consensus_peers.cc:597] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:43.928869  1681 consensus_peers.cc:597] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:43.933784  1607 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:43.999066  2268 consensus_peers.cc:597] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:44.023389  1607 consensus_peers.cc:597] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:44.062883  1607 consensus_peers.cc:597] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:44.134274  2268 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:04:44.236042  1644 consensus_queue.cc:237] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:44.276103  2268 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:04:44.296710  2299 raft_consensus.cc:1275] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Refusing update from remote peer 5184c101be204393adf3ac8b31f25ad6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:04:44.299286  9471 consensus_queue.cc:1048] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:04:44.301041  1607 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:44.337703  1681 consensus_peers.cc:597] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:44.359470  1451 catalog_manager.cc:4755] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3f22f7f5cede49bea2083e8905932fb7: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:44.383144  1607 consensus_peers.cc:597] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:44.399621  9448 raft_consensus.cc:2955] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index 5 to 12, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:44.401954  2296 raft_consensus.cc:2955] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 5 to 12, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } } }
I20260709 13:04:44.414453  1452 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ca83cd5e36eb4e22b75011b87bfcd49d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 12)
I20260709 13:04:44.434296  1462 catalog_manager.cc:5697] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 reported cstate change: config changed from index 5 to 12, VOTER 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) evicted. New cstate: current_term: 18 leader_uuid: "5184c101be204393adf3ac8b31f25ad6" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:44.511031  1699 tablet_service.cc:1558] Processing DeleteTablet for tablet ca83cd5e36eb4e22b75011b87bfcd49d with delete_type TABLET_DATA_TOMBSTONED (TS 28bc59d37f664ce0a6f5784f2619f0b5 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:57312
W20260709 13:04:44.513649  1681 consensus_peers.cc:597] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:04:44.529978  9498 tablet_replica.cc:333] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:04:44.575515  9498 raft_consensus.cc:2243] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:44.576346  9498 raft_consensus.cc:2272] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:44.618323  9498 ts_tablet_manager.cc:1916] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:44.646344  9498 ts_tablet_manager.cc:1929] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.11
I20260709 13:04:44.646946  9498 log.cc:1199] T ca83cd5e36eb4e22b75011b87bfcd49d P 28bc59d37f664ce0a6f5784f2619f0b5: Deleting WAL directory at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0/minicluster-data/ts-2-root/wals/ca83cd5e36eb4e22b75011b87bfcd49d
I20260709 13:04:44.648984  1451 catalog_manager.cc:5028] TS 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): tablet ca83cd5e36eb4e22b75011b87bfcd49d (table test-workload [id=4e07af9de3724e878ee758dbc1188478]) successfully deleted
W20260709 13:04:44.732969   431 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:44.805670  1681 consensus_peers.cc:597] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:45.025785  1607 consensus_peers.cc:597] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:45.075022  2268 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:45.145685  1681 consensus_peers.cc:597] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:45.204789  1681 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:45.221138  2268 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:45.452967  1607 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:45.458590  2268 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111) [suppressed 431 similar messages]
W20260709 13:04:45.469246  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:45.533350  2268 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:45.628336  2268 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:45.636955  1681 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:45.655288  1607 consensus_peers.cc:597] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:45.688198  1607 consensus_peers.cc:597] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:45.733475   431 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:45.768045  2268 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:45.793936  1681 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:45.825361  1681 consensus_peers.cc:597] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:45.881757  2268 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:45.890275  2268 consensus_peers.cc:597] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:45.952332  1607 consensus_peers.cc:597] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:45.981556  1681 consensus_peers.cc:597] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:46.039279  1681 consensus_peers.cc:597] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:46.052134  2268 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:46.072510  2268 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:46.085927  2268 consensus_peers.cc:597] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:46.164564  1681 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:46.181830  1681 consensus_peers.cc:597] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:46.190689  2268 consensus_peers.cc:597] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:46.208565  1681 consensus_peers.cc:597] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:46.232173  1681 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:46.235607  1681 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:46.255336  1607 consensus_peers.cc:597] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:46.285188  1681 consensus_peers.cc:597] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:46.371805  1681 consensus_peers.cc:597] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:46.427656  1607 consensus_peers.cc:597] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:46.443534  2268 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:46.445894  2268 consensus_peers.cc:597] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:46.510600  1607 consensus_peers.cc:597] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:04:46.687522  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 68370.1us
W20260709 13:04:46.720587  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 88464.9us
W20260709 13:04:46.735173   431 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:46.749006  1607 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:46.764696  1681 consensus_peers.cc:597] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:46.778476  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 56800.5us
W20260709 13:04:46.816890  1607 consensus_peers.cc:597] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:46.872329  2268 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:46.907509  1681 consensus_peers.cc:597] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:46.985123  1607 consensus_peers.cc:597] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:47.131778  1681 consensus_peers.cc:597] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:47.485978  1681 consensus_peers.cc:597] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:47.541631  1607 consensus_peers.cc:597] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:47.568445  1681 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:47.670292  1681 consensus_peers.cc:597] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:47.715173  2268 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:47.735950   431 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:47.863507  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.040920  2268 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.083822  2268 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.119975  2268 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.147400  1607 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.183066  2268 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.238118  1681 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.248427  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 55887.2us
W20260709 13:04:48.292554  1607 consensus_peers.cc:597] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.329128  1681 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:48.329698  1607 consensus_peers.cc:597] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.397814  1681 consensus_peers.cc:597] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.419116  2268 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.497634  2268 consensus_peers.cc:597] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:48.520427  2268 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.607877  1607 consensus_peers.cc:597] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.621603  1681 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.623116  2268 consensus_peers.cc:597] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:48.639421  1681 consensus_peers.cc:597] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.681249  1681 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.736474   431 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:48.745745  1681 consensus_peers.cc:597] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.770040  1681 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.785933  2268 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.800746  2268 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.825871  1681 consensus_peers.cc:597] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.838197  1681 consensus_peers.cc:597] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.838909  1681 consensus_peers.cc:597] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.840160  1607 consensus_peers.cc:597] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:48.854579  1681 consensus_peers.cc:597] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.907121  1607 consensus_peers.cc:597] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:48.920650  2268 consensus_peers.cc:597] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.957566  2268 consensus_peers.cc:597] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:48.989058  1607 consensus_peers.cc:597] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:49.057677  2268 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:49.243860  1607 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:49.302285  1607 consensus_peers.cc:597] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:49.635267  1607 consensus_peers.cc:597] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:49.737017   431 thread.cc:527] Waited for 18000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:50.154698  2268 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:50.277011  1607 consensus_peers.cc:597] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:50.283115  1681 consensus_peers.cc:597] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:50.283928  1681 consensus_peers.cc:597] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:50.297860  1681 consensus_peers.cc:597] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:50.364823  2268 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:50.378512  1607 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:50.383116  1681 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:50.419265  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:50.469924  1681 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111) [suppressed 413 similar messages]
W20260709 13:04:50.637933  2268 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:50.647248  2268 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:50.691944  1607 consensus_peers.cc:597] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:50.737488   431 thread.cc:527] Waited for 19000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:50.789731  2268 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:50.802366  1607 consensus_peers.cc:597] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:50.812017  1681 consensus_peers.cc:597] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:50.856204  2268 consensus_peers.cc:597] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:50.951877  1607 consensus_peers.cc:597] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:51.001343  1681 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:51.004508  2268 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:51.012151  2268 consensus_peers.cc:597] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:51.093302  2268 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:51.108146  1681 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.142714  1681 consensus_peers.cc:597] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:51.149993  2268 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:51.210505  2268 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:51.287051  1607 consensus_peers.cc:597] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:51.291213  2268 consensus_peers.cc:597] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:51.310361  1681 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:51.414243  2268 consensus_peers.cc:597] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:51.466349  1607 consensus_peers.cc:597] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:51.491026  1681 consensus_peers.cc:597] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:51.610081  1681 consensus_peers.cc:597] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:51.738098   431 thread.cc:527] Waited for 20000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:52.499084  1683 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.004s	sys 0.016s
W20260709 13:04:52.499873  1683 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.005s	sys 0.016s
W20260709 13:04:52.657019  1451 catalog_manager.cc:4755] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3f22f7f5cede49bea2083e8905932fb7: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:52.680701  1608 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.056s	user 0.003s	sys 0.022s
W20260709 13:04:52.681236  1608 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.057s	user 0.004s	sys 0.023s
I20260709 13:04:52.701352  9533 raft_consensus.cc:493] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
W20260709 13:04:52.705451  1609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.082s	user 0.033s	sys 0.015s
W20260709 13:04:52.707607  1609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.084s	user 0.033s	sys 0.018s
W20260709 13:04:52.708767  1609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 1.08474e+06us
I20260709 13:04:52.715222  9534 raft_consensus.cc:493] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:52.702351  9533 raft_consensus.cc:515] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.716066  9534 raft_consensus.cc:515] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.718631  9534 leader_election.cc:290] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:52.728057  9533 leader_election.cc:290] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:52.736672  2270 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.064s	user 0.008s	sys 0.019s
W20260709 13:04:52.737453  2270 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.065s	user 0.008s	sys 0.019s
W20260709 13:04:52.738737   431 thread.cc:527] Waited for 21000ms trying to join with diag-logger (tid 1590)
I20260709 13:04:52.742110  9537 raft_consensus.cc:493] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:52.744843  9534 raft_consensus.cc:493] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:52.745615  9534 raft_consensus.cc:515] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.742695  9537 raft_consensus.cc:515] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
W20260709 13:04:52.749203  2268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.075s	user 0.018s	sys 0.021s
W20260709 13:04:52.749804  2268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.075s	user 0.018s	sys 0.021s
I20260709 13:04:52.750546  9537 leader_election.cc:290] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:52.752225  9534 leader_election.cc:290] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:52.760416  9534 raft_consensus.cc:493] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:52.761107  9534 raft_consensus.cc:515] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.763477  9534 leader_election.cc:290] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:52.767266  2271 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.096s	user 0.003s	sys 0.019s
W20260709 13:04:52.769979  2271 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.099s	user 0.003s	sys 0.019s
I20260709 13:04:52.781345  9532 raft_consensus.cc:493] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:52.781910  9531 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:52.784348  9531 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:52.806861  9531 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:04:52.818993  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 55771.1us
I20260709 13:04:52.804811  9532 raft_consensus.cc:515] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:52.828426  9532 leader_election.cc:290] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:52.831269  9533 raft_consensus.cc:493] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:52.831946  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a08948b1fac416d8d27be0561a00db4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:52.831933  9533 raft_consensus.cc:515] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.836477  9532 raft_consensus.cc:493] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:52.859972  9547 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:52.860562  9534 raft_consensus.cc:493] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:52.872558  9538 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:52.873294  9538 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:52.875336  9536 raft_consensus.cc:493] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:52.876032  9536 raft_consensus.cc:515] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.877558  9543 raft_consensus.cc:493] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:52.878468  9543 raft_consensus.cc:515] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.879292  9539 raft_consensus.cc:493] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:52.879928  9539 raft_consensus.cc:515] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.890010  9548 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:52.890762  9548 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.891031  9534 raft_consensus.cc:515] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.893126  9534 leader_election.cc:290] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:52.893802  9548 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:52.862349  9532 raft_consensus.cc:515] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.894977  9533 leader_election.cc:290] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:52.895733  9536 leader_election.cc:290] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:52.895887  9532 leader_election.cc:290] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:52.897040  9539 leader_election.cc:290] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:52.897506  9538 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:52.898099  9547 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.900867  9538 raft_consensus.cc:493] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:52.901533  9538 raft_consensus.cc:515] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.921851  9554 raft_consensus.cc:493] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:52.922479  9554 raft_consensus.cc:515] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:52.937031  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 45343.8us
I20260709 13:04:52.958104  9544 raft_consensus.cc:493] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:52.889966  9543 leader_election.cc:290] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:52.958823  9544 raft_consensus.cc:515] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.903039  9548 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:52.960099  9548 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:52.962293  9548 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:52.959232  9547 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:52.936429  9539 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:52.963528  9539 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.964560  9545 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:52.965222  9545 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.967811  9545 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:52.967859  9539 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:52.968536  9538 leader_election.cc:290] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:52.969067  9544 leader_election.cc:290] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:52.971172  9544 raft_consensus.cc:493] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:52.971747  9544 raft_consensus.cc:515] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.977092  9544 leader_election.cc:290] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:52.978291  9555 raft_consensus.cc:493] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:52.979561  9533 raft_consensus.cc:493] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:52.979705  9555 raft_consensus.cc:515] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.980180  9533 raft_consensus.cc:515] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:52.982082  9555 leader_election.cc:290] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:52.982368  9533 leader_election.cc:290] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:04:52.986523  1681 consensus_peers.cc:597] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:04:52.987171  9550 raft_consensus.cc:493] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:52.979043  9538 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:53.003345  9538 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:04:53.006016  9538 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:52.987883  9548 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:52.989948  9547 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:52.991879  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.006976  9547 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.010239  9547 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:53.010861  9548 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.013031  9548 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:52.994097  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:52.994987  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33657ac7590d4739b9bda3f7bb345c7b" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:52.996515  9533 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:52.997458  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:52.997681  9543 raft_consensus.cc:493] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:53.015107  9533 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.015738  9543 raft_consensus.cc:515] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.017637  9533 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:53.017948  9543 leader_election.cc:290] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:52.998003  9545 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:52.998371  9546 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:53.019311  9545 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:52.999650  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 60866.4us
I20260709 13:04:53.019348  9546 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.021600  9546 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:52.998564  9539 raft_consensus.cc:493] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:53.022327  9539 raft_consensus.cc:515] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.023919  9550 raft_consensus.cc:515] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.024410  9539 leader_election.cc:290] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:53.021602  9545 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:53.026712  9550 leader_election.cc:290] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:53.032836  9553 raft_consensus.cc:493] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:53.033581  9553 raft_consensus.cc:515] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.036768  9553 leader_election.cc:290] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:04:53.041589  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 52863.3us
I20260709 13:04:53.044626  9545 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:53.045276  9545 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.047384  9545 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:53.048358  9539 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:53.048449  9546 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:53.048964  9539 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.049082  9546 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.052877  9545 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:53.053018  9546 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:53.053452  9545 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.053828  9546 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:53.055392  9539 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:53.055655  9545 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:53.054533  9546 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.071867  9546 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:53.072103  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d8055bc87c4624a96f13638fe667e6" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.070637  9539 raft_consensus.cc:493] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:53.073084  9539 raft_consensus.cc:515] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.080415  9539 leader_election.cc:290] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:53.081422  2302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e353d5ccbb3478ba62d1d60ecc3ec05" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.084023  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.086043  2296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5eae102421c4bc4b19781bf4470a0b0" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.078862  2301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc96896acb4fcf8b5ec5b7206248ca" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.086999  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "403bcba338a042c1b5fecfe8ac430922" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.082046  9553 raft_consensus.cc:493] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
I20260709 13:04:53.088054  9553 raft_consensus.cc:515] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.088449  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "768b897f18a644e8b1e8f8ed1c8b616f" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.089563  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408b4ec829c54266b41334aac8245405" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.090265  9553 leader_election.cc:290] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:04:53.091643  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d59c4d21b6cb46edbfc2b3d50b51812a" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.092072  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62e41b6e9a54f26a1c7c1538bdf5334" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.097400  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.097998  1709 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 23.
W20260709 13:04:53.099980  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 77771.6us
I20260709 13:04:53.109483  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "213d9f9af66a4ebd8c9a7b0163196a57" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.112212  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.112576  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3905c76f9588440c8a836e5786f6667e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.114688  1639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:53.114786  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 118963us
I20260709 13:04:53.119266  9561 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:53.119398  9554 leader_election.cc:290] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:53.119894  9561 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.123306  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29864f650d0477e8dd2e7f82eea9e47" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.118654  9549 raft_consensus.cc:493] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6208c2fa49f643b5a50ab8f6d292fab2)
I20260709 13:04:53.125077  9549 raft_consensus.cc:515] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.127103  9561 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:53.134491  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f712f9c188e44e588e739161cac39330" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.138298  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4760a45328493e87523c2ee8292599" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.130445  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9b7b078d0d47d3880d3dec3230b779" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.140728  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.128683  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.140734  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.123191  9557 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:53.132290  9549 leader_election.cc:290] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:53.156440  9557 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:53.161080  2268 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:53.172921  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7808a4bebf84e598198ed51ea156981" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:53.177943  1681 leader_election.cc:336] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.178515  1715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:53.178751  1681 leader_election.cc:336] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.162041  2304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37b565cd27a64e96addb09389c2c7d5e" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.163641  1710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a0a5f8de344870be97d83cec17c047" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.163709  1709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5ccce7ae459467a984bd58d73f701ae" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.165558  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb294ac919b44e56960745f79e83dfc4" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.181150  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82743dc3081f4edba18591ca4b0d9c09" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.182155  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.182376  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdefe70d276420ca0283414ab16fab9" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.182668  2298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.166877  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.167348  1712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d4f9d3ff1a04fcbba1e77b6fa3e6358" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.167739  1714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.167958  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.185039  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.168591  1713 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.169843  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ba36df4f014c3a916b0393305ea44a" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.169914  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a2b88a149f424591686a115bc1f0d6" candidate_uuid: "5184c101be204393adf3ac8b31f25ad6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.171795  9557 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:53.191223  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.191603  1683 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.192512  9549 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:53.192755  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
W20260709 13:04:53.146240  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 117793us
I20260709 13:04:53.194253  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c85446f87564ae6865060bb988ee1b8" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.195456  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.179387  1681 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.198396  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.199594  9557 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:53.203903  1683 leader_election.cc:304] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.205066  9561 raft_consensus.cc:2749] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:53.206243  1683 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.207216  9549 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:53.207451  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.209010  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1256ff04335423286d7c081ec776061" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.210393  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.214259  9549 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:53.216315  1635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:53.216781  1681 consensus_peers.cc:597] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:53.217389  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:53.217682  1681 leader_election.cc:336] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.218218  1681 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.219379  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.219485  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0049d6871cf4f95be599108abda58a5" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.220304  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7e7c3881a44475827c9cbca08a2fe1" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:53.221738  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca83cd5e36eb4e22b75011b87bfcd49d" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.223178  1636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.223944  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c23f0e4f742e38ca7a5ebd7778dfa" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:53.192605  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d883c107e7bd4b6daa6d09672acc2b23" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.225493  9549 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:53.226714  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:53.229562  2268 leader_election.cc:304] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.230614  1682 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:53.231956  1681 leader_election.cc:336] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.235430  9554 raft_consensus.cc:2749] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:04:53.238986  1681 leader_election.cc:336] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.239665  9549 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:53.245848  1681 leader_election.cc:304] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.249408  9557 raft_consensus.cc:2749] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:53.253533  2268 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.255878  9546 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:53.256297  9546 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:53.256655  9546 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:53.244534  1607 leader_election.cc:304] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.248526  1682 leader_election.cc:304] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.260424  9532 raft_consensus.cc:2749] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:53.262166  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.272083  9560 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:04:53.273058  2268 leader_election.cc:304] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.274408  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.265483  9549 raft_consensus.cc:2749] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:53.266697  1682 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.275817  9549 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:53.276026  9555 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:53.277189  2268 leader_election.cc:304] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.278456  9555 raft_consensus.cc:2749] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:04:53.279784  2268 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.274991  9560 raft_consensus.cc:2749] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:04:53.264338  1607 consensus_peers.cc:597] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:53.285017  1607 consensus_peers.cc:597] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:04:53.285806  1607 leader_election.cc:336] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.286314  9560 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:04:53.287899  1607 leader_election.cc:304] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.287016  2268 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:53.291201  1681 leader_election.cc:336] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.291395  9560 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:53.270045  9546 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.292366  9532 raft_consensus.cc:2749] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:53.295601  2268 leader_election.cc:304] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:53.294641  1607 leader_election.cc:336] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.296514  1607 leader_election.cc:304] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.297302  9560 raft_consensus.cc:2749] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:53.297402  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:53.298686  2268 leader_election.cc:304] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.299345  9532 raft_consensus.cc:2749] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:04:53.299998  1681 leader_election.cc:304] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.305009  9560 raft_consensus.cc:2749] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:04:53.305403  9549 raft_consensus.cc:2749] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:04:53.300014  1607 leader_election.cc:336] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.306056  1607 leader_election.cc:304] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:53.307360  9532 raft_consensus.cc:2749] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:53.309284  1607 leader_election.cc:336] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.313917  1607 leader_election.cc:304] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.304221  9546 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:53.310875  1711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f22f7f5cede49bea2083e8905932fb7" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:04:53.309460  1681 leader_election.cc:336] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.331203  1681 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:04:53.316167  1607 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:04:53.319873  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
W20260709 13:04:53.335358  1607 leader_election.cc:336] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.335865  1607 leader_election.cc:304] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:53.336809  1607 consensus_peers.cc:597] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:53.337335  1607 leader_election.cc:336] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.337723  1607 leader_election.cc:304] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:53.321813  2268 leader_election.cc:336] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.338514  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:53.339692  2268 leader_election.cc:336] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.340094  2268 leader_election.cc:304] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:53.341106  2268 leader_election.cc:336] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.341490  2268 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.314757  9532 raft_consensus.cc:2749] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:53.342783  9560 raft_consensus.cc:2749] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:53.342638  9549 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:53.343504  9560 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:53.342638  9555 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:04:53.344955  1681 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:53.345214  9513 raft_consensus.cc:2749] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:53.345556  1681 leader_election.cc:336] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.345918  9513 raft_consensus.cc:2749] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:04:53.347368  1607 leader_election.cc:336] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.349638  1607 leader_election.cc:304] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:53.348395  2268 leader_election.cc:336] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.350843  2268 leader_election.cc:304] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:04:53.351647  9541 raft_consensus.cc:2749] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:04:53.353645  9555 raft_consensus.cc:2749] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:04:53.354714  2268 leader_election.cc:336] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:53.352421  1607 leader_election.cc:336] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.355778  1607 leader_election.cc:304] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.356537  9541 raft_consensus.cc:2749] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:04:53.345949  1681 leader_election.cc:304] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.358673  9561 raft_consensus.cc:2749] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:04:53.361336  1681 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.361784  1681 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.355242  2268 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.362491  9561 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:04:53.364265  9555 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:53.363095  1681 leader_election.cc:336] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:53.365159  2268 leader_election.cc:336] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.365384  1681 leader_election.cc:304] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.365638  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.366114  9561 raft_consensus.cc:2749] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:04:53.367861  2268 leader_election.cc:336] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.367960  9555 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:04:53.368289  2268 leader_election.cc:304] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
W20260709 13:04:53.387069  2268 leader_election.cc:336] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.388096  2268 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.392144  9555 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:04:53.399151  1681 consensus_peers.cc:597] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:53.399899  1681 consensus_peers.cc:597] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:53.402594  1607 leader_election.cc:336] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.403764  1607 leader_election.cc:304] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6; no voters: 3f64b76180a54492bb015e405caeacec, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:53.404835  9536 raft_consensus.cc:2749] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:04:53.389058  9546 raft_consensus.cc:2749] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:04:53.411259  2268 leader_election.cc:336] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:04:53.411818  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.422415  9546 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:04:53.423666  2268 leader_election.cc:336] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:04:53.437597  1681 consensus_peers.cc:597] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:04:53.446578  2268 leader_election.cc:304] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.447566  9546 raft_consensus.cc:2749] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:04:53.448918  2268 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:04:53.449734  9546 raft_consensus.cc:2749] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:04:53.493607  1681 consensus_peers.cc:597] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:53.578020  1681 consensus_peers.cc:597] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:53.612519  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:53.639400  1607 consensus_peers.cc:597] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:53.650887  2305 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Rejecting Update request from peer 5184c101be204393adf3ac8b31f25ad6 for earlier term 23. Current term is 24. Ops: []
I20260709 13:04:53.652721  9536 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:53.653826  9536 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:04:53.654162  9536 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 5184c101be204393adf3ac8b31f25ad6, State: Running, Role: LEADER
I20260709 13:04:53.655054  9536 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:53.656492  9536 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:04:53.726577  1681 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:53.739327   431 thread.cc:527] Waited for 22000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:53.773214  2326 debug-util.cc:398] Leaking SignalData structure 0x7b0800415360 after lost signal to thread 434
W20260709 13:04:53.774398  2326 debug-util.cc:398] Leaking SignalData structure 0x7b0800361580 after lost signal to thread 1501
W20260709 13:04:53.775727  2326 debug-util.cc:398] Leaking SignalData structure 0x7b0800343ca0 after lost signal to thread 1668
W20260709 13:04:53.776857  2326 debug-util.cc:398] Leaking SignalData structure 0x7b0800128560 after lost signal to thread 1742
W20260709 13:04:53.778010  2326 debug-util.cc:398] Leaking SignalData structure 0x7b080005a5a0 after lost signal to thread 2329
W20260709 13:04:53.786254  2268 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:53.939632  1607 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:54.100450  1682 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 50911.1us
W20260709 13:04:54.150439  1681 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:54.188980  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 63949.6us
W20260709 13:04:54.269641  1681 consensus_peers.cc:597] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:54.287814  1607 consensus_peers.cc:597] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:54.290759  2268 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:54.373354  1681 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:54.381022  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 52098.7us
W20260709 13:04:54.459301  1681 consensus_peers.cc:597] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:54.486413  2268 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:54.487251  2268 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:54.574055  1607 consensus_peers.cc:597] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:54.576447  2268 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:54.610539  2268 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:54.621765  2268 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:54.641852  2268 consensus_peers.cc:597] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:54.659849  1681 consensus_peers.cc:597] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:54.660596  2268 consensus_peers.cc:597] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:54.739843   431 thread.cc:527] Waited for 23000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:54.822134  1681 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:54.842753  1681 consensus_peers.cc:597] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:54.873588  1681 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:54.895187  1607 consensus_peers.cc:597] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:54.904063  9566 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:54.904834  9566 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:54.908551  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
I20260709 13:04:54.909312  1641 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 24.
I20260709 13:04:54.911558  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:04:54.914089  2305 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 24: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:54.913277  9566 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:54.916340  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:04:54.917511  9566 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:04:54.917968  9566 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:04:54.918414  9566 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:04:54.927729  9566 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:54.940198  2305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:04:54.940282  9566 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:04:54.941049  2305 raft_consensus.cc:2393] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 24: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:04:54.945278  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:54.967483  1640 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 24.
I20260709 13:04:54.980165  1682 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6; no voters: 6208c2fa49f643b5a50ab8f6d292fab2
W20260709 13:04:54.978557  1681 consensus_peers.cc:597] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:04:54.981456  9565 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:04:54.982004  9565 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:54.984946  9565 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:55.027717  1607 consensus_peers.cc:597] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:55.036509  1462 catalog_manager.cc:5697] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 23 to 24, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 24 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:55.063498  2268 consensus_peers.cc:597] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:55.117566  2268 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:55.140202  2268 consensus_peers.cc:597] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:55.144395  2268 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:55.184769  9571 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.185362  9571 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:04:55.198633  1681 consensus_peers.cc:597] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:55.200490  9571 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:55.201952  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:04:55.203595  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6" is_pre_election: true
W20260709 13:04:55.210565  1607 consensus_peers.cc:597] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:55.210521  1640 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 24.
I20260709 13:04:55.213019  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:55.215095  9567 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:04:55.215533  9567 raft_consensus.cc:493] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.215893  9567 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:55.222245  2268 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:55.246335  9567 raft_consensus.cc:515] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:55.277904  1641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5184c101be204393adf3ac8b31f25ad6"
I20260709 13:04:55.278141  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344c88e0be9640f583140fb4aab3af58" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:04:55.278559  1641 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:04:55.298717  1641 raft_consensus.cc:2468] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 25.
I20260709 13:04:55.300184  2269 leader_election.cc:304] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5
I20260709 13:04:55.301131  9546 raft_consensus.cc:2804] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:04:55.304004  9546 raft_consensus.cc:697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:04:55.306288  9546 consensus_queue.cc:237] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:55.280874  9567 leader_election.cc:290] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 25 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:04:55.359759  1458 catalog_manager.cc:5697] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 24 to 25, leader changed from 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 25 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:55.409523  2297 raft_consensus.cc:1240] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 LEADER]: Rejecting Update request from peer 28bc59d37f664ce0a6f5784f2619f0b5 for earlier term 24. Current term is 25. Ops: []
W20260709 13:04:55.411607  1607 consensus_peers.cc:597] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:04:55.413625  9576 consensus_queue.cc:1059] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:04:55.414683  9576 consensus_queue.cc:1243] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Peer 6208c2fa49f643b5a50ab8f6d292fab2 log is divergent from this leader: its last log entry 25.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:04:55.415189  9566 raft_consensus.cc:3055] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:04:55.415539  9566 raft_consensus.cc:740] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:04:55.416335  9566 consensus_queue.cc:260] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:04:55.417414  9566 raft_consensus.cc:3060] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:04:55.426591  1607 consensus_peers.cc:597] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:55.463794  1681 consensus_peers.cc:597] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:55.473308  2268 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111) [suppressed 374 similar messages]
W20260709 13:04:55.499838  1681 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:55.568339  1607 consensus_peers.cc:597] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:55.686656  1607 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:04:55.740370   431 thread.cc:527] Waited for 24000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:55.774498  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33997, user_credentials={real_user=slave}} blocked reactor thread for 63093.3us
W20260709 13:04:55.782058  1681 consensus_peers.cc:597] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:55.789798  1711 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 17. (term mismatch)
I20260709 13:04:55.790299  1711 pending_rounds.cc:85] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting all ops after (but not including) 16
I20260709 13:04:55.790529  1711 pending_rounds.cc:107] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: Aborting uncommitted NO_OP operation due to leader change: 24.17
I20260709 13:04:55.790874  1711 raft_consensus.cc:2889] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:55.869717  9546 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:04:55.914106  1607 consensus_peers.cc:597] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:55.929256  1681 consensus_peers.cc:597] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:55.930943  1681 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 57749.3us
W20260709 13:04:55.961522  1681 consensus_peers.cc:597] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:55.962373  1641 raft_consensus.cc:1275] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 13:04:55.971702  1681 consensus_peers.cc:597] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:04:55.971644  9571 consensus_queue.cc:1048] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:04:56.126276  1681 consensus_peers.cc:597] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:56.129732  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:56.146576  2268 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:56.216782  1681 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:56.423489  1607 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:56.568276  1681 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:56.583053  2268 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:56.849012   431 thread.cc:527] Waited for 25000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:56.923517  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 195653us
W20260709 13:04:56.991564  1681 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:57.002718  2269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:40023, user_credentials={real_user=slave}} blocked reactor thread for 77816.2us
W20260709 13:04:57.043172  1681 consensus_peers.cc:597] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:57.044142  1681 consensus_peers.cc:597] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:57.109515  1683 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:38715, user_credentials={real_user=slave}} blocked reactor thread for 56933.9us
W20260709 13:04:57.126572  1681 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:04:57.137595  1607 consensus_peers.cc:597] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:57.141804  1607 consensus_peers.cc:597] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:04:57.150053  1456 maintenance_manager.cc:419] P 6ffeec09a12e4bd6997354c8ecdff02d: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033347
W20260709 13:04:57.166530  2268 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:57.176587  2268 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:57.207521  1681 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:04:57.251852  1455 maintenance_manager.cc:643] P 6ffeec09a12e4bd6997354c8ecdff02d: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.101s	user 0.049s	sys 0.019s Metrics: {"bytes_written":7788,"delete_count":0,"lbm_write_time_us":1246,"lbm_writes_lt_1ms":7,"reinsert_count":0,"thread_start_us":25280,"threads_started":1,"update_count":300,"wal-append.queue_time_us":25659}
I20260709 13:04:57.254068  1456 maintenance_manager.cc:419] P 6ffeec09a12e4bd6997354c8ecdff02d: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.371100
W20260709 13:04:57.272123  2268 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:57.304594  2268 consensus_peers.cc:597] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:04:57.375037  1455 maintenance_manager.cc:643] P 6ffeec09a12e4bd6997354c8ecdff02d: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.120s	user 0.097s	sys 0.013s Metrics: {"cfile_cache_miss":10,"cfile_cache_miss_bytes":28938,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":8280,"dirs.run_cpu_time_us":631,"dirs.run_wall_time_us":1693,"lbm_read_time_us":1916,"lbm_reads_lt_1ms":26,"lbm_write_time_us":1610,"lbm_writes_lt_1ms":12,"peak_mem_usage":311722,"reinsert_count":0,"thread_start_us":14844,"threads_started":2,"update_count":309}
I20260709 13:04:57.377683  1456 maintenance_manager.cc:419] P 6ffeec09a12e4bd6997354c8ecdff02d: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 13:04:57.378587  1455 maintenance_manager.cc:643] P 6ffeec09a12e4bd6997354c8ecdff02d: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
W20260709 13:04:57.475193  2268 consensus_peers.cc:597] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:57.577941  1681 consensus_peers.cc:597] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:57.630867  2268 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:57.631651  2268 consensus_peers.cc:597] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:57.681231  2268 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:57.691145  1681 consensus_peers.cc:597] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:57.691915  1681 consensus_peers.cc:597] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:57.692577  1681 consensus_peers.cc:597] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:57.694483  1607 consensus_peers.cc:597] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:57.694644  2268 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:57.731707  2268 consensus_peers.cc:597] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:57.741957  2268 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:57.760151  1607 consensus_peers.cc:597] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:04:57.831030  1665 debug-util.cc:398] Leaking SignalData structure 0x7b0800218280 after lost signal to thread 434
W20260709 13:04:57.832331  1665 debug-util.cc:398] Leaking SignalData structure 0x7b0800218700 after lost signal to thread 1501
W20260709 13:04:57.833777  1665 debug-util.cc:398] Leaking SignalData structure 0x7b08003cbf60 after lost signal to thread 1668
W20260709 13:04:57.834895  1665 debug-util.cc:398] Leaking SignalData structure 0x7b080005a400 after lost signal to thread 1742
W20260709 13:04:57.836283  1665 debug-util.cc:398] Leaking SignalData structure 0x7b08000617a0 after lost signal to thread 2329
W20260709 13:04:57.850126   431 thread.cc:527] Waited for 26000ms trying to join with diag-logger (tid 1590)
W20260709 13:04:57.878949  1607 consensus_peers.cc:597] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:58.001837  1607 consensus_peers.cc:597] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:58.069739  1681 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:58.347442  1681 consensus_peers.cc:597] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:58.349045  1607 consensus_peers.cc:597] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:04:58.359750  1681 consensus_peers.cc:597] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:58.387606  2268 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:58.394384  1607 consensus_peers.cc:597] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:04:58.403908  1681 consensus_peers.cc:597] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:58.406932  1607 consensus_peers.cc:597] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:58.583030  1607 consensus_peers.cc:597] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:58.668722  1681 consensus_peers.cc:597] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:58.669556  1681 consensus_peers.cc:597] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:58.805572  2268 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 50957.4us
W20260709 13:04:58.804267  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:36547, user_credentials={real_user=slave}} blocked reactor thread for 50657.8us
W20260709 13:04:58.893597  2268 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:58.916762  1681 consensus_peers.cc:597] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:59.037470  1681 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:04:59.069173   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:04:59.077304  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:59.080986  1607 consensus_peers.cc:597] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:59.099054  1681 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:59.175230  2268 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:59.242854  1681 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:59.280593  1681 consensus_peers.cc:597] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:04:59.295526  1607 consensus_peers.cc:597] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:59.296597  1607 consensus_peers.cc:597] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:04:59.347563   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:04:59.356840  2269 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.359354  1682 consensus_peers.cc:597] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.404461  1681 consensus_peers.cc:597] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:04:59.412616  1682 consensus_peers.cc:597] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.414260  1682 consensus_peers.cc:597] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.417966  1682 consensus_peers.cc:597] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.433634   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:59.434983   431 tablet_replica.cc:333] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.436180   431 raft_consensus.cc:2243] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.437772   431 raft_consensus.cc:2272] T 5448ccc31ba14113b3d56168031c6714 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.440995   431 tablet_replica.cc:333] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.442256   431 raft_consensus.cc:2243] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.443388   431 raft_consensus.cc:2272] T 250e1f9f055e4fda9987138306f2e0b7 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.452680   431 tablet_replica.cc:333] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.453778   431 raft_consensus.cc:2243] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.454996   431 raft_consensus.cc:2272] T bb58673aa80747e58aa935f4fed2212d P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.468789   431 tablet_replica.cc:333] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.469866   431 raft_consensus.cc:2243] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.474143   431 raft_consensus.cc:2272] T d883c107e7bd4b6daa6d09672acc2b23 P 5184c101be204393adf3ac8b31f25ad6 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:59.480320  2269 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.482909   431 tablet_replica.cc:333] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
W20260709 13:04:59.486982  2269 consensus_peers.cc:597] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.487849   431 raft_consensus.cc:2243] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.490255   431 raft_consensus.cc:2272] T 3905c76f9588440c8a836e5786f6667e P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.498626   431 tablet_replica.cc:333] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.499630   431 raft_consensus.cc:2243] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.500857   431 raft_consensus.cc:2272] T 50d8055bc87c4624a96f13638fe667e6 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.506542   431 tablet_replica.cc:333] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
W20260709 13:04:59.510588  2269 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.512645   431 raft_consensus.cc:2243] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.513913   431 raft_consensus.cc:2272] T 5a08948b1fac416d8d27be0561a00db4 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.517163   431 tablet_replica.cc:333] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.518019   431 raft_consensus.cc:2243] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.519140   431 raft_consensus.cc:2272] T 71b0b66754d049cfbccb38fbb16ef3cd P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.521713   431 tablet_replica.cc:333] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.522537   431 raft_consensus.cc:2243] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.523559   431 raft_consensus.cc:2272] T 4c0f61db23f34217bd5089a459cbb914 P 5184c101be204393adf3ac8b31f25ad6 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.526482   431 tablet_replica.cc:333] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.527422   431 raft_consensus.cc:2243] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.528297   431 raft_consensus.cc:2272] T 92a2b88a149f424591686a115bc1f0d6 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.531172   431 tablet_replica.cc:333] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.532012   431 raft_consensus.cc:2243] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.532827   431 raft_consensus.cc:2272] T 9f6815b361554c338159f274bd78c0e9 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:59.535868  1681 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:59.540663  2269 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.536149   431 tablet_replica.cc:333] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.541896   431 raft_consensus.cc:2243] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.542712   431 raft_consensus.cc:2272] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.546769   431 tablet_replica.cc:333] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.547729   431 raft_consensus.cc:2243] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.548460   431 raft_consensus.cc:2272] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:59.550052  1682 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.550733  1682 consensus_peers.cc:597] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.557353  2268 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:04:59.558499   431 tablet_replica.cc:333] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.559499   431 raft_consensus.cc:2243] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.560772   431 raft_consensus.cc:2272] T e6dc6e12b7924255b756439991155e88 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.563571   431 tablet_replica.cc:333] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.565685   431 raft_consensus.cc:2243] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:04:59.569259  2269 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.567407   431 raft_consensus.cc:2272] T d29864f650d0477e8dd2e7f82eea9e47 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.572922   431 tablet_replica.cc:333] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.573830   431 raft_consensus.cc:2243] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:59.573966  1682 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.574687  1682 consensus_peers.cc:597] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.574808   431 raft_consensus.cc:2272] T 520f39ecc282462782487a3651e28b44 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:59.575362  1682 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.576010  1682 consensus_peers.cc:597] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.577466   431 tablet_replica.cc:333] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.578176   431 raft_consensus.cc:2243] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.579047   431 raft_consensus.cc:2272] T 408b4ec829c54266b41334aac8245405 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.581337   431 tablet_replica.cc:333] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.582052   431 raft_consensus.cc:2243] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.582974   431 raft_consensus.cc:2272] T 9310eb7a2c8447e28c9096c983ddf5b5 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.585798   431 tablet_replica.cc:333] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.586609   431 raft_consensus.cc:2243] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.587280   431 raft_consensus.cc:2272] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.589985   431 tablet_replica.cc:333] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.590809   431 raft_consensus.cc:2243] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.592413   431 raft_consensus.cc:2272] T d1256ff04335423286d7c081ec776061 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:59.599916  1681 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:04:59.607779   431 tablet_replica.cc:333] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.608680   431 raft_consensus.cc:2243] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.609462   431 raft_consensus.cc:2272] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.618342   431 tablet_replica.cc:333] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.619128   431 raft_consensus.cc:2243] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.620860   431 raft_consensus.cc:2272] T 2792e3e29993414a8d6f55078ae13994 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.624153   431 tablet_replica.cc:333] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.626376   431 raft_consensus.cc:2243] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.627673   431 raft_consensus.cc:2272] T 33657ac7590d4739b9bda3f7bb345c7b P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.630419   431 tablet_replica.cc:333] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.631201   431 raft_consensus.cc:2243] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:59.631261  2269 consensus_peers.cc:597] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.632153   431 raft_consensus.cc:2272] T c129438e9d58493a93890208b25765cf P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.634811   431 tablet_replica.cc:333] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.635618   431 raft_consensus.cc:2243] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.636221   431 raft_consensus.cc:2272] T d4bc96896acb4fcf8b5ec5b7206248ca P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.645295   431 tablet_replica.cc:333] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.646196   431 raft_consensus.cc:2243] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.647537   431 raft_consensus.cc:2272] T 3f22f7f5cede49bea2083e8905932fb7 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:59.647930  2269 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.652135  2269 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.652858   431 tablet_replica.cc:333] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.663048   431 raft_consensus.cc:2243] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.666332   431 raft_consensus.cc:2272] T c46cf090bf9b4084b648d00f5f0053e2 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:59.669438  1682 consensus_peers.cc:597] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.670147  1682 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.671360   431 tablet_replica.cc:333] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
W20260709 13:04:59.672541  2269 consensus_peers.cc:597] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.672644   431 raft_consensus.cc:2243] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.673725   431 raft_consensus.cc:2272] T 9df7dc71f943412e83fbc8d4f788c9f2 P 5184c101be204393adf3ac8b31f25ad6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.675894   431 tablet_replica.cc:333] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.676628   431 raft_consensus.cc:2243] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.677436   431 raft_consensus.cc:2272] T 403bcba338a042c1b5fecfe8ac430922 P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.680888   431 tablet_replica.cc:333] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.681638   431 raft_consensus.cc:2243] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.683241   431 raft_consensus.cc:2272] T c2832827f30046328715699fa6ee0c52 P 5184c101be204393adf3ac8b31f25ad6 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.685467   431 tablet_replica.cc:333] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.686065   431 raft_consensus.cc:2243] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.686795   431 raft_consensus.cc:2272] T fbfadedb9a5e4ca0abc25384a524c546 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.689018   431 tablet_replica.cc:333] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.689599   431 raft_consensus.cc:2243] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.690526   431 raft_consensus.cc:2272] T 8a4760a45328493e87523c2ee8292599 P 5184c101be204393adf3ac8b31f25ad6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.692915   431 tablet_replica.cc:333] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.693656   431 raft_consensus.cc:2243] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.694275   431 raft_consensus.cc:2272] T e62e41b6e9a54f26a1c7c1538bdf5334 P 5184c101be204393adf3ac8b31f25ad6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.697187   431 tablet_replica.cc:333] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.698089   431 raft_consensus.cc:2243] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.699277   431 raft_consensus.cc:2272] T 2b9b7b078d0d47d3880d3dec3230b779 P 5184c101be204393adf3ac8b31f25ad6 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.704723   431 tablet_replica.cc:333] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.705494   431 raft_consensus.cc:2243] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 18 LEADER]: Raft consensus shutting down.
W20260709 13:04:59.706012  1682 consensus_peers.cc:597] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.707398   431 raft_consensus.cc:2272] T ca83cd5e36eb4e22b75011b87bfcd49d P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.710311   431 tablet_replica.cc:333] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.711131   431 raft_consensus.cc:2243] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.711838   431 raft_consensus.cc:2272] T 6711207c14044518a57e7c8914ac0091 P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.716331   431 tablet_replica.cc:333] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.718802   431 raft_consensus.cc:2243] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.719806   431 raft_consensus.cc:2272] T 147a8e85db5b4380bf845a82736ec5fd P 5184c101be204393adf3ac8b31f25ad6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.722681   431 tablet_replica.cc:333] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.723429   431 raft_consensus.cc:2243] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.724545   431 raft_consensus.cc:2272] T 8c85446f87564ae6865060bb988ee1b8 P 5184c101be204393adf3ac8b31f25ad6 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:59.727044  1682 consensus_peers.cc:597] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.727248   431 tablet_replica.cc:333] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.728367   431 raft_consensus.cc:2243] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.729952   431 raft_consensus.cc:2272] T d0049d6871cf4f95be599108abda58a5 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.732709   431 tablet_replica.cc:333] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.733538   431 raft_consensus.cc:2243] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.734872   431 raft_consensus.cc:2272] T 1b7e7c3881a44475827c9cbca08a2fe1 P 5184c101be204393adf3ac8b31f25ad6 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.738320   431 tablet_replica.cc:333] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.739428   431 raft_consensus.cc:2243] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.740198   431 raft_consensus.cc:2272] T c5eae102421c4bc4b19781bf4470a0b0 P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:59.742631  2269 consensus_peers.cc:597] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.743314   431 tablet_replica.cc:333] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.743993   431 raft_consensus.cc:2243] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.744572   431 raft_consensus.cc:2272] T 344c88e0be9640f583140fb4aab3af58 P 5184c101be204393adf3ac8b31f25ad6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.748330   431 tablet_replica.cc:333] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.749019   431 raft_consensus.cc:2243] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.751174   431 raft_consensus.cc:2272] T f712f9c188e44e588e739161cac39330 P 5184c101be204393adf3ac8b31f25ad6 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.754846   431 tablet_replica.cc:333] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.755542   431 raft_consensus.cc:2243] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.757586   431 raft_consensus.cc:2272] T d59c4d21b6cb46edbfc2b3d50b51812a P 5184c101be204393adf3ac8b31f25ad6 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.760288   431 tablet_replica.cc:333] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6: stopping tablet replica
I20260709 13:04:59.760939   431 raft_consensus.cc:2243] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.761641   431 raft_consensus.cc:2272] T d5ccce7ae459467a984bd58d73f701ae P 5184c101be204393adf3ac8b31f25ad6 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:59.774216  2269 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.794629  1682 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.864003  1682 consensus_peers.cc:597] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.867930  1682 consensus_peers.cc:597] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.891644  1682 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.952427  2268 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:04:59.960757  1682 consensus_peers.cc:597] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.988761  2268 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:00.033178  1681 consensus_peers.cc:597] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:00.033215  2268 consensus_peers.cc:597] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:00.078016  2268 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:00.131139  2268 consensus_peers.cc:597] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:00.134757  2268 consensus_peers.cc:597] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:00.135598  1681 consensus_peers.cc:597] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:00.161172  1681 consensus_peers.cc:597] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:00.181324  2268 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:00.201503  2268 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:00.345067  1681 consensus_peers.cc:597] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:00.385165  2268 consensus_peers.cc:597] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:05:00.398257  2268 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:00.483402  2268 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111) [suppressed 469 similar messages]
W20260709 13:05:00.572038  1681 consensus_peers.cc:597] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:00.626850  1681 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:00.729620  1681 consensus_peers.cc:597] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:05:00.764837  9608 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.765612  9608 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:00.769126  9608 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:05:00.778719  2268 leader_election.cc:336] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:05:00.778961  9546 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.780913  9546 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:00.783377  9546 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:05:00.784098  9546 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.784653  9546 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:00.779556  9609 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.787268  9609 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:00.789582  9609 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:05:00.790827  9546 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:05:00.791564  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
W20260709 13:05:00.794896  2269 leader_election.cc:336] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:00.795723  1717 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 25.
I20260709 13:05:00.798408  2269 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:00.799407  9601 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.796209  9546 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.797950  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:05:00.800022  9601 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:00.800637  9546 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:00.803468  9546 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:05:00.807190  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:05:00.800040  9609 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:00.807976  9601 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:05:00.815191  2303 raft_consensus.cc:2468] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 25.
I20260709 13:05:00.812191  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:05:00.816411  9609 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:05:00.816793  9609 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.817170  9609 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:00.817176  1683 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:05:00.819031  9611 raft_consensus.cc:2804] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:05:00.819449  9611 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.819815  9611 raft_consensus.cc:3060] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:05:00.825718  9609 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:05:00.827019  2269 leader_election.cc:336] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
W20260709 13:05:00.830506  2269 leader_election.cc:336] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
W20260709 13:05:00.827659  2268 leader_election.cc:336] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:05:00.831260  2268 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:05:00.833976  9615 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:00.835371  1682 leader_election.cc:336] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
W20260709 13:05:00.835909  2268 leader_election.cc:336] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:05:00.836503  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
W20260709 13:05:00.838804  2269 leader_election.cc:336] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:00.839855  9608 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:00.842034  9608 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.842577  9608 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:00.844800  9608 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:05:00.847733  9601 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.848469  9601 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:05:00.850868  2269 leader_election.cc:336] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:00.852739  9615 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.853139  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:05:00.856959  9615 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:00.858713  2297 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 23.
I20260709 13:05:00.858758  9601 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:05:00.862353  1683 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
I20260709 13:05:00.863420  9576 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:05:00.863801  9576 raft_consensus.cc:493] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.864169  9576 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:05:00.871241  9615 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:05:00.872881  2268 leader_election.cc:336] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:05:00.873880  2268 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:00.860965  9611 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:00.876299  9601 raft_consensus.cc:493] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.885226  9601 raft_consensus.cc:515] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } attrs { replace: true } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:00.881114  9615 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:00.893863  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:05:00.894722  1716 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 26: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:05:00.895955  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:05:00.882798  9576 raft_consensus.cc:515] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:00.856132  9609 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 26 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:05:00.878075  2269 leader_election.cc:336] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:00.886730  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5448ccc31ba14113b3d56168031c6714" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
I20260709 13:05:00.897186  1716 raft_consensus.cc:2393] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6208c2fa49f643b5a50ab8f6d292fab2 in current term 24: Already voted for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in this term.
I20260709 13:05:00.897660  2297 raft_consensus.cc:2393] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 28bc59d37f664ce0a6f5784f2619f0b5 in current term 26: Already voted for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in this term.
I20260709 13:05:00.891028  9611 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 26 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:05:00.891705  1682 leader_election.cc:336] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:00.901641  2297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6dc6e12b7924255b756439991155e88" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2"
W20260709 13:05:00.902236  2269 leader_election.cc:336] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:00.902331  2297 raft_consensus.cc:3060] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:05:00.913434  2269 leader_election.cc:336] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:00.903173  9576 leader_election.cc:290] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:05:00.911204  1681 leader_election.cc:336] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:05:00.903650  1682 leader_election.cc:336] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:00.917182  1682 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:05:00.917989  9576 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:05:00.921969  2297 raft_consensus.cc:2468] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 24.
I20260709 13:05:00.922314  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2832827f30046328715699fa6ee0c52" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:05:00.923070  2303 raft_consensus.cc:2468] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bc59d37f664ce0a6f5784f2619f0b5 in term 16.
W20260709 13:05:00.923947  1682 leader_election.cc:336] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:00.924925  9576 raft_consensus.cc:493] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.925489  1683 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:00.925405  9576 raft_consensus.cc:515] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:05:00.927582  9576 leader_election.cc:290] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:05:00.928114  9576 raft_consensus.cc:2804] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:05:00.929391  9576 raft_consensus.cc:697] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 LEADER]: Becoming Leader. State: Replica: 28bc59d37f664ce0a6f5784f2619f0b5, State: Running, Role: LEADER
I20260709 13:05:00.930433  9576 consensus_queue.cc:237] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:05:00.932413  1681 consensus_peers.cc:597] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:00.932895  1681 leader_election.cc:336] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:05:00.903393  2268 leader_election.cc:336] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:05:00.933586  2268 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:00.934346  9609 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:00.938675  9601 leader_election.cc:290] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
W20260709 13:05:00.939596  1682 leader_election.cc:336] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:00.944284  1682 leader_election.cc:304] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:00.956000  9601 raft_consensus.cc:2749] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:00.937856  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:00.957619  1462 catalog_manager.cc:5697] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 reported cstate change: term changed from 23 to 24, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195). New cstate: current_term: 24 leader_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:00.958841  1682 leader_election.cc:336] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:00.959002  9609 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:05:00.959982  2268 leader_election.cc:304] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:00.970453  9597 raft_consensus.cc:493] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.977393  9597 raft_consensus.cc:515] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:05:00.979128  9615 raft_consensus.cc:2749] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:05:00.979933  9597 leader_election.cc:290] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:05:00.960021  1682 leader_election.cc:304] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:00.984963  9601 raft_consensus.cc:2749] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:00.993314  1681 leader_election.cc:336] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:05:00.996291  9611 raft_consensus.cc:493] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:00.996892  9611 raft_consensus.cc:515] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:05:01.000770  9611 leader_election.cc:290] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:05:01.000883  1682 leader_election.cc:336] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:01.001992  1682 leader_election.cc:304] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:00.994424  9597 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:01.004648  9597 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:05:01.006397  1681 leader_election.cc:336] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:05:01.007205  9597 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196:38715)
I20260709 13:05:01.008342  9601 raft_consensus.cc:2749] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:05:01.008522  2303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" is_pre_election: true
I20260709 13:05:01.013123  9615 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:01.013741  9615 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:05:01.014253  1682 leader_election.cc:336] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:01.014750  1682 leader_election.cc:304] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:01.015515  9597 raft_consensus.cc:2749] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:05:01.017114  1682 leader_election.cc:336] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:01.016026  9615 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:05:01.017570  1682 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 5184c101be204393adf3ac8b31f25ad6, 6208c2fa49f643b5a50ab8f6d292fab2
I20260709 13:05:01.018353  9576 raft_consensus.cc:2749] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:05:01.025975  2269 leader_election.cc:336] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:01.028581  9615 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:01.029407  9615 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:01.031991  9615 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:05:01.040448  2269 leader_election.cc:336] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
W20260709 13:05:01.045043  2268 leader_election.cc:336] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:05:01.045665  2268 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:01.046509  9615 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:01.046753  9576 raft_consensus.cc:493] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:01.047338  9576 raft_consensus.cc:515] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } }
I20260709 13:05:01.049237  9576 leader_election.cc:290] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:05:01.052614  2268 leader_election.cc:336] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:05:01.053642  2268 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:01.054768  9615 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:01.056241  2268 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:01.058104  1681 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:05:01.061961  1682 leader_election.cc:336] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:01.062582  1682 leader_election.cc:304] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:01.063349  9576 raft_consensus.cc:2749] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:05:01.134374  1681 consensus_peers.cc:597] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:01.151158  1681 consensus_peers.cc:597] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:05:01.166816  9615 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:01.167472  9615 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:01.169554  9615 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:05:01.172338  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" is_pre_election: true
I20260709 13:05:01.172991  1716 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 20.
I20260709 13:05:01.177691  2268 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 
W20260709 13:05:01.179615  2269 leader_election.cc:336] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:01.181222  9615 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:01.181758  9615 raft_consensus.cc:493] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:01.182188  9615 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:01.198948  9615 raft_consensus.cc:515] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:01.201153  9615 leader_election.cc:290] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Requested vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:05:01.204375  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9df7dc71f943412e83fbc8d4f788c9f2" candidate_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "28bc59d37f664ce0a6f5784f2619f0b5"
I20260709 13:05:01.205027  1716 raft_consensus.cc:3060] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:01.205869  2269 leader_election.cc:336] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:01.212639  1716 raft_consensus.cc:2468] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6208c2fa49f643b5a50ab8f6d292fab2 in term 21.
I20260709 13:05:01.214082  2268 leader_election.cc:304] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28bc59d37f664ce0a6f5784f2619f0b5, 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:01.215292  9615 raft_consensus.cc:2804] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:01.215806  9615 raft_consensus.cc:697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Becoming Leader. State: Replica: 6208c2fa49f643b5a50ab8f6d292fab2, State: Running, Role: LEADER
I20260709 13:05:01.216748  9615 consensus_queue.cc:237] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:01.234863  1462 catalog_manager.cc:5697] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 reported cstate change: term changed from 20 to 21, leader changed from 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194) to 6208c2fa49f643b5a50ab8f6d292fab2 (127.0.107.196). New cstate: current_term: 21 leader_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:01.256819  1681 consensus_peers.cc:597] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:01.341607  2268 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:01.368818  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:05:01.399111  2297 raft_consensus.cc:1275] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Refusing update from remote peer 28bc59d37f664ce0a6f5784f2619f0b5: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:05:01.401937  9601 consensus_queue.cc:1048] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:05:01.415654  1681 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:01.450176  1682 consensus_peers.cc:597] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:01.509454  1682 consensus_peers.cc:597] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:01.608868  1681 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:01.611122  1682 consensus_peers.cc:597] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:01.652112  1716 raft_consensus.cc:1275] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Refusing update from remote peer 6208c2fa49f643b5a50ab8f6d292fab2: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:05:01.654145  9615 consensus_queue.cc:1048] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:05:01.678578  2269 consensus_peers.cc:597] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:01.734490  1681 consensus_peers.cc:597] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:05:01.754170  1681 consensus_peers.cc:597] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:01.768132  2268 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:01.784538  2269 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:01.816718   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:05:01.826726  1682 consensus_peers.cc:597] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:01.832389  1681 consensus_peers.cc:597] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:01.863371  1682 consensus_peers.cc:597] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:01.877993  2268 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:01.880720  2269 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:01.922835  2269 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:01.928333  1682 consensus_peers.cc:597] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:01.929932  2269 consensus_peers.cc:597] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:01.939294   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:05:01.960672  2269 consensus_peers.cc:597] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:02.001786  2268 consensus_peers.cc:597] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:02.001919  1682 consensus_peers.cc:597] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:02.006067  1682 consensus_peers.cc:597] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:02.013535   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:02.014743   431 tablet_replica.cc:333] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.015978   431 raft_consensus.cc:2243] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.017067   431 raft_consensus.cc:2272] T 4594cfed297144b39e1a29473c9707d9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.019862   431 tablet_replica.cc:333] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.020509   431 raft_consensus.cc:2243] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.021042   431 raft_consensus.cc:2272] T 5448ccc31ba14113b3d56168031c6714 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.023109   431 tablet_replica.cc:333] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.023738   431 raft_consensus.cc:2243] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.024607   431 raft_consensus.cc:2272] T 250e1f9f055e4fda9987138306f2e0b7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.026813   431 tablet_replica.cc:333] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.027611   431 raft_consensus.cc:2243] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.028499   431 raft_consensus.cc:2272] T b7808a4bebf84e598198ed51ea156981 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.031829   431 tablet_replica.cc:333] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.032934   431 raft_consensus.cc:2243] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.034247   431 raft_consensus.cc:2272] T d883c107e7bd4b6daa6d09672acc2b23 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.037299   431 tablet_replica.cc:333] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.038172   431 raft_consensus.cc:2243] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.039305   431 raft_consensus.cc:2272] T bb58673aa80747e58aa935f4fed2212d P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:02.039525  1682 consensus_peers.cc:597] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:02.041697   431 tablet_replica.cc:333] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.042434   431 raft_consensus.cc:2243] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.043147   431 raft_consensus.cc:2272] T 84f86a4ba55242bd8d2459c6b9df56d2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.046106   431 tablet_replica.cc:333] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.047062   431 raft_consensus.cc:2243] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.048281   431 raft_consensus.cc:2272] T 50d8055bc87c4624a96f13638fe667e6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:02.049580  2268 consensus_peers.cc:597] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:02.051530   431 tablet_replica.cc:333] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.052549   431 raft_consensus.cc:2243] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.053295   431 raft_consensus.cc:2272] T 5a08948b1fac416d8d27be0561a00db4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:02.054432  2269 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:02.056200   431 tablet_replica.cc:333] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.057217   431 raft_consensus.cc:2243] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.058614   431 raft_consensus.cc:2272] T 71b0b66754d049cfbccb38fbb16ef3cd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.060953   431 tablet_replica.cc:333] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
W20260709 13:05:02.061302  1682 consensus_peers.cc:597] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:02.061795   431 raft_consensus.cc:2243] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.062988   431 raft_consensus.cc:2272] T 4c0f61db23f34217bd5089a459cbb914 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:02.063095  2268 consensus_peers.cc:597] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:02.065265   431 tablet_replica.cc:333] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.065836   431 raft_consensus.cc:2243] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.066653   431 raft_consensus.cc:2272] T 213d9f9af66a4ebd8c9a7b0163196a57 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.068706   431 tablet_replica.cc:333] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.069245   431 raft_consensus.cc:2243] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.069720   431 raft_consensus.cc:2272] T 3905c76f9588440c8a836e5786f6667e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.072062   431 tablet_replica.cc:333] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.072629   431 raft_consensus.cc:2243] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.073523   431 raft_consensus.cc:2272] T 768b897f18a644e8b1e8f8ed1c8b616f P 28bc59d37f664ce0a6f5784f2619f0b5 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.076313   431 tablet_replica.cc:333] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.077335   431 raft_consensus.cc:2243] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.078553   431 raft_consensus.cc:2272] T 1d4f9d3ff1a04fcbba1e77b6fa3e6358 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:02.080248  1682 consensus_peers.cc:597] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:02.081214   431 tablet_replica.cc:333] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.081874   431 raft_consensus.cc:2243] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.082883   431 raft_consensus.cc:2272] T 9310eb7a2c8447e28c9096c983ddf5b5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.085248   431 tablet_replica.cc:333] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.085867   431 raft_consensus.cc:2243] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.086365   431 raft_consensus.cc:2272] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.088359   431 tablet_replica.cc:333] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.089196   431 raft_consensus.cc:2243] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.090191   431 raft_consensus.cc:2272] T d9ba36df4f014c3a916b0393305ea44a P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.092303   431 tablet_replica.cc:333] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.092851   431 raft_consensus.cc:2243] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.093348   431 raft_consensus.cc:2272] T 92a2b88a149f424591686a115bc1f0d6 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.095417   431 tablet_replica.cc:333] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.095933   431 raft_consensus.cc:2243] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.096838   431 raft_consensus.cc:2272] T e6dc6e12b7924255b756439991155e88 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.100426   431 tablet_replica.cc:333] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.101244   431 raft_consensus.cc:2243] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.102337   431 raft_consensus.cc:2272] T b3fdbd88a9d440e59ddd4e49c79ab2f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:02.102977  2268 consensus_peers.cc:597] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:02.104923   431 tablet_replica.cc:333] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.105650   431 raft_consensus.cc:2243] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.106343   431 raft_consensus.cc:2272] T d29864f650d0477e8dd2e7f82eea9e47 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.109115   431 tablet_replica.cc:333] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.109742   431 raft_consensus.cc:2243] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:05:02.110366  1682 consensus_peers.cc:597] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:02.110885   431 raft_consensus.cc:2272] T 520f39ecc282462782487a3651e28b44 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.113965   431 tablet_replica.cc:333] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.114851   431 raft_consensus.cc:2243] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.116014   431 raft_consensus.cc:2272] T c129438e9d58493a93890208b25765cf P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.118737   431 tablet_replica.cc:333] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.119657   431 raft_consensus.cc:2243] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.120599   431 raft_consensus.cc:2272] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.123395   431 tablet_replica.cc:333] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.124244   431 raft_consensus.cc:2243] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.125306   431 raft_consensus.cc:2272] T 3f22f7f5cede49bea2083e8905932fb7 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.128355   431 tablet_replica.cc:333] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
W20260709 13:05:02.129151  2268 consensus_peers.cc:597] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:02.129266   431 raft_consensus.cc:2243] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.130558   431 raft_consensus.cc:2272] T c46cf090bf9b4084b648d00f5f0053e2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.132808   431 tablet_replica.cc:333] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.133581   431 raft_consensus.cc:2243] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.134222   431 raft_consensus.cc:2272] T 9df7dc71f943412e83fbc8d4f788c9f2 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.137415   431 tablet_replica.cc:333] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
W20260709 13:05:02.138240  1682 consensus_peers.cc:597] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:02.138293   431 raft_consensus.cc:2243] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.139454   431 raft_consensus.cc:2272] T fb294ac919b44e56960745f79e83dfc4 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.142078   431 tablet_replica.cc:333] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.142833   431 raft_consensus.cc:2243] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.143852   431 raft_consensus.cc:2272] T c2832827f30046328715699fa6ee0c52 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:02.144191  2269 consensus_peers.cc:597] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:02.146471   431 tablet_replica.cc:333] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
W20260709 13:05:02.147135  2269 consensus_peers.cc:597] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:02.147436   431 raft_consensus.cc:2243] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.148296   431 raft_consensus.cc:2272] T 3fe2946ff23d498f98c4627ee968a4e3 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.150784   431 tablet_replica.cc:333] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.151512   431 raft_consensus.cc:2243] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.152475   431 raft_consensus.cc:2272] T 2b9b7b078d0d47d3880d3dec3230b779 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.155296   431 tablet_replica.cc:333] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.156762   431 raft_consensus.cc:2243] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.157359   431 raft_consensus.cc:2272] T fbfadedb9a5e4ca0abc25384a524c546 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:02.159575  2268 consensus_peers.cc:597] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:02.160115   431 tablet_replica.cc:333] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.160943   431 raft_consensus.cc:2243] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.161621   431 raft_consensus.cc:2272] T 82743dc3081f4edba18591ca4b0d9c09 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.163793   431 tablet_replica.cc:333] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.164656   431 raft_consensus.cc:2243] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.165196   431 raft_consensus.cc:2272] T 37b565cd27a64e96addb09389c2c7d5e P 28bc59d37f664ce0a6f5784f2619f0b5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.167658   431 tablet_replica.cc:333] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.168382   431 raft_consensus.cc:2243] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.169486   431 raft_consensus.cc:2272] T f712f9c188e44e588e739161cac39330 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.171865   431 tablet_replica.cc:333] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
W20260709 13:05:02.172504  2269 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:02.173219   431 raft_consensus.cc:2243] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.174062   431 raft_consensus.cc:2272] T 344c88e0be9640f583140fb4aab3af58 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.176875   431 tablet_replica.cc:333] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.177698   431 raft_consensus.cc:2243] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.178508   431 raft_consensus.cc:2272] T 6bdefe70d276420ca0283414ab16fab9 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.181141   431 tablet_replica.cc:333] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.181715   431 raft_consensus.cc:2243] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:02.182502   431 raft_consensus.cc:2272] T cdf54b70d44944ef9013c4adb0970687 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.184747   431 tablet_replica.cc:333] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.185287   431 raft_consensus.cc:2243] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.186210   431 raft_consensus.cc:2272] T d0049d6871cf4f95be599108abda58a5 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.188410   431 tablet_replica.cc:333] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.189385   431 raft_consensus.cc:2243] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.190341   431 raft_consensus.cc:2272] T 1b7e7c3881a44475827c9cbca08a2fe1 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.192646   431 tablet_replica.cc:333] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.193221   431 raft_consensus.cc:2243] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.194098   431 raft_consensus.cc:2272] T 70a0a5f8de344870be97d83cec17c047 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.196723   431 tablet_replica.cc:333] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.197450   431 raft_consensus.cc:2243] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.198438   431 raft_consensus.cc:2272] T 6711207c14044518a57e7c8914ac0091 P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:02.199699  2269 consensus_peers.cc:597] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:02.201006   431 tablet_replica.cc:333] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.201699   431 raft_consensus.cc:2243] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.202533   431 raft_consensus.cc:2272] T 147a8e85db5b4380bf845a82736ec5fd P 28bc59d37f664ce0a6f5784f2619f0b5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:02.205219   431 tablet_replica.cc:333] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5: stopping tablet replica
I20260709 13:05:02.205801   431 raft_consensus.cc:2243] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:05:02.206727   431 raft_consensus.cc:2272] T d5ccce7ae459467a984bd58d73f701ae P 28bc59d37f664ce0a6f5784f2619f0b5 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:02.207620  2268 consensus_peers.cc:597] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:02.249464  2269 consensus_peers.cc:597] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:02.289292  2268 connection.cc:573] client connection to 127.0.107.195:33997 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:05:02.296689  2268 consensus_peers.cc:597] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33997: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:02.312930  2268 consensus_peers.cc:597] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33997: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:02.327008  2268 consensus_peers.cc:597] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:05:02.354018  2268 consensus_peers.cc:597] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33997: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:02.368988  2268 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33997: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:02.449879  2268 consensus_peers.cc:597] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Couldn't send request to peer 28bc59d37f664ce0a6f5784f2619f0b5. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33997: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:02.505781  2269 consensus_peers.cc:597] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Couldn't send request to peer 5184c101be204393adf3ac8b31f25ad6. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:02.536543  2268 consensus_peers.cc:597] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:05:02.656710  9628 raft_consensus.cc:493] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:02.657370  9628 raft_consensus.cc:515] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:02.659914  9628 leader_election.cc:290] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:05:02.669991  2269 leader_election.cc:336] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
W20260709 13:05:02.678342  2268 leader_election.cc:336] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:05:02.678900  2268 leader_election.cc:304] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:02.679668  9628 raft_consensus.cc:2749] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:05:02.683583  2268 consensus_peers.cc:597] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:05:02.688484  2268 consensus_peers.cc:597] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:02.696013  2268 consensus_peers.cc:597] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:02.720275  2268 consensus_peers.cc:597] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:05:02.760875  2268 consensus_peers.cc:597] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:05:02.784318   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:05:02.796190  9627 raft_consensus.cc:493] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.796839  9627 raft_consensus.cc:515] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:02.799104  9627 leader_election.cc:290] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
W20260709 13:05:02.806023  2269 leader_election.cc:336] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
W20260709 13:05:02.815690  2268 leader_election.cc:336] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Network error: Client connection negotiation failed: client connection to 127.0.107.195:33997: connect: Connection refused (error 111)
I20260709 13:05:02.816322  2268 leader_election.cc:304] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:02.817250  9627 raft_consensus.cc:2749] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:05:02.880028  9627 raft_consensus.cc:493] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:02.880591  9627 raft_consensus.cc:515] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:02.882828  9627 leader_election.cc:290] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:05:02.890146  2268 leader_election.cc:336] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:05:02.893944  2269 leader_election.cc:336] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
W20260709 13:05:02.894310  2268 consensus_peers.cc:597] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:05:02.895247  2269 leader_election.cc:304] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:02.897652  9627 raft_consensus.cc:2749] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:05:02.931018  2268 consensus_peers.cc:597] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:05:02.937162   431 tablet_server.cc:179] TabletServer@127.0.107.196:0 shutting down...
I20260709 13:05:02.939409  9627 raft_consensus.cc:493] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:02.940191  9627 raft_consensus.cc:515] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:02.941973  9627 leader_election.cc:290] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:05:02.946324  2268 leader_election.cc:336] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:05:02.947676  2269 leader_election.cc:336] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:02.948171  2269 leader_election.cc:304] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:02.948971  9627 raft_consensus.cc:2749] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:02.959337  9627 raft_consensus.cc:493] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:02.960044  9627 raft_consensus.cc:515] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:02.963708  9627 leader_election.cc:290] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
W20260709 13:05:02.975016  2268 leader_election.cc:336] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
W20260709 13:05:02.978358  2269 leader_election.cc:336] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:03.000018  2269 leader_election.cc:304] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:03.000933  9627 raft_consensus.cc:2749] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:05:03.004576  2268 consensus_peers.cc:597] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 -> Peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Couldn't send request to peer 3f64b76180a54492bb015e405caeacec. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:05:03.018999   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:03.020164   431 tablet_replica.cc:333] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.021080   431 raft_consensus.cc:2243] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.021773   431 raft_consensus.cc:2272] T c2832827f30046328715699fa6ee0c52 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.024387   431 tablet_replica.cc:333] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.025141   431 raft_consensus.cc:2243] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.026399   431 raft_consensus.cc:2272] T fb294ac919b44e56960745f79e83dfc4 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.027268  9627 raft_consensus.cc:493] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:03.027832  9627 raft_consensus.cc:515] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:03.029815   431 tablet_replica.cc:333] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.030489  9627 leader_election.cc:290] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:05:03.031011   431 raft_consensus.cc:2243] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.035527   431 raft_consensus.cc:2272] T 3f22f7f5cede49bea2083e8905932fb7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.036499  9627 raft_consensus.cc:493] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 28bc59d37f664ce0a6f5784f2619f0b5)
W20260709 13:05:03.034996  2268 leader_election.cc:336] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:05:03.037153  9627 raft_consensus.cc:515] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "28bc59d37f664ce0a6f5784f2619f0b5" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33997 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
W20260709 13:05:03.038378  2269 leader_election.cc:336] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:03.038842  2269 leader_election.cc:304] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:03.039932   431 tablet_replica.cc:333] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.039753  9628 raft_consensus.cc:2749] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:05:03.040962   431 raft_consensus.cc:2243] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.041616   431 raft_consensus.cc:2272] T 250e1f9f055e4fda9987138306f2e0b7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.039721  9627 leader_election.cc:290] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997)
I20260709 13:05:03.044448   431 tablet_replica.cc:333] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.045383   431 raft_consensus.cc:2243] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.046617   431 raft_consensus.cc:2272] T 84f86a4ba55242bd8d2459c6b9df56d2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.049522   431 tablet_replica.cc:333] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.050381   431 raft_consensus.cc:2243] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.051112   431 raft_consensus.cc:2272] T 2a8ba0cc212d41e5b0ad586b885d07a7 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:03.053438  2268 leader_election.cc:336] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:05:03.054158   431 tablet_replica.cc:333] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.055058   431 raft_consensus.cc:2243] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:05:03.055953  2268 leader_election.cc:336] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 28bc59d37f664ce0a6f5784f2619f0b5 (127.0.107.195:33997): Network error: Client connection negotiation failed: client connection to 127.0.107.195:33997: connect: Connection refused (error 111)
I20260709 13:05:03.056191   431 raft_consensus.cc:2272] T 82743dc3081f4edba18591ca4b0d9c09 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.056368  2268 leader_election.cc:304] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 28bc59d37f664ce0a6f5784f2619f0b5, 3f64b76180a54492bb015e405caeacec
I20260709 13:05:03.057258  9609 raft_consensus.cc:2749] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:03.059242   431 tablet_replica.cc:333] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.060211   431 raft_consensus.cc:2243] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.061321   431 raft_consensus.cc:2272] T 1e353d5ccbb3478ba62d1d60ecc3ec05 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.064323   431 tablet_replica.cc:333] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.064642  9628 raft_consensus.cc:493] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5184c101be204393adf3ac8b31f25ad6)
I20260709 13:05:03.065135   431 raft_consensus.cc:2243] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.065124  9628 raft_consensus.cc:515] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3f64b76180a54492bb015e405caeacec" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 36547 } } peers { permanent_uuid: "5184c101be204393adf3ac8b31f25ad6" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 40023 } } peers { permanent_uuid: "6208c2fa49f643b5a50ab8f6d292fab2" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 38715 } attrs { promote: false } }
I20260709 13:05:03.065938   431 raft_consensus.cc:2272] T 70a0a5f8de344870be97d83cec17c047 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.067313  9628 leader_election.cc:290] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547), 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023)
I20260709 13:05:03.068868   431 tablet_replica.cc:333] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.071174   431 raft_consensus.cc:2243] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.072232   431 raft_consensus.cc:2272] T 92a2b88a149f424591686a115bc1f0d6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:03.072785  2269 leader_election.cc:336] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5184c101be204393adf3ac8b31f25ad6 (127.0.107.194:40023): Network error: Client connection negotiation failed: client connection to 127.0.107.194:40023: connect: Connection refused (error 111)
I20260709 13:05:03.074869   431 tablet_replica.cc:333] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
W20260709 13:05:03.074906  2268 leader_election.cc:336] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:05:03.075582  2268 leader_election.cc:304] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6208c2fa49f643b5a50ab8f6d292fab2; no voters: 3f64b76180a54492bb015e405caeacec, 5184c101be204393adf3ac8b31f25ad6
I20260709 13:05:03.075856   431 raft_consensus.cc:2243] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.076269  9628 raft_consensus.cc:2749] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:05:03.077436   431 raft_consensus.cc:2272] T 37b565cd27a64e96addb09389c2c7d5e P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.080827   431 tablet_replica.cc:333] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.081638   431 raft_consensus.cc:2243] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.082998   431 raft_consensus.cc:2272] T 403bcba338a042c1b5fecfe8ac430922 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.085888   431 tablet_replica.cc:333] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.086833   431 raft_consensus.cc:2243] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.087522   431 raft_consensus.cc:2272] T 33657ac7590d4739b9bda3f7bb345c7b P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.090312   431 tablet_replica.cc:333] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.091105   431 raft_consensus.cc:2243] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.092099   431 raft_consensus.cc:2272] T 408b4ec829c54266b41334aac8245405 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.094729   431 tablet_replica.cc:333] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.095495   431 raft_consensus.cc:2243] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.096546   431 raft_consensus.cc:2272] T d59c4d21b6cb46edbfc2b3d50b51812a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.099257   431 tablet_replica.cc:333] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.100033   431 raft_consensus.cc:2243] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.100688   431 raft_consensus.cc:2272] T ca83cd5e36eb4e22b75011b87bfcd49d P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.103281   431 tablet_replica.cc:333] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.103994   431 raft_consensus.cc:2243] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.104537   431 raft_consensus.cc:2272] T d5ccce7ae459467a984bd58d73f701ae P 6208c2fa49f643b5a50ab8f6d292fab2 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.107543   431 tablet_replica.cc:333] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.108337   431 raft_consensus.cc:2243] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.109143   431 raft_consensus.cc:2272] T 1b7e7c3881a44475827c9cbca08a2fe1 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.112273   431 tablet_replica.cc:333] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.113029   431 raft_consensus.cc:2243] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.114081   431 raft_consensus.cc:2272] T c5eae102421c4bc4b19781bf4470a0b0 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.117167   431 tablet_replica.cc:333] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.117937   431 raft_consensus.cc:2243] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.118523   431 raft_consensus.cc:2272] T d0049d6871cf4f95be599108abda58a5 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.121948   431 tablet_replica.cc:333] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.122754   431 raft_consensus.cc:2243] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.123405   431 raft_consensus.cc:2272] T 50d8055bc87c4624a96f13638fe667e6 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.126116   431 tablet_replica.cc:333] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.126900   431 raft_consensus.cc:2243] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.127560   431 raft_consensus.cc:2272] T d883c107e7bd4b6daa6d09672acc2b23 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.130128   431 tablet_replica.cc:333] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.131064   431 raft_consensus.cc:2243] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.132197   431 raft_consensus.cc:2272] T e62e41b6e9a54f26a1c7c1538bdf5334 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.135123   431 tablet_replica.cc:333] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.135885   431 raft_consensus.cc:2243] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.136447   431 raft_consensus.cc:2272] T 8a4760a45328493e87523c2ee8292599 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.139117   431 tablet_replica.cc:333] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.139861   431 raft_consensus.cc:2243] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.140471   431 raft_consensus.cc:2272] T d9ba36df4f014c3a916b0393305ea44a P 6208c2fa49f643b5a50ab8f6d292fab2 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.143486   431 tablet_replica.cc:333] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.144310   431 raft_consensus.cc:2243] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.145226   431 raft_consensus.cc:2272] T 9f6815b361554c338159f274bd78c0e9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.147909   431 tablet_replica.cc:333] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.148708   431 raft_consensus.cc:2243] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.150171   431 raft_consensus.cc:2272] T cdf54b70d44944ef9013c4adb0970687 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.152990   431 tablet_replica.cc:333] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.153757   431 raft_consensus.cc:2243] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.154286   431 raft_consensus.cc:2272] T 8c85446f87564ae6865060bb988ee1b8 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.156939   431 tablet_replica.cc:333] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.157694   431 raft_consensus.cc:2243] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.158718   431 raft_consensus.cc:2272] T 344c88e0be9640f583140fb4aab3af58 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.161494   431 tablet_replica.cc:333] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.162405   431 raft_consensus.cc:2243] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.163021   431 raft_consensus.cc:2272] T 147a8e85db5b4380bf845a82736ec5fd P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.165647   431 tablet_replica.cc:333] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.166386   431 raft_consensus.cc:2243] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.167551   431 raft_consensus.cc:2272] T 9df7dc71f943412e83fbc8d4f788c9f2 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.170185   431 tablet_replica.cc:333] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.170953   431 raft_consensus.cc:2243] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.172008   431 raft_consensus.cc:2272] T d4bc96896acb4fcf8b5ec5b7206248ca P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.174706   431 tablet_replica.cc:333] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.175522   431 raft_consensus.cc:2243] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.176596   431 raft_consensus.cc:2272] T 6bdefe70d276420ca0283414ab16fab9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.179263   431 tablet_replica.cc:333] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.180006   431 raft_consensus.cc:2243] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.181052   431 raft_consensus.cc:2272] T ac4c23f0e4f742e38ca7a5ebd7778dfa P 6208c2fa49f643b5a50ab8f6d292fab2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.183917   431 tablet_replica.cc:333] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.184684   431 raft_consensus.cc:2243] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.185221   431 raft_consensus.cc:2272] T d1256ff04335423286d7c081ec776061 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.187745   431 tablet_replica.cc:333] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.188455   431 raft_consensus.cc:2243] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.189255   431 raft_consensus.cc:2272] T 213d9f9af66a4ebd8c9a7b0163196a57 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.191886   431 tablet_replica.cc:333] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.192680   431 raft_consensus.cc:2243] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.193221   431 raft_consensus.cc:2272] T 5448ccc31ba14113b3d56168031c6714 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.196082   431 tablet_replica.cc:333] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.196790   431 raft_consensus.cc:2243] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.197418   431 raft_consensus.cc:2272] T 4594cfed297144b39e1a29473c9707d9 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.200063   431 tablet_replica.cc:333] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.200798   431 raft_consensus.cc:2243] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.201514   431 raft_consensus.cc:2272] T 3fe2946ff23d498f98c4627ee968a4e3 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.204118   431 tablet_replica.cc:333] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.204730   431 raft_consensus.cc:2243] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.205291   431 raft_consensus.cc:2272] T 2792e3e29993414a8d6f55078ae13994 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.207327   431 tablet_replica.cc:333] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.207857   431 raft_consensus.cc:2243] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.208711   431 raft_consensus.cc:2272] T fbfadedb9a5e4ca0abc25384a524c546 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.210597   431 tablet_replica.cc:333] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.211194   431 raft_consensus.cc:2243] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.211655   431 raft_consensus.cc:2272] T 768b897f18a644e8b1e8f8ed1c8b616f P 6208c2fa49f643b5a50ab8f6d292fab2 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.213837   431 tablet_replica.cc:333] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.214366   431 raft_consensus.cc:2243] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:05:03.215265   431 raft_consensus.cc:2272] T b7808a4bebf84e598198ed51ea156981 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.217201   431 tablet_replica.cc:333] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2: stopping tablet replica
I20260709 13:05:03.217680   431 raft_consensus.cc:2243] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:03.218104   431 raft_consensus.cc:2272] T e6dc6e12b7924255b756439991155e88 P 6208c2fa49f643b5a50ab8f6d292fab2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:03.620594   431 tablet_server.cc:196] TabletServer@127.0.107.196:0 shutdown complete.
I20260709 13:05:03.698369   431 master.cc:562] Master@127.0.107.254:37201 shutting down...
W20260709 13:05:03.738294  1515 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:05:03.738794  1515 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:05:03.739148  1515 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:05:09.051514  1451 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111) [suppressed 354 similar messages]
W20260709 13:05:09.056002  1451 catalog_manager.cc:4755] TS 3f64b76180a54492bb015e405caeacec (127.0.107.193:36547): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3f22f7f5cede49bea2083e8905932fb7: Network error: Client connection negotiation failed: client connection to 127.0.107.193:36547: connect: Connection refused (error 111)
I20260709 13:05:09.339916   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:09.341226   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:09.341621   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6ffeec09a12e4bd6997354c8ecdff02d: stopping tablet replica
I20260709 13:05:09.507306   431 master.cc:584] Master@127.0.107.254:37201 shutdown complete.
I20260709 13:05:09.564038   431 test_util.cc:182] -----------------------------------------------
I20260709 13:05:09.564255   431 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602014632082-431-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (269856 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:05:09.567849   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:42979
I20260709 13:05:09.569053   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:09.575388  9633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:09.575403  9634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:09.576347  9636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:09.577487   431 server_base.cc:1061] running on GCE node
I20260709 13:05:09.581462   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:09.581713   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:09.581883   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602309581864 us; error 0 us; skew 500 ppm
I20260709 13:05:09.582429   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:09.585160   431 webserver.cc:533] Webserver started at http://127.0.107.254:35223/ using document root <none> and password file <none>
I20260709 13:05:09.585690   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:09.585901   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:09.586184   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:09.587581   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/master-0-root/instance:
uuid: "242b0818bc8c46a6b7943f2e8320d4fd"
format_stamp: "Formatted at 2026-07-09 13:05:09 on dist-test-slave-cq4t"
I20260709 13:05:09.592324   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:05:09.596163  9641 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:09.597139   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:05:09.597530   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/master-0-root
uuid: "242b0818bc8c46a6b7943f2e8320d4fd"
format_stamp: "Formatted at 2026-07-09 13:05:09 on dist-test-slave-cq4t"
I20260709 13:05:09.597934   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:09.625401   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:09.626653   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:09.668963   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:42979
I20260709 13:05:09.669091  9692 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:42979 every 8 connection(s)
I20260709 13:05:09.673390  9693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:09.685899  9693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd: Bootstrap starting.
I20260709 13:05:09.690881  9693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:09.695583  9693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd: No bootstrap required, opened a new log
I20260709 13:05:09.698402  9693 raft_consensus.cc:359] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b0818bc8c46a6b7943f2e8320d4fd" member_type: VOTER }
I20260709 13:05:09.699040  9693 raft_consensus.cc:385] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:09.699296  9693 raft_consensus.cc:740] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242b0818bc8c46a6b7943f2e8320d4fd, State: Initialized, Role: FOLLOWER
I20260709 13:05:09.699936  9693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b0818bc8c46a6b7943f2e8320d4fd" member_type: VOTER }
I20260709 13:05:09.700474  9693 raft_consensus.cc:399] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:05:09.700780  9693 raft_consensus.cc:493] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:05:09.701054  9693 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:09.705981  9693 raft_consensus.cc:515] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b0818bc8c46a6b7943f2e8320d4fd" member_type: VOTER }
I20260709 13:05:09.706594  9693 leader_election.cc:304] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 242b0818bc8c46a6b7943f2e8320d4fd; no voters: 
I20260709 13:05:09.707794  9693 leader_election.cc:290] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:05:09.708241  9696 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:09.709748  9696 raft_consensus.cc:697] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [term 1 LEADER]: Becoming Leader. State: Replica: 242b0818bc8c46a6b7943f2e8320d4fd, State: Running, Role: LEADER
I20260709 13:05:09.710531  9696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b0818bc8c46a6b7943f2e8320d4fd" member_type: VOTER }
I20260709 13:05:09.710861  9693 sys_catalog.cc:565] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:05:09.717440  9698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 242b0818bc8c46a6b7943f2e8320d4fd. Latest consensus state: current_term: 1 leader_uuid: "242b0818bc8c46a6b7943f2e8320d4fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b0818bc8c46a6b7943f2e8320d4fd" member_type: VOTER } }
I20260709 13:05:09.717549  9697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "242b0818bc8c46a6b7943f2e8320d4fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b0818bc8c46a6b7943f2e8320d4fd" member_type: VOTER } }
I20260709 13:05:09.718153  9698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:09.718446  9697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:09.720474  9704 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:05:09.728567  9704 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:05:09.729722   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:05:09.738811  9704 catalog_manager.cc:1383] Generated new cluster ID: e032fa7d390441be83ecb1394b29f4bd
I20260709 13:05:09.739169  9704 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:05:09.756632  9704 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:05:09.757949  9704 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:05:09.782198  9704 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd: Generated new TSK 0
I20260709 13:05:09.782851  9704 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:05:09.796622   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:09.803458  9714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:09.804325  9715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:09.806265  9717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:09.807230   431 server_base.cc:1061] running on GCE node
I20260709 13:05:09.809422   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:09.809641   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:09.809837   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602309809822 us; error 0 us; skew 500 ppm
I20260709 13:05:09.810433   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:09.813310   431 webserver.cc:533] Webserver started at http://127.0.107.193:36667/ using document root <none> and password file <none>
I20260709 13:05:09.813858   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:09.814054   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:09.814344   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:09.815716   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-0-root/instance:
uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
format_stamp: "Formatted at 2026-07-09 13:05:09 on dist-test-slave-cq4t"
I20260709 13:05:09.820554   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:05:09.824409  9722 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:09.825201   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 13:05:09.825527   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-0-root
uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
format_stamp: "Formatted at 2026-07-09 13:05:09 on dist-test-slave-cq4t"
I20260709 13:05:09.825801   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:09.839526   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:09.840795   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:09.842360   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:09.844779   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:09.845014   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:09.845268   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:09.845460   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:09.891305   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:41383
I20260709 13:05:09.891427  9784 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:41383 every 8 connection(s)
I20260709 13:05:09.896731   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:09.905836  9789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:09.906850  9790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:09.911013   431 server_base.cc:1061] running on GCE node
W20260709 13:05:09.911636  9792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:09.914008  9785 heartbeater.cc:344] Connected to a master server at 127.0.107.254:42979
I20260709 13:05:09.914407   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:05:09.914501  9785 heartbeater.cc:461] Registering TS with master...
W20260709 13:05:09.914736   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:09.914973   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602309914956 us; error 0 us; skew 500 ppm
I20260709 13:05:09.915514  9785 heartbeater.cc:507] Master 127.0.107.254:42979 requested a full tablet report, sending...
I20260709 13:05:09.915655   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:09.918007  9658 ts_manager.cc:194] Registered new tserver with Master: 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:09.918875   431 webserver.cc:533] Webserver started at http://127.0.107.194:42417/ using document root <none> and password file <none>
I20260709 13:05:09.919535   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:09.919734   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:09.920164   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:09.920176  9658 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47608
I20260709 13:05:09.921993   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-1-root/instance:
uuid: "a1cbca39e34743d8a40816c3bc5f2030"
format_stamp: "Formatted at 2026-07-09 13:05:09 on dist-test-slave-cq4t"
I20260709 13:05:09.926833   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:05:09.931362  9797 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:09.932224   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:05:09.932578   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-1-root
uuid: "a1cbca39e34743d8a40816c3bc5f2030"
format_stamp: "Formatted at 2026-07-09 13:05:09 on dist-test-slave-cq4t"
I20260709 13:05:09.932902   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:09.971200   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:09.972633   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:09.974365   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:09.977255   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:09.977541   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:09.977818   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:09.978014   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:10.027947   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:34259
I20260709 13:05:10.028106  9859 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:34259 every 8 connection(s)
I20260709 13:05:10.033257   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:10.041877  9863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:10.044049  9864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:10.046406   431 server_base.cc:1061] running on GCE node
I20260709 13:05:10.047235  9860 heartbeater.cc:344] Connected to a master server at 127.0.107.254:42979
W20260709 13:05:10.047483  9866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:10.047755  9860 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:10.048748   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:05:10.048815  9860 heartbeater.cc:507] Master 127.0.107.254:42979 requested a full tablet report, sending...
W20260709 13:05:10.049113   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:10.049353   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602310049333 us; error 0 us; skew 500 ppm
I20260709 13:05:10.050151   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:10.051611  9658 ts_manager.cc:194] Registered new tserver with Master: a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:10.053031   431 webserver.cc:533] Webserver started at http://127.0.107.195:43343/ using document root <none> and password file <none>
I20260709 13:05:10.053574  9658 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47614
I20260709 13:05:10.053650   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:10.054064   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:10.054350   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:10.055922   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-2-root/instance:
uuid: "6e10aab26c644389b8161974dcabc5d0"
format_stamp: "Formatted at 2026-07-09 13:05:10 on dist-test-slave-cq4t"
I20260709 13:05:10.060914   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:05:10.065007  9871 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:10.066103   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:05:10.066514   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-2-root
uuid: "6e10aab26c644389b8161974dcabc5d0"
format_stamp: "Formatted at 2026-07-09 13:05:10 on dist-test-slave-cq4t"
I20260709 13:05:10.066845   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:10.108531   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:10.109808   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:10.111222   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:10.113821   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:10.114012   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:10.114336   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:10.114508   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:10.160460   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:45019
I20260709 13:05:10.160571  9933 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:45019 every 8 connection(s)
I20260709 13:05:10.165772   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:10.173861  9937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:10.176298  9938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:10.180913   431 server_base.cc:1061] running on GCE node
W20260709 13:05:10.181008  9940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:10.181700  9934 heartbeater.cc:344] Connected to a master server at 127.0.107.254:42979
I20260709 13:05:10.182245  9934 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:10.182512   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:10.182837   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:10.183095   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602310183074 us; error 0 us; skew 500 ppm
I20260709 13:05:10.183379  9934 heartbeater.cc:507] Master 127.0.107.254:42979 requested a full tablet report, sending...
I20260709 13:05:10.184072   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:10.186316  9658 ts_manager.cc:194] Registered new tserver with Master: 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:10.187413   431 webserver.cc:533] Webserver started at http://127.0.107.196:36627/ using document root <none> and password file <none>
I20260709 13:05:10.188060   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:10.188140  9658 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47620
I20260709 13:05:10.188290   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:10.188869   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:10.190749   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-3-root/instance:
uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
format_stamp: "Formatted at 2026-07-09 13:05:10 on dist-test-slave-cq4t"
I20260709 13:05:10.196239   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:05:10.200349  9945 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:10.201248   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:05:10.201562   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-3-root
uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
format_stamp: "Formatted at 2026-07-09 13:05:10 on dist-test-slave-cq4t"
I20260709 13:05:10.201865   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:10.218364   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:10.219771   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:10.221062   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:10.223451   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:10.223677   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:10.223984   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:10.224157   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:10.273468   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.196:34321
I20260709 13:05:10.273579 10007 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.196:34321 every 8 connection(s)
I20260709 13:05:10.289098 10008 heartbeater.cc:344] Connected to a master server at 127.0.107.254:42979
I20260709 13:05:10.289716 10008 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:10.290973 10008 heartbeater.cc:507] Master 127.0.107.254:42979 requested a full tablet report, sending...
I20260709 13:05:10.293731  9658 ts_manager.cc:194] Registered new tserver with Master: 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:10.294885   431 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.017734877s
I20260709 13:05:10.295707  9658 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47622
I20260709 13:05:10.328861  9658 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47628:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:05:10.604286  9825 tablet_service.cc:1511] Processing CreateTablet for tablet 148fd9eeaa2647bca8ad75116e461ac6 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:10.606022  9825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 148fd9eeaa2647bca8ad75116e461ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.608129  9824 tablet_service.cc:1511] Processing CreateTablet for tablet aaf1d483dd4049a68642f7d94f1cfc70 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:10.608950  9822 tablet_service.cc:1511] Processing CreateTablet for tablet 251be49566d64412bd3d6e3d72c5795e (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:10.610708  9823 tablet_service.cc:1511] Processing CreateTablet for tablet ba93a39b54a742c6b391a2eb491f3d79 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:10.611231  9972 tablet_service.cc:1511] Processing CreateTablet for tablet 778ba1e8c7b04438a6633db058c14508 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:10.611873  9899 tablet_service.cc:1511] Processing CreateTablet for tablet aaf1d483dd4049a68642f7d94f1cfc70 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:10.612946  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778ba1e8c7b04438a6633db058c14508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.613523  9899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaf1d483dd4049a68642f7d94f1cfc70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.616351  9971 tablet_service.cc:1511] Processing CreateTablet for tablet ba93a39b54a742c6b391a2eb491f3d79 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:10.617681  9897 tablet_service.cc:1511] Processing CreateTablet for tablet ba93a39b54a742c6b391a2eb491f3d79 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:05:10.618539  9971 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba93a39b54a742c6b391a2eb491f3d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.619155  9897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba93a39b54a742c6b391a2eb491f3d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.618608  9750 tablet_service.cc:1511] Processing CreateTablet for tablet 148fd9eeaa2647bca8ad75116e461ac6 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:10.620239  9750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 148fd9eeaa2647bca8ad75116e461ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.620338  9749 tablet_service.cc:1511] Processing CreateTablet for tablet aaf1d483dd4049a68642f7d94f1cfc70 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:05:10.616351  9898 tablet_service.cc:1511] Processing CreateTablet for tablet 778ba1e8c7b04438a6633db058c14508 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:10.624101  9898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778ba1e8c7b04438a6633db058c14508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.609225  9973 tablet_service.cc:1511] Processing CreateTablet for tablet 148fd9eeaa2647bca8ad75116e461ac6 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:05:10.625829  9748 tablet_service.cc:1511] Processing CreateTablet for tablet 778ba1e8c7b04438a6633db058c14508 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:05:10.626973  9973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 148fd9eeaa2647bca8ad75116e461ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.609567  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaf1d483dd4049a68642f7d94f1cfc70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.628942  9747 tablet_service.cc:1511] Processing CreateTablet for tablet 251be49566d64412bd3d6e3d72c5795e (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:10.622088  9749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaf1d483dd4049a68642f7d94f1cfc70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.630879  9822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 251be49566d64412bd3d6e3d72c5795e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.635370  9747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 251be49566d64412bd3d6e3d72c5795e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.639434  9748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778ba1e8c7b04438a6633db058c14508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.641920  9823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba93a39b54a742c6b391a2eb491f3d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.662739  9896 tablet_service.cc:1511] Processing CreateTablet for tablet 251be49566d64412bd3d6e3d72c5795e (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:05:10.664495  9896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 251be49566d64412bd3d6e3d72c5795e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.666958  9746 tablet_service.cc:1511] Processing CreateTablet for tablet 4b0376b68ba34b8596a5ad96711a0c6e (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:10.668721  9746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b0376b68ba34b8596a5ad96711a0c6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.669472  9970 tablet_service.cc:1511] Processing CreateTablet for tablet 4b0376b68ba34b8596a5ad96711a0c6e (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:10.671159  9970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b0376b68ba34b8596a5ad96711a0c6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.699252 10029 tablet_bootstrap.cc:492] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:10.701696  9821 tablet_service.cc:1511] Processing CreateTablet for tablet 4b0376b68ba34b8596a5ad96711a0c6e (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:05:10.703802  9895 tablet_service.cc:1511] Processing CreateTablet for tablet c2684c424702483b92758b1173702381 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:10.705440 10030 tablet_bootstrap.cc:492] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:10.712723 10028 tablet_bootstrap.cc:492] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:10.712888  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2684c424702483b92758b1173702381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.720484 10028 tablet_bootstrap.cc:654] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:10.720631 10030 tablet_bootstrap.cc:654] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:10.736470  9821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b0376b68ba34b8596a5ad96711a0c6e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:10.744717  9706 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:10.748368  9972 tablet_service.cc:1511] Processing CreateTablet for tablet c2684c424702483b92758b1173702381 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:10.750349  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2684c424702483b92758b1173702381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.770095  9750 tablet_service.cc:1511] Processing CreateTablet for tablet c2684c424702483b92758b1173702381 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:05:10.771587  9750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2684c424702483b92758b1173702381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.772475  9822 tablet_service.cc:1511] Processing CreateTablet for tablet adca2c30b75149c988cf71f9effb9d6c (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:10.773790  9822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adca2c30b75149c988cf71f9effb9d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.778983 10033 tablet_bootstrap.cc:492] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:10.793260  9895 tablet_service.cc:1511] Processing CreateTablet for tablet adca2c30b75149c988cf71f9effb9d6c (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:10.796373  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adca2c30b75149c988cf71f9effb9d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.798331 10029 tablet_bootstrap.cc:654] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:10.800772  9973 tablet_service.cc:1511] Processing CreateTablet for tablet adca2c30b75149c988cf71f9effb9d6c (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:05:10.802469  9973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adca2c30b75149c988cf71f9effb9d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.802763  9749 tablet_service.cc:1511] Processing CreateTablet for tablet d41e6a20c04a4d99ba9b1ea99106480a (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:05:10.808133  9749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d41e6a20c04a4d99ba9b1ea99106480a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.813174 10028 tablet_bootstrap.cc:492] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:10.813793 10028 ts_tablet_manager.cc:1403] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.101s	user 0.032s	sys 0.025s
I20260709 13:05:10.826506  9896 tablet_service.cc:1511] Processing CreateTablet for tablet d41e6a20c04a4d99ba9b1ea99106480a (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:05:10.835923  9746 tablet_service.cc:1511] Processing CreateTablet for tablet aec2f732f8024a00ae0b4d7bc00bb8e3 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:10.837471  9746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aec2f732f8024a00ae0b4d7bc00bb8e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.839380 10030 tablet_bootstrap.cc:492] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:10.840303 10030 ts_tablet_manager.cc:1403] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.138s	user 0.036s	sys 0.042s
I20260709 13:05:10.840087  9824 tablet_service.cc:1511] Processing CreateTablet for tablet d41e6a20c04a4d99ba9b1ea99106480a (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:05:10.841713  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d41e6a20c04a4d99ba9b1ea99106480a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.844122 10030 raft_consensus.cc:359] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:10.845068 10030 raft_consensus.cc:385] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:10.845469 10030 raft_consensus.cc:740] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:10.846482  9823 tablet_service.cc:1511] Processing CreateTablet for tablet aec2f732f8024a00ae0b4d7bc00bb8e3 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:10.846606 10030 consensus_queue.cc:260] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:10.848066  9823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aec2f732f8024a00ae0b4d7bc00bb8e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.828418 10028 raft_consensus.cc:359] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:10.831089  9970 tablet_service.cc:1511] Processing CreateTablet for tablet aec2f732f8024a00ae0b4d7bc00bb8e3 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:05:10.859017  9970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aec2f732f8024a00ae0b4d7bc00bb8e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.852478  9896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d41e6a20c04a4d99ba9b1ea99106480a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.862699 10028 raft_consensus.cc:385] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:10.863128 10028 raft_consensus.cc:740] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:10.866605 10028 consensus_queue.cc:260] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:10.820973 10033 tablet_bootstrap.cc:654] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:10.879299 10029 tablet_bootstrap.cc:492] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:10.880136 10029 ts_tablet_manager.cc:1403] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.181s	user 0.025s	sys 0.017s
I20260709 13:05:10.885193 10029 raft_consensus.cc:359] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:10.886265 10029 raft_consensus.cc:385] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:10.886706 10029 raft_consensus.cc:740] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:10.896248 10029 consensus_queue.cc:260] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:10.913376  9895 tablet_service.cc:1511] Processing CreateTablet for tablet a8a61e076bba425385e5327d369522de (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:10.916464 10008 heartbeater.cc:499] Master 127.0.107.254:42979 was elected leader, sending a full tablet report...
I20260709 13:05:10.916603  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8a61e076bba425385e5327d369522de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.916498  9972 tablet_service.cc:1511] Processing CreateTablet for tablet a8a61e076bba425385e5327d369522de (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:10.918299  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8a61e076bba425385e5327d369522de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.922214  9825 tablet_service.cc:1511] Processing CreateTablet for tablet b011ab28e64b483db8daaf9ed27f45ce (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:10.923979  9825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b011ab28e64b483db8daaf9ed27f45ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.927362  9785 heartbeater.cc:499] Master 127.0.107.254:42979 was elected leader, sending a full tablet report...
I20260709 13:05:10.932117  9750 tablet_service.cc:1511] Processing CreateTablet for tablet a8a61e076bba425385e5327d369522de (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:05:10.933594  9750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8a61e076bba425385e5327d369522de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.944810  9934 heartbeater.cc:499] Master 127.0.107.254:42979 was elected leader, sending a full tablet report...
I20260709 13:05:10.948657 10030 ts_tablet_manager.cc:1434] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.108s	user 0.020s	sys 0.032s
I20260709 13:05:10.972966 10028 ts_tablet_manager.cc:1434] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.159s	user 0.009s	sys 0.006s
I20260709 13:05:10.974172 10028 tablet_bootstrap.cc:492] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:10.982381 10033 tablet_bootstrap.cc:492] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:10.991849  9973 tablet_service.cc:1511] Processing CreateTablet for tablet b011ab28e64b483db8daaf9ed27f45ce (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:10.992537  9746 tablet_service.cc:1511] Processing CreateTablet for tablet eeefc94068a14a0b9dd656a55fd68ccc (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:10.995985  9895 tablet_service.cc:1511] Processing CreateTablet for tablet b011ab28e64b483db8daaf9ed27f45ce (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:05:10.998839 10030 tablet_bootstrap.cc:492] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:10.999660  9746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeefc94068a14a0b9dd656a55fd68ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:10.999994 10033 ts_tablet_manager.cc:1403] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.221s	user 0.011s	sys 0.023s
I20260709 13:05:11.001202  9825 tablet_service.cc:1511] Processing CreateTablet for tablet eeefc94068a14a0b9dd656a55fd68ccc (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:11.002651  9825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeefc94068a14a0b9dd656a55fd68ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.005128 10033 raft_consensus.cc:359] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.005959 10039 raft_consensus.cc:493] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.011396 10039 raft_consensus.cc:515] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:11.007782  9973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b011ab28e64b483db8daaf9ed27f45ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.016430 10028 tablet_bootstrap.cc:654] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:11.008504  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b011ab28e64b483db8daaf9ed27f45ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.023550  9896 tablet_service.cc:1511] Processing CreateTablet for tablet eeefc94068a14a0b9dd656a55fd68ccc (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:05:11.024318 10029 ts_tablet_manager.cc:1434] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.144s	user 0.029s	sys 0.028s
I20260709 13:05:11.025287  9896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeefc94068a14a0b9dd656a55fd68ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.025537 10029 tablet_bootstrap.cc:492] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:11.028563 10009 tablet.cc:2406] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:05:11.010056 10033 raft_consensus.cc:385] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:11.039287 10029 tablet_bootstrap.cc:654] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:11.043423 10030 tablet_bootstrap.cc:654] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:11.041762 10039 leader_election.cc:290] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:11.040606 10033 raft_consensus.cc:740] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:11.057569  9860 heartbeater.cc:499] Master 127.0.107.254:42979 was elected leader, sending a full tablet report...
I20260709 13:05:11.068279  9972 tablet_service.cc:1511] Processing CreateTablet for tablet 6a2d12d5566b4c82a5572035a797a82d (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:11.069736  9823 tablet_service.cc:1511] Processing CreateTablet for tablet 6a2d12d5566b4c82a5572035a797a82d (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:11.074622  9823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2d12d5566b4c82a5572035a797a82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.074935  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2d12d5566b4c82a5572035a797a82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.080227  9750 tablet_service.cc:1511] Processing CreateTablet for tablet 6a2d12d5566b4c82a5572035a797a82d (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:05:11.082062  9750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2d12d5566b4c82a5572035a797a82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.047777 10033 consensus_queue.cc:260] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.100049 10028 tablet_bootstrap.cc:492] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:11.101696 10028 ts_tablet_manager.cc:1403] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.128s	user 0.012s	sys 0.016s
I20260709 13:05:11.107594 10033 ts_tablet_manager.cc:1434] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.107s	user 0.011s	sys 0.008s
I20260709 13:05:11.108815 10033 tablet_bootstrap.cc:492] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:11.110695  9899 tablet_service.cc:1511] Processing CreateTablet for tablet af25fe6172fa4032a6570d9a3c7757c6 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:11.112838  9899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af25fe6172fa4032a6570d9a3c7757c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.116619 10033 tablet_bootstrap.cc:654] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:11.123446  9970 tablet_service.cc:1511] Processing CreateTablet for tablet af25fe6172fa4032a6570d9a3c7757c6 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:11.125133  9970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af25fe6172fa4032a6570d9a3c7757c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.099275 10045 raft_consensus.cc:493] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.161595  9746 tablet_service.cc:1511] Processing CreateTablet for tablet af25fe6172fa4032a6570d9a3c7757c6 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:05:11.162468 10029 tablet_bootstrap.cc:492] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:11.162276  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778ba1e8c7b04438a6633db058c14508" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:11.163014  9746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af25fe6172fa4032a6570d9a3c7757c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.163020 10029 ts_tablet_manager.cc:1403] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.138s	user 0.013s	sys 0.034s
I20260709 13:05:11.163645 10036 raft_consensus.cc:493] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.166406  9824 tablet_service.cc:1511] Processing CreateTablet for tablet aa9f688643024d419097e12e80ced0d2 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:11.168011 10029 raft_consensus.cc:359] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.168629  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778ba1e8c7b04438a6633db058c14508" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:11.168970 10029 raft_consensus.cc:385] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:11.169313 10029 raft_consensus.cc:740] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:11.169397  9983 raft_consensus.cc:2468] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:11.170338 10029 consensus_queue.cc:260] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
W20260709 13:05:11.171446  9726 leader_election.cc:343] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:11.172245 10036 raft_consensus.cc:515] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.106734 10028 raft_consensus.cc:359] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:11.174108 10028 raft_consensus.cc:385] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:11.174425 10028 raft_consensus.cc:740] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:11.176014 10029 ts_tablet_manager.cc:1434] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:05:11.176640  9724 leader_election.cc:304] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 6e10aab26c644389b8161974dcabc5d0
I20260709 13:05:11.177479 10039 raft_consensus.cc:493] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.178032 10038 raft_consensus.cc:2804] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:11.177932 10039 raft_consensus.cc:515] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.178467 10038 raft_consensus.cc:493] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.178821 10038 raft_consensus.cc:3060] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:11.181710 10039 leader_election.cc:290] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:11.183791 10030 tablet_bootstrap.cc:492] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:11.184401 10030 ts_tablet_manager.cc:1403] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.206s	user 0.025s	sys 0.030s
I20260709 13:05:11.184109 10028 consensus_queue.cc:260] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:11.186388  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa9f688643024d419097e12e80ced0d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.187950  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148fd9eeaa2647bca8ad75116e461ac6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:11.154340 10045 raft_consensus.cc:515] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.188997 10029 tablet_bootstrap.cc:492] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:11.189607  9724 leader_election.cc:343] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:11.191890 10036 leader_election.cc:290] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:11.187983 10030 raft_consensus.cc:359] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.224814 10030 raft_consensus.cc:385] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:11.225252 10033 tablet_bootstrap.cc:492] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:11.232847 10030 raft_consensus.cc:740] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:11.233083 10033 ts_tablet_manager.cc:1403] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.124s	user 0.013s	sys 0.010s
I20260709 13:05:11.234683 10030 consensus_queue.cc:260] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.236824 10033 raft_consensus.cc:359] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:11.246199 10033 raft_consensus.cc:385] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:11.246451 10029 tablet_bootstrap.cc:654] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:11.231972 10045 leader_election.cc:290] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:11.240864 10036 raft_consensus.cc:493] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.246590 10033 raft_consensus.cc:740] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:11.247279 10036 raft_consensus.cc:515] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:11.247991 10033 consensus_queue.cc:260] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:11.249912 10036 leader_election.cc:290] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:11.250273 10033 ts_tablet_manager.cc:1434] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.001s
I20260709 13:05:11.218253 10038 raft_consensus.cc:515] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:11.276940  9749 tablet_service.cc:1511] Processing CreateTablet for tablet 3505eea4224942d7a938f9eaf7427723 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:11.272430 10033 tablet_bootstrap.cc:492] T ba93a39b54a742c6b391a2eb491f3d79 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:11.280912  9973 tablet_service.cc:1511] Processing CreateTablet for tablet aa9f688643024d419097e12e80ced0d2 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:11.288152 10030 ts_tablet_manager.cc:1434] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.103s	user 0.008s	sys 0.003s
I20260709 13:05:11.289674 10030 tablet_bootstrap.cc:492] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:11.290221  9749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3505eea4224942d7a938f9eaf7427723. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.298688  9973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa9f688643024d419097e12e80ced0d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.323302  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778ba1e8c7b04438a6633db058c14508" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:11.324340  9983 raft_consensus.cc:3060] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:11.319526 10028 ts_tablet_manager.cc:1434] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.217s	user 0.024s	sys 0.032s
I20260709 13:05:11.327843 10038 leader_election.cc:290] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:11.333582 10033 tablet_bootstrap.cc:654] T ba93a39b54a742c6b391a2eb491f3d79 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:11.333817  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba93a39b54a742c6b391a2eb491f3d79" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:11.334600  9982 raft_consensus.cc:2468] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:11.331768  9896 tablet_service.cc:1511] Processing CreateTablet for tablet aa9f688643024d419097e12e80ced0d2 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:05:11.336694  9896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa9f688643024d419097e12e80ced0d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.338773 10028 tablet_bootstrap.cc:492] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:11.340183  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2684c424702483b92758b1173702381" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:11.341005  9983 raft_consensus.cc:2468] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:11.351379  9724 leader_election.cc:304] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:11.352293  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778ba1e8c7b04438a6633db058c14508" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:11.353803 10045 raft_consensus.cc:493] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.354408 10045 raft_consensus.cc:515] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:11.355276  9972 tablet_service.cc:1511] Processing CreateTablet for tablet 23871dd822614a5da07c2289f80141c1 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:11.356798  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23871dd822614a5da07c2289f80141c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.356649  9823 tablet_service.cc:1511] Processing CreateTablet for tablet 3505eea4224942d7a938f9eaf7427723 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:11.359869 10045 leader_election.cc:290] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
W20260709 13:05:11.361847  9726 leader_election.cc:343] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:11.366631  9823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3505eea4224942d7a938f9eaf7427723. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.373028  9873 leader_election.cc:304] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 
I20260709 13:05:11.378525  9747 tablet_service.cc:1511] Processing CreateTablet for tablet 23871dd822614a5da07c2289f80141c1 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:11.380235  9747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23871dd822614a5da07c2289f80141c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.382977 10049 raft_consensus.cc:2804] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:11.383752 10049 raft_consensus.cc:493] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.384302 10049 raft_consensus.cc:3060] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:11.389997  9873 leader_election.cc:343] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:11.396860  9895 tablet_service.cc:1511] Processing CreateTablet for tablet 3505eea4224942d7a938f9eaf7427723 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:05:11.398247  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3505eea4224942d7a938f9eaf7427723. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.371438 10038 raft_consensus.cc:2804] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:11.403425 10038 raft_consensus.cc:697] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:11.415299 10038 consensus_queue.cc:237] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:11.431958  9825 tablet_service.cc:1511] Processing CreateTablet for tablet 23871dd822614a5da07c2289f80141c1 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:05:11.433401  9825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23871dd822614a5da07c2289f80141c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.433195  9898 tablet_service.cc:1511] Processing CreateTablet for tablet 755ff96760b04fd49a520c059f90b1fe (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:11.434587  9898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 755ff96760b04fd49a520c059f90b1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.438650  9970 tablet_service.cc:1511] Processing CreateTablet for tablet 755ff96760b04fd49a520c059f90b1fe (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:11.446758  9970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 755ff96760b04fd49a520c059f90b1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.464609 10029 tablet_bootstrap.cc:492] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:11.465245 10029 ts_tablet_manager.cc:1403] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.288s	user 0.009s	sys 0.013s
I20260709 13:05:11.465343  9746 tablet_service.cc:1511] Processing CreateTablet for tablet 755ff96760b04fd49a520c059f90b1fe (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:05:11.467777 10030 tablet_bootstrap.cc:654] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:11.466847  9746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 755ff96760b04fd49a520c059f90b1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.470419  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251be49566d64412bd3d6e3d72c5795e" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
W20260709 13:05:11.472642  9801 leader_election.cc:343] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:11.480136 10028 tablet_bootstrap.cc:654] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:11.481312  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2684c424702483b92758b1173702381" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:11.491546  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba93a39b54a742c6b391a2eb491f3d79" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
W20260709 13:05:11.487969  9872 leader_election.cc:343] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:11.493108  9872 leader_election.cc:304] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:11.492995  9821 tablet_service.cc:1511] Processing CreateTablet for tablet ea4f626ff2a94a99bb6505423eba3352 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:11.494110 10036 raft_consensus.cc:2749] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:11.494649  9821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea4f626ff2a94a99bb6505423eba3352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.482264  9657 catalog_manager.cc:5697] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:11.498167  9899 tablet_service.cc:1511] Processing CreateTablet for tablet ea4f626ff2a94a99bb6505423eba3352 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:11.499593  9899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea4f626ff2a94a99bb6505423eba3352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.502240  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148fd9eeaa2647bca8ad75116e461ac6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
W20260709 13:05:11.502519  9874 leader_election.cc:343] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:11.502993  9835 raft_consensus.cc:2468] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:11.503837  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148fd9eeaa2647bca8ad75116e461ac6" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:11.504559  9760 raft_consensus.cc:2468] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:11.504827  9725 leader_election.cc:304] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:11.506310  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251be49566d64412bd3d6e3d72c5795e" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:11.507753 10038 raft_consensus.cc:2804] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:11.515302 10038 raft_consensus.cc:493] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.515869 10038 raft_consensus.cc:3060] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:11.515447 10029 raft_consensus.cc:359] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:11.516566 10029 raft_consensus.cc:385] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:11.516145 10049 raft_consensus.cc:515] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.521641  9973 tablet_service.cc:1511] Processing CreateTablet for tablet ea4f626ff2a94a99bb6505423eba3352 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:05:11.527007  9798 leader_election.cc:304] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
W20260709 13:05:11.528080  9798 leader_election.cc:343] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:11.528561  9798 leader_election.cc:304] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:11.529692 10045 raft_consensus.cc:2804] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:11.530081 10045 raft_consensus.cc:493] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.530434 10045 raft_consensus.cc:3060] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:11.530486 10033 tablet_bootstrap.cc:492] T ba93a39b54a742c6b391a2eb491f3d79 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:11.531200 10033 ts_tablet_manager.cc:1403] T ba93a39b54a742c6b391a2eb491f3d79 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.266s	user 0.014s	sys 0.006s
I20260709 13:05:11.540874  9897 tablet_service.cc:1511] Processing CreateTablet for tablet 1dddadb853be408b8e6a1103a45ce0a1 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:11.541981 10044 raft_consensus.cc:2749] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:11.543503  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148fd9eeaa2647bca8ad75116e461ac6" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:11.541190  9824 tablet_service.cc:1511] Processing CreateTablet for tablet 1dddadb853be408b8e6a1103a45ce0a1 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:11.549057  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dddadb853be408b8e6a1103a45ce0a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.551548  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba93a39b54a742c6b391a2eb491f3d79" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:11.551903 10045 raft_consensus.cc:515] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.541185 10049 leader_election.cc:290] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:11.552871  9983 raft_consensus.cc:3060] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:11.522279 10029 raft_consensus.cc:740] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
W20260709 13:05:11.555538  9799 leader_election.cc:343] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:11.560022  9897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dddadb853be408b8e6a1103a45ce0a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.563413 10029 consensus_queue.cc:260] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:11.565850 10033 raft_consensus.cc:359] T ba93a39b54a742c6b391a2eb491f3d79 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.566844 10029 ts_tablet_manager.cc:1434] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.101s	user 0.009s	sys 0.001s
I20260709 13:05:11.567034 10033 raft_consensus.cc:385] T ba93a39b54a742c6b391a2eb491f3d79 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:11.567585 10033 raft_consensus.cc:740] T ba93a39b54a742c6b391a2eb491f3d79 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:11.568063 10029 tablet_bootstrap.cc:492] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:11.571718  9973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea4f626ff2a94a99bb6505423eba3352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.572933  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba93a39b54a742c6b391a2eb491f3d79" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:11.572983  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148fd9eeaa2647bca8ad75116e461ac6" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:11.579907  9747 tablet_service.cc:1511] Processing CreateTablet for tablet 1dddadb853be408b8e6a1103a45ce0a1 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:05:11.579484  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148fd9eeaa2647bca8ad75116e461ac6" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:11.592003  9747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dddadb853be408b8e6a1103a45ce0a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.586495 10029 tablet_bootstrap.cc:654] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:11.579588 10045 leader_election.cc:290] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
W20260709 13:05:11.593138  9799 leader_election.cc:343] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:11.579784 10033 consensus_queue.cc:260] T ba93a39b54a742c6b391a2eb491f3d79 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.598291  9835 raft_consensus.cc:3060] T ba93a39b54a742c6b391a2eb491f3d79 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:11.599692 10033 ts_tablet_manager.cc:1434] T ba93a39b54a742c6b391a2eb491f3d79 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.002s
I20260709 13:05:11.600793 10033 tablet_bootstrap.cc:492] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:11.601682 10038 raft_consensus.cc:515] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.602128  9896 tablet_service.cc:1511] Processing CreateTablet for tablet 716c25a1e46547c999c1c7c860e23f3b (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:11.603150  9759 raft_consensus.cc:2393] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a1cbca39e34743d8a40816c3bc5f2030 in current term 1: Already voted for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in this term.
I20260709 13:05:11.603664  9896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 716c25a1e46547c999c1c7c860e23f3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.605625  9825 tablet_service.cc:1511] Processing CreateTablet for tablet e80fbd6291544644996b96fec0e8f2c3 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:05:11.607204  9798 leader_election.cc:304] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:11.608549 10045 raft_consensus.cc:2749] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:11.609135  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148fd9eeaa2647bca8ad75116e461ac6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:11.609413  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148fd9eeaa2647bca8ad75116e461ac6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:11.610009  9834 raft_consensus.cc:2393] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 1: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
W20260709 13:05:11.611068  9724 leader_election.cc:343] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:11.611675  9725 leader_election.cc:304] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:11.612010  9825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e80fbd6291544644996b96fec0e8f2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.612596 10039 raft_consensus.cc:2749] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:11.621779  9971 tablet_service.cc:1511] Processing CreateTablet for tablet 716c25a1e46547c999c1c7c860e23f3b (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:11.623378  9971 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 716c25a1e46547c999c1c7c860e23f3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.578840  9983 raft_consensus.cc:2468] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:11.632607  9873 leader_election.cc:304] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 
I20260709 13:05:11.633948 10049 raft_consensus.cc:2804] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:11.634816 10049 raft_consensus.cc:697] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:11.636482 10049 consensus_queue.cc:237] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.638347 10039 raft_consensus.cc:493] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.639050 10039 raft_consensus.cc:515] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:11.639891 10033 tablet_bootstrap.cc:654] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:11.641322 10039 leader_election.cc:290] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:11.643229  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf1d483dd4049a68642f7d94f1cfc70" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
W20260709 13:05:11.644969  9726 leader_election.cc:343] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:11.646261 10028 tablet_bootstrap.cc:492] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:11.647184 10038 leader_election.cc:290] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:11.647847  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf1d483dd4049a68642f7d94f1cfc70" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:11.648514 10028 ts_tablet_manager.cc:1403] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.311s	user 0.030s	sys 0.026s
W20260709 13:05:11.649430  9725 leader_election.cc:343] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:11.649916  9725 leader_election.cc:304] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:11.664979  9749 tablet_service.cc:1511] Processing CreateTablet for tablet 716c25a1e46547c999c1c7c860e23f3b (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:05:11.666348  9749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 716c25a1e46547c999c1c7c860e23f3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.652307 10038 raft_consensus.cc:2749] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:11.672536  9898 tablet_service.cc:1511] Processing CreateTablet for tablet c0c62d82ebe5420bb298d787cda221aa (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:05:11.677238  9898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c62d82ebe5420bb298d787cda221aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.680591  9970 tablet_service.cc:1511] Processing CreateTablet for tablet c0c62d82ebe5420bb298d787cda221aa (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:05:11.682030  9970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c62d82ebe5420bb298d787cda221aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.687114  9824 tablet_service.cc:1511] Processing CreateTablet for tablet c0c62d82ebe5420bb298d787cda221aa (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:05:11.688655  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c62d82ebe5420bb298d787cda221aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.689049  9750 tablet_service.cc:1511] Processing CreateTablet for tablet db11dbab2c874e55b70a46aba53095e0 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:11.690387  9750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db11dbab2c874e55b70a46aba53095e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.691504  9972 tablet_service.cc:1511] Processing CreateTablet for tablet e80fbd6291544644996b96fec0e8f2c3 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:05:11.693455  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e80fbd6291544644996b96fec0e8f2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.695474  9969 tablet_service.cc:1511] Processing CreateTablet for tablet b189ef5046cd47ebb3d431e2ef24d6fb (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:05:11.696877  9969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b189ef5046cd47ebb3d431e2ef24d6fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.698112 10030 tablet_bootstrap.cc:492] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:11.698657 10030 ts_tablet_manager.cc:1403] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.409s	user 0.015s	sys 0.027s
I20260709 13:05:11.701304  9895 tablet_service.cc:1511] Processing CreateTablet for tablet db11dbab2c874e55b70a46aba53095e0 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:11.702425  9823 tablet_service.cc:1511] Processing CreateTablet for tablet db11dbab2c874e55b70a46aba53095e0 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:05:11.703446  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db11dbab2c874e55b70a46aba53095e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.703860  9823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db11dbab2c874e55b70a46aba53095e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.654589  9746 tablet_service.cc:1511] Processing CreateTablet for tablet e80fbd6291544644996b96fec0e8f2c3 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:05:11.708444  9748 tablet_service.cc:1511] Processing CreateTablet for tablet b189ef5046cd47ebb3d431e2ef24d6fb (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:05:11.709438 10028 raft_consensus.cc:359] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:11.710327 10028 raft_consensus.cc:385] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:11.715049 10028 raft_consensus.cc:740] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:11.716240 10028 consensus_queue.cc:260] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:11.711654  9821 tablet_service.cc:1511] Processing CreateTablet for tablet b189ef5046cd47ebb3d431e2ef24d6fb (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:05:11.712656  9894 tablet_service.cc:1511] Processing CreateTablet for tablet fbbee4fe35264c5aa3ea32adc034b9c7 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:11.723026  9821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b189ef5046cd47ebb3d431e2ef24d6fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.729895  9973 tablet_service.cc:1511] Processing CreateTablet for tablet fbbee4fe35264c5aa3ea32adc034b9c7 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:11.730906  9745 tablet_service.cc:1511] Processing CreateTablet for tablet fbbee4fe35264c5aa3ea32adc034b9c7 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:05:11.731472  9973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbbee4fe35264c5aa3ea32adc034b9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.730541  9658 catalog_manager.cc:5697] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:11.708478  9746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e80fbd6291544644996b96fec0e8f2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.724361 10028 ts_tablet_manager.cc:1434] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.075s	user 0.008s	sys 0.002s
I20260709 13:05:11.738962 10028 tablet_bootstrap.cc:492] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:11.740418 10033 tablet_bootstrap.cc:492] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:11.740088  9822 tablet_service.cc:1511] Processing CreateTablet for tablet c2d7bb9a0b174b168df9040b02c70ab9 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:11.741045 10033 ts_tablet_manager.cc:1403] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.140s	user 0.012s	sys 0.007s
I20260709 13:05:11.741624  9822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2d7bb9a0b174b168df9040b02c70ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.745208  9894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbbee4fe35264c5aa3ea32adc034b9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.702474 10030 raft_consensus.cc:359] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
W20260709 13:05:11.748751  9706 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:11.749723 10030 raft_consensus.cc:385] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:11.750761 10030 raft_consensus.cc:740] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:11.751700 10030 consensus_queue.cc:260] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.752354 10033 raft_consensus.cc:359] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:11.753137 10033 raft_consensus.cc:385] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:11.753437 10033 raft_consensus.cc:740] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:11.754387 10033 consensus_queue.cc:260] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:11.756893  9745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbbee4fe35264c5aa3ea32adc034b9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.757630  9748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b189ef5046cd47ebb3d431e2ef24d6fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.765296 10029 tablet_bootstrap.cc:492] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:11.765878 10029 ts_tablet_manager.cc:1403] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.198s	user 0.038s	sys 0.032s
I20260709 13:05:11.766264  9835 raft_consensus.cc:2468] T ba93a39b54a742c6b391a2eb491f3d79 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:11.769196 10029 raft_consensus.cc:359] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:11.770051 10029 raft_consensus.cc:385] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:11.770358 10029 raft_consensus.cc:740] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:11.771317 10029 consensus_queue.cc:260] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:11.777151 10029 ts_tablet_manager.cc:1434] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:05:11.777657  9899 tablet_service.cc:1511] Processing CreateTablet for tablet c2d7bb9a0b174b168df9040b02c70ab9 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:11.778357 10029 tablet_bootstrap.cc:492] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:11.779081  9899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2d7bb9a0b174b168df9040b02c70ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.796399 10033 ts_tablet_manager.cc:1434] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.000s
I20260709 13:05:11.797655 10033 tablet_bootstrap.cc:492] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:11.802181  9744 tablet_service.cc:1511] Processing CreateTablet for tablet c23fd23d082a4c58ab8404e8b9895ad4 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:11.804194  9744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c23fd23d082a4c58ab8404e8b9895ad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.820735 10038 consensus_queue.cc:1048] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:11.823572  9820 tablet_service.cc:1511] Processing CreateTablet for tablet c23fd23d082a4c58ab8404e8b9895ad4 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:11.826972 10028 tablet_bootstrap.cc:654] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:11.829756  9820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c23fd23d082a4c58ab8404e8b9895ad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.831831  9909 raft_consensus.cc:3060] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:11.844561  9970 tablet_service.cc:1511] Processing CreateTablet for tablet 9579648f7487465aa44ecf73e23731d8 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:11.847893 10070 raft_consensus.cc:493] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.848480 10070 raft_consensus.cc:515] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:11.860594 10030 ts_tablet_manager.cc:1434] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.161s	user 0.008s	sys 0.006s
I20260709 13:05:11.862972  9897 tablet_service.cc:1511] Processing CreateTablet for tablet c23fd23d082a4c58ab8404e8b9895ad4 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:05:11.862663 10070 leader_election.cc:290] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:11.874153  9898 tablet_service.cc:1511] Processing CreateTablet for tablet ab473e8b9be44bdaae62679a55757fe9 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:11.878268 10029 tablet_bootstrap.cc:654] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:11.878945 10038 consensus_queue.cc:1048] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:11.881541 10030 tablet_bootstrap.cc:492] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:11.879004  9970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9579648f7487465aa44ecf73e23731d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.882110  9897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c23fd23d082a4c58ab8404e8b9895ad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.881129  9825 tablet_service.cc:1511] Processing CreateTablet for tablet 9579648f7487465aa44ecf73e23731d8 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:11.887786  9825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9579648f7487465aa44ecf73e23731d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.887867  9750 tablet_service.cc:1511] Processing CreateTablet for tablet 9579648f7487465aa44ecf73e23731d8 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:05:11.889248  9750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9579648f7487465aa44ecf73e23731d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.890556  9898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab473e8b9be44bdaae62679a55757fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.886684 10033 tablet_bootstrap.cc:654] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:11.908496  9895 tablet_service.cc:1511] Processing CreateTablet for tablet 0e044b992b684d52b54e3e8a4aa9720a (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:11.910126  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e044b992b684d52b54e3e8a4aa9720a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.890492  9745 tablet_service.cc:1511] Processing CreateTablet for tablet ab473e8b9be44bdaae62679a55757fe9 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:11.913069  9745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab473e8b9be44bdaae62679a55757fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.936291 10030 tablet_bootstrap.cc:654] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:11.949359  9823 tablet_service.cc:1511] Processing CreateTablet for tablet 0e044b992b684d52b54e3e8a4aa9720a (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:11.950906  9823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e044b992b684d52b54e3e8a4aa9720a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.952426  9824 tablet_service.cc:1511] Processing CreateTablet for tablet 196922ea9dc14e078a3da6af1206f79a (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:11.953794  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 196922ea9dc14e078a3da6af1206f79a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.908488  9969 tablet_service.cc:1511] Processing CreateTablet for tablet c2d7bb9a0b174b168df9040b02c70ab9 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:05:11.965474  9744 tablet_service.cc:1511] Processing CreateTablet for tablet 196922ea9dc14e078a3da6af1206f79a (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:11.966770  9744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 196922ea9dc14e078a3da6af1206f79a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:11.969466  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148fd9eeaa2647bca8ad75116e461ac6" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:11.972265  9835 raft_consensus.cc:2393] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23cbd6234c5e43d89f22c235ed6bb96a in current term 1: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
I20260709 13:05:11.961489  9969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2d7bb9a0b174b168df9040b02c70ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.000315 10045 raft_consensus.cc:493] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.000938 10045 raft_consensus.cc:515] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.001330 10028 tablet_bootstrap.cc:492] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:12.002105 10028 ts_tablet_manager.cc:1403] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.263s	user 0.017s	sys 0.007s
I20260709 13:05:12.005203 10045 leader_election.cc:290] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:12.005527  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf1d483dd4049a68642f7d94f1cfc70" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:12.005545 10028 raft_consensus.cc:359] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:12.007753 10028 raft_consensus.cc:385] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:12.008054 10028 raft_consensus.cc:740] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
W20260709 13:05:12.008493  9801 leader_election.cc:343] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:12.006520  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf1d483dd4049a68642f7d94f1cfc70" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:12.008852 10028 consensus_queue.cc:260] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:12.009436  9759 raft_consensus.cc:2468] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:12.010891  9798 leader_election.cc:304] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 6e10aab26c644389b8161974dcabc5d0
I20260709 13:05:12.011606 10028 ts_tablet_manager.cc:1434] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:05:12.011899 10044 raft_consensus.cc:2804] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:12.012292 10044 raft_consensus.cc:493] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.012601 10044 raft_consensus.cc:3060] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.030508 10044 raft_consensus.cc:515] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.051913 10029 tablet_bootstrap.cc:492] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:12.052554 10029 ts_tablet_manager.cc:1403] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.274s	user 0.043s	sys 0.060s
I20260709 13:05:12.055863 10029 raft_consensus.cc:359] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:12.057631 10029 raft_consensus.cc:385] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:12.058063 10029 raft_consensus.cc:740] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:12.058792 10033 tablet_bootstrap.cc:492] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:12.059407 10033 ts_tablet_manager.cc:1403] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.262s	user 0.016s	sys 0.004s
I20260709 13:05:12.059007 10029 consensus_queue.cc:260] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
W20260709 13:05:12.060024 10079 log.cc:927] Time spent T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:05:12.063195 10028 tablet_bootstrap.cc:492] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:12.064893  9972 tablet_service.cc:1511] Processing CreateTablet for tablet ab473e8b9be44bdaae62679a55757fe9 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:05:12.065202  9973 tablet_service.cc:1511] Processing CreateTablet for tablet 0e044b992b684d52b54e3e8a4aa9720a (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:05:12.066293  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab473e8b9be44bdaae62679a55757fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.067020  9973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e044b992b684d52b54e3e8a4aa9720a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.073689 10030 tablet_bootstrap.cc:492] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:12.074527 10030 ts_tablet_manager.cc:1403] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.193s	user 0.026s	sys 0.006s
I20260709 13:05:12.070482  9895 tablet_service.cc:1511] Processing CreateTablet for tablet 196922ea9dc14e078a3da6af1206f79a (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:05:12.075680  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 196922ea9dc14e078a3da6af1206f79a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.077796 10030 raft_consensus.cc:359] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.082043 10030 raft_consensus.cc:385] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:12.082396 10030 raft_consensus.cc:740] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:12.083316 10030 consensus_queue.cc:260] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.085405 10039 raft_consensus.cc:493] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.085835  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148fd9eeaa2647bca8ad75116e461ac6" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:12.085925 10039 raft_consensus.cc:515] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.086771  9759 raft_consensus.cc:2393] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23cbd6234c5e43d89f22c235ed6bb96a in current term 1: Already voted for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in this term.
I20260709 13:05:12.087354 10044 leader_election.cc:290] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:12.088165 10039 leader_election.cc:290] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:12.088912  9946 leader_election.cc:304] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:12.062630 10033 raft_consensus.cc:359] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.090196 10033 raft_consensus.cc:385] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:12.090258  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf1d483dd4049a68642f7d94f1cfc70" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:12.090626 10033 raft_consensus.cc:740] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:12.092295 10029 ts_tablet_manager.cc:1434] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.003s
I20260709 13:05:12.092530  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b0376b68ba34b8596a5ad96711a0c6e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:12.093431 10029 tablet_bootstrap.cc:492] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:12.094810  9801 leader_election.cc:343] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:12.096769 10030 ts_tablet_manager.cc:1434] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:05:12.097915 10030 tablet_bootstrap.cc:492] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:12.098420  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf1d483dd4049a68642f7d94f1cfc70" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:12.099077  9759 raft_consensus.cc:3060] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.100281 10028 tablet_bootstrap.cc:654] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:12.101445 10072 raft_consensus.cc:3060] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.116859  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b0376b68ba34b8596a5ad96711a0c6e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:12.117662  9983 raft_consensus.cc:2468] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:12.118434  9759 raft_consensus.cc:2468] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:12.118952  9724 leader_election.cc:304] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:12.121415  9798 leader_election.cc:304] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 6e10aab26c644389b8161974dcabc5d0
I20260709 13:05:12.122332 10038 raft_consensus.cc:2804] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:12.122869 10038 raft_consensus.cc:493] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.123301 10038 raft_consensus.cc:3060] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.126219 10044 raft_consensus.cc:2804] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:12.129556 10044 raft_consensus.cc:697] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:12.130421 10044 consensus_queue.cc:237] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.132341 10030 tablet_bootstrap.cc:654] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:12.142478 10072 raft_consensus.cc:2749] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:12.127806 10033 consensus_queue.cc:260] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.151260  9835 raft_consensus.cc:2468] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:12.155746 10033 ts_tablet_manager.cc:1434] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.000s
I20260709 13:05:12.157378 10033 tablet_bootstrap.cc:492] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:12.172640  9972 tablet_service.cc:1511] Processing CreateTablet for tablet 27c539d1a0b54f8ca31dba142cb38599 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:12.174116  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27c539d1a0b54f8ca31dba142cb38599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.176148 10028 tablet_bootstrap.cc:492] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:12.176988 10028 ts_tablet_manager.cc:1403] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.130s	user 0.027s	sys 0.005s
I20260709 13:05:12.178706  9750 tablet_service.cc:1511] Processing CreateTablet for tablet 27c539d1a0b54f8ca31dba142cb38599 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:12.179795  9823 tablet_service.cc:1511] Processing CreateTablet for tablet 27c539d1a0b54f8ca31dba142cb38599 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:05:12.182554  9823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27c539d1a0b54f8ca31dba142cb38599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.188402  9750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27c539d1a0b54f8ca31dba142cb38599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.190860 10028 raft_consensus.cc:359] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.191715 10028 raft_consensus.cc:385] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:12.192032 10028 raft_consensus.cc:740] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:12.192986 10028 consensus_queue.cc:260] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.195708 10028 ts_tablet_manager.cc:1434] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.006s
I20260709 13:05:12.196766 10028 tablet_bootstrap.cc:492] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:12.196779  9895 tablet_service.cc:1511] Processing CreateTablet for tablet d8f99aa4491c4458bb5ebc03bc0cd813 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:12.208534  9745 tablet_service.cc:1511] Processing CreateTablet for tablet d8f99aa4491c4458bb5ebc03bc0cd813 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:12.209107  9969 tablet_service.cc:1511] Processing CreateTablet for tablet d8f99aa4491c4458bb5ebc03bc0cd813 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:05:12.209715 10029 tablet_bootstrap.cc:654] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:12.210245  9745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f99aa4491c4458bb5ebc03bc0cd813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.210646  9969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f99aa4491c4458bb5ebc03bc0cd813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.216773  9898 tablet_service.cc:1511] Processing CreateTablet for tablet e01c76363b154905982dc75dbdead3be (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:12.220914  9973 tablet_service.cc:1511] Processing CreateTablet for tablet e01c76363b154905982dc75dbdead3be (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:12.221748 10049 consensus_queue.cc:1048] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:12.222415  9973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e01c76363b154905982dc75dbdead3be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.209107  9820 tablet_service.cc:1511] Processing CreateTablet for tablet e01c76363b154905982dc75dbdead3be (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:05:12.224239  9820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e01c76363b154905982dc75dbdead3be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.228159 10033 tablet_bootstrap.cc:654] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:12.230093 10049 raft_consensus.cc:493] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.230693 10049 raft_consensus.cc:515] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.234370  9744 tablet_service.cc:1511] Processing CreateTablet for tablet e82b4b9bc0c546adac703578be161959 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:05:12.234767 10038 raft_consensus.cc:515] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.233896  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf1d483dd4049a68642f7d94f1cfc70" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:12.235862  9744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e82b4b9bc0c546adac703578be161959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.236332  9759 raft_consensus.cc:2393] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e10aab26c644389b8161974dcabc5d0 in current term 1: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
I20260709 13:05:12.237936  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b0376b68ba34b8596a5ad96711a0c6e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:12.238590  9982 raft_consensus.cc:3060] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.240267  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf1d483dd4049a68642f7d94f1cfc70" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:12.242143  9874 leader_election.cc:304] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:12.243165 10036 raft_consensus.cc:3060] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.245780 10049 leader_election.cc:290] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:12.249274  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b0376b68ba34b8596a5ad96711a0c6e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:12.249961  9834 raft_consensus.cc:3060] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.253950  9824 tablet_service.cc:1511] Processing CreateTablet for tablet e82b4b9bc0c546adac703578be161959 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:05:12.255417  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e82b4b9bc0c546adac703578be161959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.257889  9834 raft_consensus.cc:2468] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:12.259233  9725 leader_election.cc:304] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:12.260102 10039 raft_consensus.cc:2804] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:12.260653 10039 raft_consensus.cc:697] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:12.260555 10049 consensus_queue.cc:1048] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:12.261708 10039 consensus_queue.cc:237] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.260864 10038 leader_election.cc:290] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:12.266784 10036 raft_consensus.cc:2749] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:12.279560  9970 tablet_service.cc:1511] Processing CreateTablet for tablet dc0e147704a7413397055b8d3a812a8e (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:12.281648  9970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc0e147704a7413397055b8d3a812a8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.284341  9982 raft_consensus.cc:2468] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:12.285696  9750 tablet_service.cc:1511] Processing CreateTablet for tablet dc0e147704a7413397055b8d3a812a8e (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:12.290524  9750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc0e147704a7413397055b8d3a812a8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.289194 10033 tablet_bootstrap.cc:492] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:12.304400 10033 ts_tablet_manager.cc:1403] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.147s	user 0.013s	sys 0.000s
I20260709 13:05:12.309705 10036 raft_consensus.cc:493] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.310523 10036 raft_consensus.cc:515] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:12.316617  9824 tablet_service.cc:1511] Processing CreateTablet for tablet dc0e147704a7413397055b8d3a812a8e (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:05:12.317700  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251be49566d64412bd3d6e3d72c5795e" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:12.318101  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc0e147704a7413397055b8d3a812a8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.318471  9759 raft_consensus.cc:2468] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:12.319796  9969 tablet_service.cc:1511] Processing CreateTablet for tablet 168ffc86a17d47e8aba438779717eab6 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:05:12.320740  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251be49566d64412bd3d6e3d72c5795e" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:12.321110  9969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 168ffc86a17d47e8aba438779717eab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.321504  9834 raft_consensus.cc:2468] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:12.323318  9874 leader_election.cc:304] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:12.325673 10036 leader_election.cc:290] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:12.326514 10036 raft_consensus.cc:2804] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:12.314114  9658 catalog_manager.cc:5697] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:12.326900 10036 raft_consensus.cc:493] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.329972 10036 raft_consensus.cc:3060] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.344512  9898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e01c76363b154905982dc75dbdead3be. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:12.344995  9873 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 90680.4us
I20260709 13:05:12.339372  9744 tablet_service.cc:1511] Processing CreateTablet for tablet 168ffc86a17d47e8aba438779717eab6 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:05:12.351447  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f99aa4491c4458bb5ebc03bc0cd813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.355808  9744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 168ffc86a17d47e8aba438779717eab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.356406  9899 tablet_service.cc:1511] Processing CreateTablet for tablet e82b4b9bc0c546adac703578be161959 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:05:12.358355  9899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e82b4b9bc0c546adac703578be161959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.360855  9972 tablet_service.cc:1511] Processing CreateTablet for tablet 2b228f0642a242db93a7d11c1afab0ea (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:05:12.362250  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b228f0642a242db93a7d11c1afab0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.365041 10030 tablet_bootstrap.cc:492] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:12.365808 10030 ts_tablet_manager.cc:1403] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.268s	user 0.016s	sys 0.015s
I20260709 13:05:12.367938  9750 tablet_service.cc:1511] Processing CreateTablet for tablet 1064d1c0e3ba4ab68fb20fd5f093abef (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:12.369299  9750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1064d1c0e3ba4ab68fb20fd5f093abef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.372705  9897 tablet_service.cc:1511] Processing CreateTablet for tablet 168ffc86a17d47e8aba438779717eab6 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:05:12.379523  9897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 168ffc86a17d47e8aba438779717eab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.327706 10033 raft_consensus.cc:359] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:12.382222 10033 raft_consensus.cc:385] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:12.382537 10033 raft_consensus.cc:740] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:12.383912  9894 tablet_service.cc:1511] Processing CreateTablet for tablet 2b228f0642a242db93a7d11c1afab0ea (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:05:12.386468  9894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b228f0642a242db93a7d11c1afab0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.396855  9895 tablet_service.cc:1511] Processing CreateTablet for tablet 1064d1c0e3ba4ab68fb20fd5f093abef (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:12.401759  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1064d1c0e3ba4ab68fb20fd5f093abef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.407532  9824 tablet_service.cc:1511] Processing CreateTablet for tablet 1064d1c0e3ba4ab68fb20fd5f093abef (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:05:12.383477 10033 consensus_queue.cc:260] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:12.408803  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1064d1c0e3ba4ab68fb20fd5f093abef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.411782 10029 tablet_bootstrap.cc:492] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:12.412518 10029 ts_tablet_manager.cc:1403] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.319s	user 0.015s	sys 0.018s
I20260709 13:05:12.439595 10029 raft_consensus.cc:359] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:12.440531 10029 raft_consensus.cc:385] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:12.440914 10029 raft_consensus.cc:740] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:12.441746 10029 consensus_queue.cc:260] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:12.443312 10030 raft_consensus.cc:359] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:12.444151 10030 raft_consensus.cc:385] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:12.444454 10030 raft_consensus.cc:740] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:12.445201 10030 consensus_queue.cc:260] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:12.445853 10036 raft_consensus.cc:515] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:12.448333 10036 leader_election.cc:290] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:12.448601  9746 tablet_service.cc:1511] Processing CreateTablet for tablet e6f94cfb8a734a83872ed245aeea11e6 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:12.449471  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251be49566d64412bd3d6e3d72c5795e" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:12.454257  9835 raft_consensus.cc:3060] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.451299 10033 ts_tablet_manager.cc:1434] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.147s	user 0.009s	sys 0.000s
I20260709 13:05:12.458086 10033 tablet_bootstrap.cc:492] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:12.451313  9896 tablet_service.cc:1511] Processing CreateTablet for tablet 30b0a6c62c14440c9e613e73781c542e (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:12.450484  9823 tablet_service.cc:1511] Processing CreateTablet for tablet e6f94cfb8a734a83872ed245aeea11e6 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:12.459363  9896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30b0a6c62c14440c9e613e73781c542e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.459668  9823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6f94cfb8a734a83872ed245aeea11e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.461064  9820 tablet_service.cc:1511] Processing CreateTablet for tablet 2b228f0642a242db93a7d11c1afab0ea (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:05:12.451701  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251be49566d64412bd3d6e3d72c5795e" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:12.456874 10030 ts_tablet_manager.cc:1434] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.091s	user 0.010s	sys 0.000s
I20260709 13:05:12.454294  9746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6f94cfb8a734a83872ed245aeea11e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.462993  9759 raft_consensus.cc:3060] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.462893  9820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b228f0642a242db93a7d11c1afab0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.445822 10028 tablet_bootstrap.cc:654] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:12.464169 10029 ts_tablet_manager.cc:1434] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 13:05:12.485256 10030 tablet_bootstrap.cc:492] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:12.490106  9747 tablet_service.cc:1511] Processing CreateTablet for tablet 30b0a6c62c14440c9e613e73781c542e (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:12.491619  9747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30b0a6c62c14440c9e613e73781c542e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.484092 10029 tablet_bootstrap.cc:492] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:12.482844  9972 tablet_service.cc:1511] Processing CreateTablet for tablet e6f94cfb8a734a83872ed245aeea11e6 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:05:12.500154  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6f94cfb8a734a83872ed245aeea11e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.500687 10044 raft_consensus.cc:493] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.501230 10044 raft_consensus.cc:515] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:12.464184  9656 catalog_manager.cc:5697] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:12.504769 10044 leader_election.cc:290] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:12.505512 10030 tablet_bootstrap.cc:654] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:12.506472  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adca2c30b75149c988cf71f9effb9d6c" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
W20260709 13:05:12.513918  9801 leader_election.cc:343] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:12.514731  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adca2c30b75149c988cf71f9effb9d6c" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:12.517064  9799 leader_election.cc:343] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:12.517544  9799 leader_election.cc:304] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0
I20260709 13:05:12.519438 10044 raft_consensus.cc:2749] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:12.536392  9970 tablet_service.cc:1511] Processing CreateTablet for tablet 30b0a6c62c14440c9e613e73781c542e (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:05:12.538275  9970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30b0a6c62c14440c9e613e73781c542e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.540589  9824 tablet_service.cc:1511] Processing CreateTablet for tablet b799cecf522841009ed9a2b1b421be50 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:12.541567  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b799cecf522841009ed9a2b1b421be50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.559290 10029 tablet_bootstrap.cc:654] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:12.563297  9895 tablet_service.cc:1511] Processing CreateTablet for tablet b799cecf522841009ed9a2b1b421be50 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:12.566025  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b799cecf522841009ed9a2b1b421be50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.577857  9759 raft_consensus.cc:2468] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:12.585160  9872 leader_election.cc:304] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:12.586678 10049 raft_consensus.cc:2804] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:12.587249 10049 raft_consensus.cc:697] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:12.582036  9972 tablet_service.cc:1511] Processing CreateTablet for tablet b799cecf522841009ed9a2b1b421be50 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:05:12.589524  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b799cecf522841009ed9a2b1b421be50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.588088 10049 consensus_queue.cc:237] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:12.597486  9750 tablet_service.cc:1511] Processing CreateTablet for tablet 748a545c945244afa32bec8019157d0c (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:05:12.598699  9750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 748a545c945244afa32bec8019157d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.602095 10030 tablet_bootstrap.cc:492] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:12.615937 10033 tablet_bootstrap.cc:654] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:12.591405  9835 raft_consensus.cc:2468] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:12.621168  9824 tablet_service.cc:1511] Processing CreateTablet for tablet 748a545c945244afa32bec8019157d0c (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:05:12.623107  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 748a545c945244afa32bec8019157d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.624029 10028 tablet_bootstrap.cc:492] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:12.624609 10028 ts_tablet_manager.cc:1403] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.428s	user 0.010s	sys 0.017s
I20260709 13:05:12.618888 10030 ts_tablet_manager.cc:1403] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.153s	user 0.029s	sys 0.037s
I20260709 13:05:12.633891 10029 tablet_bootstrap.cc:492] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:12.634560 10029 ts_tablet_manager.cc:1403] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.155s	user 0.007s	sys 0.011s
I20260709 13:05:12.637962 10029 raft_consensus.cc:359] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:12.634783 10030 raft_consensus.cc:359] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.640136 10030 raft_consensus.cc:385] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:12.638829 10029 raft_consensus.cc:385] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:12.640847 10029 raft_consensus.cc:740] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:12.641952 10029 consensus_queue.cc:260] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:12.646808 10044 consensus_queue.cc:1048] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:12.650933  9895 tablet_service.cc:1511] Processing CreateTablet for tablet 748a545c945244afa32bec8019157d0c (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:05:12.652357  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 748a545c945244afa32bec8019157d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.652945  9656 catalog_manager.cc:5697] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:12.657882 10033 tablet_bootstrap.cc:492] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:12.660537 10028 raft_consensus.cc:359] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:12.663885 10028 raft_consensus.cc:385] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:12.662528 10033 ts_tablet_manager.cc:1403] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.205s	user 0.011s	sys 0.020s
I20260709 13:05:12.640637 10030 raft_consensus.cc:740] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:12.664410 10028 raft_consensus.cc:740] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:12.669329  9970 tablet_service.cc:1511] Processing CreateTablet for tablet 10a930896e984848a0516f17a1213ab5 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:12.669839  9746 tablet_service.cc:1511] Processing CreateTablet for tablet 10a930896e984848a0516f17a1213ab5 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:12.670714  9970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10a930896e984848a0516f17a1213ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.671496  9746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10a930896e984848a0516f17a1213ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.669812 10028 consensus_queue.cc:260] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:12.665081 10030 consensus_queue.cc:260] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.674335 10029 ts_tablet_manager.cc:1434] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.003s
I20260709 13:05:12.678450  9824 tablet_service.cc:1511] Processing CreateTablet for tablet 10a930896e984848a0516f17a1213ab5 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:05:12.685596 10033 raft_consensus.cc:359] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.686496 10033 raft_consensus.cc:385] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:12.686830 10033 raft_consensus.cc:740] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:12.687671 10033 consensus_queue.cc:260] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.689097 10030 ts_tablet_manager.cc:1434] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.001s
I20260709 13:05:12.680667  9895 tablet_service.cc:1511] Processing CreateTablet for tablet fe46065991d847dfa3d934961da0830a (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:05:12.693727  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe46065991d847dfa3d934961da0830a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.693888 10030 tablet_bootstrap.cc:492] T adca2c30b75149c988cf71f9effb9d6c P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:12.681327  9972 tablet_service.cc:1511] Processing CreateTablet for tablet fe46065991d847dfa3d934961da0830a (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:05:12.689881  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10a930896e984848a0516f17a1213ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.696798  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe46065991d847dfa3d934961da0830a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.675872 10028 ts_tablet_manager.cc:1434] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.001s
I20260709 13:05:12.700575 10028 tablet_bootstrap.cc:492] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:12.702000 10045 consensus_queue.cc:1048] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:12.708140 10039 raft_consensus.cc:493] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.708757 10039 raft_consensus.cc:515] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:12.711971  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2684c424702483b92758b1173702381" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:12.712343  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2684c424702483b92758b1173702381" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:12.712963  9908 raft_consensus.cc:2468] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:12.713109  9982 raft_consensus.cc:2468] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:12.713999 10039 leader_election.cc:290] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:12.715081  9724 leader_election.cc:304] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:12.715984 10039 raft_consensus.cc:2804] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:12.716356 10039 raft_consensus.cc:493] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.716704 10039 raft_consensus.cc:3060] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.731001 10049 raft_consensus.cc:493] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.731612 10049 raft_consensus.cc:515] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:12.733899 10049 leader_election.cc:290] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:12.715127 10033 ts_tablet_manager.cc:1434] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.049s	user 0.004s	sys 0.005s
I20260709 13:05:12.736546 10033 tablet_bootstrap.cc:492] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:12.736997  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adca2c30b75149c988cf71f9effb9d6c" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:12.738061  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adca2c30b75149c988cf71f9effb9d6c" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:12.738758  9835 raft_consensus.cc:2468] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:12.685839 10029 tablet_bootstrap.cc:492] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:12.739267 10030 tablet_bootstrap.cc:654] T adca2c30b75149c988cf71f9effb9d6c P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:12.747535  9874 leader_election.cc:304] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:12.748445 10049 raft_consensus.cc:2804] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:12.748889  9873 leader_election.cc:343] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:12.750819 10082 consensus_queue.cc:1048] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:12.753548  9746 tablet_service.cc:1511] Processing CreateTablet for tablet fe46065991d847dfa3d934961da0830a (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:05:12.754875  9746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe46065991d847dfa3d934961da0830a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:12.755174  9706 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:12.757823  9824 tablet_service.cc:1511] Processing CreateTablet for tablet 4cd54dca51434fcf8e5a473955555bfa (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:05:12.759255  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cd54dca51434fcf8e5a473955555bfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.759670 10033 tablet_bootstrap.cc:654] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:12.748910 10049 raft_consensus.cc:493] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.760805 10049 raft_consensus.cc:3060] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.761318  9895 tablet_service.cc:1511] Processing CreateTablet for tablet 4cd54dca51434fcf8e5a473955555bfa (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:05:12.762543  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cd54dca51434fcf8e5a473955555bfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.776605  9972 tablet_service.cc:1511] Processing CreateTablet for tablet 4cd54dca51434fcf8e5a473955555bfa (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:05:12.777993  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cd54dca51434fcf8e5a473955555bfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.784672  9746 tablet_service.cc:1511] Processing CreateTablet for tablet a6bd3b28a03c44fda0c31e4198a86726 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:05:12.789093  9746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6bd3b28a03c44fda0c31e4198a86726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.789305  9824 tablet_service.cc:1511] Processing CreateTablet for tablet a6bd3b28a03c44fda0c31e4198a86726 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:05:12.791185  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6bd3b28a03c44fda0c31e4198a86726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.785271 10028 tablet_bootstrap.cc:654] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:12.793990 10029 tablet_bootstrap.cc:654] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:12.779549 10039 raft_consensus.cc:515] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:12.798061 10039 leader_election.cc:290] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:12.798287  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2684c424702483b92758b1173702381" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:12.800726  9909 raft_consensus.cc:3060] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.809389 10103 raft_consensus.cc:493] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.810022 10103 raft_consensus.cc:515] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.813769  9896 tablet_service.cc:1511] Processing CreateTablet for tablet a6bd3b28a03c44fda0c31e4198a86726 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:05:12.815325  9896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6bd3b28a03c44fda0c31e4198a86726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.800944  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2684c424702483b92758b1173702381" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:12.818106  9983 raft_consensus.cc:3060] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.819067  9820 tablet_service.cc:1511] Processing CreateTablet for tablet 96d8c823d37d4e9a864b46403d267028 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:05:12.820577  9820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96d8c823d37d4e9a864b46403d267028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.822314  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aec2f732f8024a00ae0b4d7bc00bb8e3" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
W20260709 13:05:12.824101  9798 leader_election.cc:343] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:12.825098 10103 leader_election.cc:290] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:12.824797  9894 tablet_service.cc:1511] Processing CreateTablet for tablet d69b492d298d42e18ab6b82be92774ca (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:05:12.826447  9894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d69b492d298d42e18ab6b82be92774ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.826546  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aec2f732f8024a00ae0b4d7bc00bb8e3" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:12.827425  9981 raft_consensus.cc:2468] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:12.828465  9746 tablet_service.cc:1511] Processing CreateTablet for tablet 96d8c823d37d4e9a864b46403d267028 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:05:12.828652  9799 leader_election.cc:304] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:12.829473  9970 tablet_service.cc:1511] Processing CreateTablet for tablet 96d8c823d37d4e9a864b46403d267028 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:05:12.830770  9970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96d8c823d37d4e9a864b46403d267028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.831897  9746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96d8c823d37d4e9a864b46403d267028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.830830 10103 raft_consensus.cc:2804] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:12.832973 10103 raft_consensus.cc:493] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.833307 10103 raft_consensus.cc:3060] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.837790  9969 tablet_service.cc:1511] Processing CreateTablet for tablet d69b492d298d42e18ab6b82be92774ca (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:05:12.844480  9750 tablet_service.cc:1511] Processing CreateTablet for tablet d69b492d298d42e18ab6b82be92774ca (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:05:12.845777  9750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d69b492d298d42e18ab6b82be92774ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.845862  9823 tablet_service.cc:1511] Processing CreateTablet for tablet 912bd6d0efac4042884ce6823f7e8e73 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:12.847162  9823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912bd6d0efac4042884ce6823f7e8e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.847078  9895 tablet_service.cc:1511] Processing CreateTablet for tablet 912bd6d0efac4042884ce6823f7e8e73 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:12.847818  9969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d69b492d298d42e18ab6b82be92774ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.848313  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912bd6d0efac4042884ce6823f7e8e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.851233 10030 tablet_bootstrap.cc:492] T adca2c30b75149c988cf71f9effb9d6c P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:12.851809 10030 ts_tablet_manager.cc:1403] T adca2c30b75149c988cf71f9effb9d6c P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.158s	user 0.017s	sys 0.003s
I20260709 13:05:12.855002  9983 raft_consensus.cc:2468] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:12.856463  9724 leader_election.cc:304] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:12.857555 10049 raft_consensus.cc:515] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:12.858738  9972 tablet_service.cc:1511] Processing CreateTablet for tablet 912bd6d0efac4042884ce6823f7e8e73 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:05:12.857424 10039 raft_consensus.cc:2804] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:12.860185 10039 raft_consensus.cc:697] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:12.861176 10039 consensus_queue.cc:237] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:12.860217  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912bd6d0efac4042884ce6823f7e8e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.838688  9909 raft_consensus.cc:2468] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:12.871268  9747 tablet_service.cc:1511] Processing CreateTablet for tablet f9623952612f4929825a90c043f1c6ba (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:05:12.872439  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adca2c30b75149c988cf71f9effb9d6c" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:12.872653  9898 tablet_service.cc:1511] Processing CreateTablet for tablet f9623952612f4929825a90c043f1c6ba (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:05:12.873750  9825 tablet_service.cc:1511] Processing CreateTablet for tablet f9623952612f4929825a90c043f1c6ba (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:05:12.895610  9825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9623952612f4929825a90c043f1c6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.874756  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adca2c30b75149c988cf71f9effb9d6c" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:12.874538 10049 leader_election.cc:290] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:12.868227 10082 consensus_queue.cc:1048] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:05:12.900271  9747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9623952612f4929825a90c043f1c6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.875588 10030 raft_consensus.cc:359] T adca2c30b75149c988cf71f9effb9d6c P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:12.900847 10030 raft_consensus.cc:385] T adca2c30b75149c988cf71f9effb9d6c P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:12.901230 10030 raft_consensus.cc:740] T adca2c30b75149c988cf71f9effb9d6c P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:12.876294  9973 tablet_service.cc:1511] Processing CreateTablet for tablet 622d51c8b2124715a814796353745c9b (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:12.879432  9744 tablet_service.cc:1511] Processing CreateTablet for tablet 622d51c8b2124715a814796353745c9b (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:12.881772 10028 tablet_bootstrap.cc:492] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:12.903986  9898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9623952612f4929825a90c043f1c6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.904569  9835 raft_consensus.cc:3060] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.902149 10030 consensus_queue.cc:260] T adca2c30b75149c988cf71f9effb9d6c P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:12.906819  9823 tablet_service.cc:1511] Processing CreateTablet for tablet 622d51c8b2124715a814796353745c9b (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:05:12.919343  9823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 622d51c8b2124715a814796353745c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.907554  9973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 622d51c8b2124715a814796353745c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.924674  9897 tablet_service.cc:1511] Processing CreateTablet for tablet a981502782314272a8a9bee8da734427 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:12.920156 10028 ts_tablet_manager.cc:1403] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.220s	user 0.009s	sys 0.016s
I20260709 13:05:12.919739  9983 raft_consensus.cc:3060] T adca2c30b75149c988cf71f9effb9d6c P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:12.936250  9744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 622d51c8b2124715a814796353745c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.938892  9897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a981502782314272a8a9bee8da734427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.944133 10029 tablet_bootstrap.cc:492] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:12.944881 10029 ts_tablet_manager.cc:1403] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.259s	user 0.024s	sys 0.018s
I20260709 13:05:12.949527  9656 catalog_manager.cc:5697] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:12.951493 10028 raft_consensus.cc:359] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:12.952277 10028 raft_consensus.cc:385] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:12.952596 10028 raft_consensus.cc:740] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:12.953423 10028 consensus_queue.cc:260] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:12.955732 10028 ts_tablet_manager.cc:1434] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:05:12.976208 10038 raft_consensus.cc:493] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.976799 10038 raft_consensus.cc:515] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:12.979163  9835 raft_consensus.cc:2468] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:12.980949 10038 leader_election.cc:290] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:12.981228  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41e6a20c04a4d99ba9b1ea99106480a" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:12.982085  9909 raft_consensus.cc:2468] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:12.982683  9874 leader_election.cc:304] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:12.983557  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41e6a20c04a4d99ba9b1ea99106480a" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:12.983763 10049 raft_consensus.cc:2804] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:12.984311 10049 raft_consensus.cc:697] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
W20260709 13:05:12.985251  9725 leader_election.cc:343] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:12.985291 10049 consensus_queue.cc:237] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:12.989756 10033 tablet_bootstrap.cc:492] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:12.990283 10033 ts_tablet_manager.cc:1403] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.254s	user 0.022s	sys 0.031s
I20260709 13:05:12.993335  9726 leader_election.cc:304] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:12.995872 10082 raft_consensus.cc:2804] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:12.996371 10082 raft_consensus.cc:493] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.996783 10082 raft_consensus.cc:3060] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.003526 10036 consensus_queue.cc:1048] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:13.005029 10028 tablet_bootstrap.cc:492] T a8a61e076bba425385e5327d369522de P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:13.004873 10029 raft_consensus.cc:359] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.009755 10029 raft_consensus.cc:385] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.010193 10029 raft_consensus.cc:740] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.016052 10029 consensus_queue.cc:260] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:12.935446 10103 raft_consensus.cc:515] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.014708  9824 tablet_service.cc:1511] Processing CreateTablet for tablet 140e604980a14cfe9b9ee5918dc45bb2 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:13.016837  9895 tablet_service.cc:1511] Processing CreateTablet for tablet 140e604980a14cfe9b9ee5918dc45bb2 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:13.017737  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 140e604980a14cfe9b9ee5918dc45bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:13.017978  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 140e604980a14cfe9b9ee5918dc45bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:13.006654  9750 tablet_service.cc:1511] Processing CreateTablet for tablet a981502782314272a8a9bee8da734427 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:13.020530  9750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a981502782314272a8a9bee8da734427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:13.009169  9983 raft_consensus.cc:2468] T adca2c30b75149c988cf71f9effb9d6c P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:13.026077 10030 ts_tablet_manager.cc:1434] T adca2c30b75149c988cf71f9effb9d6c P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.174s	user 0.010s	sys 0.000s
I20260709 13:05:13.027372 10030 tablet_bootstrap.cc:492] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:13.027720  9969 tablet_service.cc:1511] Processing CreateTablet for tablet a981502782314272a8a9bee8da734427 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:05:13.028659 10082 raft_consensus.cc:515] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.031653  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41e6a20c04a4d99ba9b1ea99106480a" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:13.031827  9972 tablet_service.cc:1511] Processing CreateTablet for tablet 140e604980a14cfe9b9ee5918dc45bb2 (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:05:13.033248  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 140e604980a14cfe9b9ee5918dc45bb2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:13.033299  9725 leader_election.cc:343] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:13.031693 10033 raft_consensus.cc:359] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.034855 10033 raft_consensus.cc:385] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.035207 10033 raft_consensus.cc:740] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.036010 10033 consensus_queue.cc:260] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.040547  9969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a981502782314272a8a9bee8da734427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:13.043285 10033 ts_tablet_manager.cc:1434] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.053s	user 0.009s	sys 0.000s
I20260709 13:05:13.047338 10033 tablet_bootstrap.cc:492] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:13.047959 10029 ts_tablet_manager.cc:1434] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.103s	user 0.004s	sys 0.003s
I20260709 13:05:13.045516 10082 leader_election.cc:290] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:13.044376 10103 leader_election.cc:290] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:13.045658  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aec2f732f8024a00ae0b4d7bc00bb8e3" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:13.050041  9982 raft_consensus.cc:3060] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.049113 10029 tablet_bootstrap.cc:492] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:13.056911  9747 tablet_service.cc:1511] Processing CreateTablet for tablet ecae0626a5c744f0ad9571147ccc239d (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:05:13.058590  9747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecae0626a5c744f0ad9571147ccc239d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:13.059559  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41e6a20c04a4d99ba9b1ea99106480a" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:13.060084  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aec2f732f8024a00ae0b4d7bc00bb8e3" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:13.060307  9909 raft_consensus.cc:3060] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.060712  9760 raft_consensus.cc:3060] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.083822 10028 tablet_bootstrap.cc:654] T a8a61e076bba425385e5327d369522de P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.086575  9982 raft_consensus.cc:2468] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:13.100984  9909 raft_consensus.cc:2468] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:13.102685  9726 leader_election.cc:304] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:13.104584 10030 tablet_bootstrap.cc:654] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.104475 10082 raft_consensus.cc:2804] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:13.106057 10082 raft_consensus.cc:697] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:13.107012 10082 consensus_queue.cc:237] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.107632  9799 leader_election.cc:304] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:13.109258 10103 raft_consensus.cc:2804] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:13.109787 10103 raft_consensus.cc:697] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:13.110504  9760 raft_consensus.cc:2468] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:13.112545  9896 tablet_service.cc:1511] Processing CreateTablet for tablet ecae0626a5c744f0ad9571147ccc239d (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:05:13.110700 10103 consensus_queue.cc:237] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.114054  9896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecae0626a5c744f0ad9571147ccc239d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:13.115514 10091 consensus_queue.cc:1048] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:13.115924  9972 tablet_service.cc:1511] Processing CreateTablet for tablet b6ff13f827e7411d9a5782fb06e93c9b (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:05:13.117085 10033 tablet_bootstrap.cc:654] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.118180  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ff13f827e7411d9a5782fb06e93c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:13.118057  9824 tablet_service.cc:1511] Processing CreateTablet for tablet ecae0626a5c744f0ad9571147ccc239d (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:05:13.119558  9824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecae0626a5c744f0ad9571147ccc239d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:13.133013 10044 raft_consensus.cc:493] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.143352  9658 catalog_manager.cc:5697] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.148841  9744 tablet_service.cc:1511] Processing CreateTablet for tablet b6ff13f827e7411d9a5782fb06e93c9b (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:05:13.149729 10028 tablet_bootstrap.cc:492] T a8a61e076bba425385e5327d369522de P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:13.150235  9744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ff13f827e7411d9a5782fb06e93c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:13.150291 10028 ts_tablet_manager.cc:1403] T a8a61e076bba425385e5327d369522de P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.146s	user 0.017s	sys 0.009s
I20260709 13:05:13.134196 10044 raft_consensus.cc:515] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.153446 10029 tablet_bootstrap.cc:654] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.160531  9825 tablet_service.cc:1511] Processing CreateTablet for tablet b6ff13f827e7411d9a5782fb06e93c9b (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:05:13.162122  9825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ff13f827e7411d9a5782fb06e93c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:13.161872 10028 raft_consensus.cc:359] T a8a61e076bba425385e5327d369522de P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.162788 10028 raft_consensus.cc:385] T a8a61e076bba425385e5327d369522de P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.163793 10028 raft_consensus.cc:740] T a8a61e076bba425385e5327d369522de P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.164677 10028 consensus_queue.cc:260] T a8a61e076bba425385e5327d369522de P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.167775 10044 leader_election.cc:290] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:13.168704  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b011ab28e64b483db8daaf9ed27f45ce" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:13.170830  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b011ab28e64b483db8daaf9ed27f45ce" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:13.173221  9801 leader_election.cc:343] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:13.174006  9799 leader_election.cc:343] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:13.174468  9799 leader_election.cc:304] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0
I20260709 13:05:13.175724 10044 raft_consensus.cc:2749] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:13.176445 10028 ts_tablet_manager.cc:1434] T a8a61e076bba425385e5327d369522de P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.004s
I20260709 13:05:13.177510 10028 tablet_bootstrap.cc:492] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:13.182744  9895 tablet_service.cc:1511] Processing CreateTablet for tablet 98a516bf1d774804b94074f9765006ad (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:13.183293  9972 tablet_service.cc:1511] Processing CreateTablet for tablet 98a516bf1d774804b94074f9765006ad (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:13.184509  9895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98a516bf1d774804b94074f9765006ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:13.188217  9750 tablet_service.cc:1511] Processing CreateTablet for tablet 98a516bf1d774804b94074f9765006ad (DEFAULT_TABLE table=test-workload [id=cf996adae9fb4c13b0b375dc4eebca9e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:05:13.189898  9750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98a516bf1d774804b94074f9765006ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:13.192030 10028 tablet_bootstrap.cc:654] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.207494 10033 tablet_bootstrap.cc:492] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:13.208036 10033 ts_tablet_manager.cc:1403] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.161s	user 0.009s	sys 0.006s
I20260709 13:05:13.211488 10033 raft_consensus.cc:359] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.212229 10033 raft_consensus.cc:385] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.212543 10033 raft_consensus.cc:740] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.213441 10033 consensus_queue.cc:260] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.215775 10033 ts_tablet_manager.cc:1434] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:13.222061 10033 tablet_bootstrap.cc:492] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:13.205971  9656 catalog_manager.cc:5697] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.184533  9972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98a516bf1d774804b94074f9765006ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:13.240653  9657 catalog_manager.cc:5697] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:13.231667 10033 tablet_bootstrap.cc:654] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.259538 10045 raft_consensus.cc:493] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.262360 10030 tablet_bootstrap.cc:492] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:13.267282 10045 raft_consensus.cc:515] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.267951 10030 ts_tablet_manager.cc:1403] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.241s	user 0.013s	sys 0.014s
I20260709 13:05:13.269876 10045 leader_election.cc:290] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:13.271595  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148fd9eeaa2647bca8ad75116e461ac6" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:13.272053  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148fd9eeaa2647bca8ad75116e461ac6" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:13.272291  9982 raft_consensus.cc:2468] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:13.272835  9760 raft_consensus.cc:2468] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:13.273768  9799 leader_election.cc:304] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:13.274995 10030 raft_consensus.cc:359] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.275518 10045 raft_consensus.cc:2804] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:13.275718 10030 raft_consensus.cc:385] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.275998 10045 raft_consensus.cc:493] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.276135 10030 raft_consensus.cc:740] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.276568 10045 raft_consensus.cc:3060] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:13.277077 10030 consensus_queue.cc:260] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.278715 10029 tablet_bootstrap.cc:492] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:13.279225 10029 ts_tablet_manager.cc:1403] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.230s	user 0.016s	sys 0.004s
I20260709 13:05:13.281911 10029 raft_consensus.cc:359] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.282584 10029 raft_consensus.cc:385] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.284497 10045 raft_consensus.cc:515] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.286029 10028 tablet_bootstrap.cc:492] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:13.286571 10028 ts_tablet_manager.cc:1403] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.109s	user 0.026s	sys 0.033s
I20260709 13:05:13.287446  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148fd9eeaa2647bca8ad75116e461ac6" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:13.288086  9760 raft_consensus.cc:3060] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:13.289384 10028 raft_consensus.cc:359] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.290129 10028 raft_consensus.cc:385] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.290385 10028 raft_consensus.cc:740] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.291095 10028 consensus_queue.cc:260] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.282859 10029 raft_consensus.cc:740] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.292609 10029 consensus_queue.cc:260] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.294605 10029 ts_tablet_manager.cc:1434] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.000s
I20260709 13:05:13.296005 10029 tablet_bootstrap.cc:492] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:13.293303 10030 ts_tablet_manager.cc:1434] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.001s
I20260709 13:05:13.300269 10030 tablet_bootstrap.cc:492] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:13.300926 10028 ts_tablet_manager.cc:1434] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:05:13.301924 10028 tablet_bootstrap.cc:492] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:13.307894 10030 tablet_bootstrap.cc:654] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.308617 10028 tablet_bootstrap.cc:654] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.314109 10029 tablet_bootstrap.cc:654] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.318013  9760 raft_consensus.cc:2468] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 2.
I20260709 13:05:13.319783  9798 leader_election.cc:304] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:13.321022 10044 raft_consensus.cc:2804] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:13.321708 10044 raft_consensus.cc:697] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:13.322832 10044 consensus_queue.cc:237] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.325368 10028 tablet_bootstrap.cc:492] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:13.326105 10028 ts_tablet_manager.cc:1403] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.000s
I20260709 13:05:13.329659 10028 raft_consensus.cc:359] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.330345 10028 raft_consensus.cc:385] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.330600 10028 raft_consensus.cc:740] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.331339  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148fd9eeaa2647bca8ad75116e461ac6" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:13.332051  9982 raft_consensus.cc:3060] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:13.333841 10029 tablet_bootstrap.cc:492] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:13.334339 10029 ts_tablet_manager.cc:1403] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.039s	user 0.004s	sys 0.008s
I20260709 13:05:13.337798 10039 consensus_queue.cc:1048] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:13.338317 10033 tablet_bootstrap.cc:492] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:13.339021 10033 ts_tablet_manager.cc:1403] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.117s	user 0.014s	sys 0.021s
I20260709 13:05:13.339792  9982 raft_consensus.cc:2468] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 2.
I20260709 13:05:13.339699 10029 raft_consensus.cc:359] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.340559 10029 raft_consensus.cc:385] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.340869 10029 raft_consensus.cc:740] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.341532 10029 consensus_queue.cc:260] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.342187 10045 leader_election.cc:290] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:13.342355 10033 raft_consensus.cc:359] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.343202 10033 raft_consensus.cc:385] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.343626 10033 raft_consensus.cc:740] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.344359 10028 consensus_queue.cc:260] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.343742 10029 ts_tablet_manager.cc:1434] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:13.344583 10033 consensus_queue.cc:260] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.346364 10029 tablet_bootstrap.cc:492] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:13.347615 10033 ts_tablet_manager.cc:1434] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:13.348645 10033 tablet_bootstrap.cc:492] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:13.352821 10029 tablet_bootstrap.cc:654] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.354982 10030 tablet_bootstrap.cc:492] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:13.356220 10028 ts_tablet_manager.cc:1434] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.000s
I20260709 13:05:13.356726 10033 tablet_bootstrap.cc:654] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.357236 10028 tablet_bootstrap.cc:492] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:13.359686 10036 raft_consensus.cc:493] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.360390 10036 raft_consensus.cc:515] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.364550 10036 leader_election.cc:290] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:13.364977 10028 tablet_bootstrap.cc:654] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.364789  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeefc94068a14a0b9dd656a55fd68ccc" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:13.365536  9835 raft_consensus.cc:2468] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:13.355476 10030 ts_tablet_manager.cc:1403] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.055s	user 0.016s	sys 0.002s
I20260709 13:05:13.366827  9874 leader_election.cc:304] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:13.367904 10036 raft_consensus.cc:2804] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:13.368325 10036 raft_consensus.cc:493] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.368748 10036 raft_consensus.cc:3060] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.370292  9658 catalog_manager.cc:5697] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 2, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.371068 10030 raft_consensus.cc:359] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.372169 10030 raft_consensus.cc:385] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.372541 10030 raft_consensus.cc:740] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.373342 10030 consensus_queue.cc:260] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.375463 10030 ts_tablet_manager.cc:1434] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:05:13.377122 10030 tablet_bootstrap.cc:492] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:13.383764 10030 tablet_bootstrap.cc:654] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.391100 10072 raft_consensus.cc:493] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.391865 10072 raft_consensus.cc:515] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.369220  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeefc94068a14a0b9dd656a55fd68ccc" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:13.396603  9760 raft_consensus.cc:2468] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:13.403644 10033 tablet_bootstrap.cc:492] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:13.404165 10033 ts_tablet_manager.cc:1403] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.056s	user 0.017s	sys 0.000s
I20260709 13:05:13.410327  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a61e076bba425385e5327d369522de" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:13.411325  9760 raft_consensus.cc:2468] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 0.
I20260709 13:05:13.412992 10036 raft_consensus.cc:515] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.414279  9946 leader_election.cc:304] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:13.416280 10033 raft_consensus.cc:359] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.417171 10033 raft_consensus.cc:385] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.417443 10033 raft_consensus.cc:740] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.407573 10072 leader_election.cc:290] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:13.421108 10029 tablet_bootstrap.cc:492] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:13.421630 10029 ts_tablet_manager.cc:1403] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.076s	user 0.021s	sys 0.013s
I20260709 13:05:13.423272 10033 consensus_queue.cc:260] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.429615 10072 raft_consensus.cc:2804] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:13.430907  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeefc94068a14a0b9dd656a55fd68ccc" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:13.433900  9760 raft_consensus.cc:3060] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.435527 10028 tablet_bootstrap.cc:492] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:13.436036 10028 ts_tablet_manager.cc:1403] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.079s	user 0.027s	sys 0.027s
I20260709 13:05:13.430579 10072 raft_consensus.cc:493] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.441318 10033 ts_tablet_manager.cc:1434] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.003s
I20260709 13:05:13.442940 10033 tablet_bootstrap.cc:492] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:13.442847 10028 raft_consensus.cc:359] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.443722 10028 raft_consensus.cc:385] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.443993 10028 raft_consensus.cc:740] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.444510  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeefc94068a14a0b9dd656a55fd68ccc" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:13.445132  9835 raft_consensus.cc:3060] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.444761 10028 consensus_queue.cc:260] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.453953 10033 tablet_bootstrap.cc:654] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.455438 10028 ts_tablet_manager.cc:1434] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 13:05:13.457724 10028 tablet_bootstrap.cc:492] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:13.461198  9835 raft_consensus.cc:2468] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:13.442006 10072 raft_consensus.cc:3060] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.442647 10036 leader_election.cc:290] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:13.449220 10029 raft_consensus.cc:359] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.470077  9760 raft_consensus.cc:2468] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:13.471570  9872 leader_election.cc:304] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:13.471858 10082 consensus_queue.cc:1048] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:13.472484 10116 raft_consensus.cc:2804] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:13.473049 10116 raft_consensus.cc:697] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:13.473901 10028 tablet_bootstrap.cc:654] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.473905 10116 consensus_queue.cc:237] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.477483 10072 raft_consensus.cc:515] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.480327 10029 raft_consensus.cc:385] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.480703 10029 raft_consensus.cc:740] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.482003 10029 consensus_queue.cc:260] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.489514 10029 ts_tablet_manager.cc:1434] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.068s	user 0.009s	sys 0.000s
I20260709 13:05:13.491829 10030 tablet_bootstrap.cc:492] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:13.492413 10030 ts_tablet_manager.cc:1403] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.115s	user 0.013s	sys 0.012s
I20260709 13:05:13.492504 10033 tablet_bootstrap.cc:492] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:13.493317 10033 ts_tablet_manager.cc:1403] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.010s
I20260709 13:05:13.498697 10029 tablet_bootstrap.cc:492] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:13.503718 10033 raft_consensus.cc:359] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.504556 10033 raft_consensus.cc:385] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.504832 10033 raft_consensus.cc:740] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.506204 10033 consensus_queue.cc:260] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.508981 10030 raft_consensus.cc:359] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.509863 10030 raft_consensus.cc:385] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.510149 10028 tablet_bootstrap.cc:492] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:13.510681 10028 ts_tablet_manager.cc:1403] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.054s	user 0.021s	sys 0.009s
I20260709 13:05:13.510787 10072 leader_election.cc:290] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:13.512669 10029 tablet_bootstrap.cc:654] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.512583  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a61e076bba425385e5327d369522de" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:13.513243  9760 raft_consensus.cc:3060] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.510201 10030 raft_consensus.cc:740] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.514273 10028 raft_consensus.cc:359] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.515584 10028 raft_consensus.cc:385] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.515904 10033 ts_tablet_manager.cc:1434] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.004s
I20260709 13:05:13.516422 10030 consensus_queue.cc:260] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.516957 10033 tablet_bootstrap.cc:492] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:13.518499 10030 ts_tablet_manager.cc:1434] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:05:13.519435 10030 tablet_bootstrap.cc:492] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:13.515870 10028 raft_consensus.cc:740] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.521983 10028 consensus_queue.cc:260] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.524909 10028 ts_tablet_manager.cc:1434] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:05:13.526190 10028 tablet_bootstrap.cc:492] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:13.533004 10036 raft_consensus.cc:493] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.533586 10036 raft_consensus.cc:515] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.537273 10036 leader_election.cc:290] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:13.538743  9760 raft_consensus.cc:2468] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:13.539536 10033 tablet_bootstrap.cc:654] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.540951  9946 leader_election.cc:304] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:13.541769  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b011ab28e64b483db8daaf9ed27f45ce" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:13.541787  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b011ab28e64b483db8daaf9ed27f45ce" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:13.542198  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a61e076bba425385e5327d369522de" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:13.542711  9835 raft_consensus.cc:2468] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:13.542821  9909 raft_consensus.cc:3060] T a8a61e076bba425385e5327d369522de P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.543522  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a61e076bba425385e5327d369522de" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:13.545519 10030 tablet_bootstrap.cc:654] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.545331  9874 leader_election.cc:304] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:13.542800  9983 raft_consensus.cc:2468] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:13.547451 10072 raft_consensus.cc:2804] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:13.548029 10072 raft_consensus.cc:697] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:13.548589 10036 raft_consensus.cc:2804] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:13.549063  9835 raft_consensus.cc:3060] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.548992 10036 raft_consensus.cc:493] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.550249 10036 raft_consensus.cc:3060] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.550796  9656 catalog_manager.cc:5697] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:13.548903 10072 consensus_queue.cc:237] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.561414 10028 tablet_bootstrap.cc:654] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.563898 10082 consensus_queue.cc:1048] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:13.572235  9909 raft_consensus.cc:2468] T a8a61e076bba425385e5327d369522de P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:13.575115 10116 raft_consensus.cc:493] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.575855 10116 raft_consensus.cc:515] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.578100 10036 raft_consensus.cc:515] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.582391  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b011ab28e64b483db8daaf9ed27f45ce" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:13.583159  9835 raft_consensus.cc:3060] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.584558  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af25fe6172fa4032a6570d9a3c7757c6" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:13.585332 10116 leader_election.cc:290] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:13.585664  9982 raft_consensus.cc:2468] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:13.586333  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af25fe6172fa4032a6570d9a3c7757c6" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:13.592408 10036 leader_election.cc:290] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:13.593036  9873 leader_election.cc:304] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 
I20260709 13:05:13.593283 10036 consensus_queue.cc:1048] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:13.594743 10091 raft_consensus.cc:2804] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:13.595469 10091 raft_consensus.cc:493] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.595908 10091 raft_consensus.cc:3060] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:13.600173  9872 leader_election.cc:343] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:13.608565 10029 tablet_bootstrap.cc:492] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:13.607160  9658 catalog_manager.cc:5697] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 0 to 1, leader changed from <none> to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 1 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.611415 10033 tablet_bootstrap.cc:492] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:13.611994 10033 ts_tablet_manager.cc:1403] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.095s	user 0.005s	sys 0.011s
I20260709 13:05:13.615139 10033 raft_consensus.cc:359] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.615916 10033 raft_consensus.cc:385] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.616225 10033 raft_consensus.cc:740] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.600693  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b011ab28e64b483db8daaf9ed27f45ce" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:13.620846  9982 raft_consensus.cc:3060] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.621780 10049 consensus_queue.cc:1048] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:13.625388 10039 consensus_queue.cc:1048] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:13.625483 10033 consensus_queue.cc:260] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.627866  9835 raft_consensus.cc:2468] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:13.630604 10033 ts_tablet_manager.cc:1434] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.005s
I20260709 13:05:13.632931 10033 tablet_bootstrap.cc:492] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:13.634495  9874 leader_election.cc:304] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:13.640931 10038 raft_consensus.cc:493] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.641552 10038 raft_consensus.cc:515] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.642663 10036 raft_consensus.cc:2804] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:13.643409 10044 consensus_queue.cc:1048] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:13.598055 10030 tablet_bootstrap.cc:492] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:13.645076 10030 ts_tablet_manager.cc:1403] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.126s	user 0.012s	sys 0.005s
I20260709 13:05:13.641953 10028 tablet_bootstrap.cc:492] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:13.645891  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2d12d5566b4c82a5572035a797a82d" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:13.646216 10028 ts_tablet_manager.cc:1403] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.120s	user 0.018s	sys 0.025s
I20260709 13:05:13.645745 10036 raft_consensus.cc:697] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:13.646907 10045 raft_consensus.cc:493] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.649538 10045 raft_consensus.cc:515] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.649999 10038 leader_election.cc:290] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:13.616478 10029 ts_tablet_manager.cc:1403] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.118s	user 0.015s	sys 0.014s
I20260709 13:05:13.651504  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2d12d5566b4c82a5572035a797a82d" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:13.652079 10045 leader_election.cc:290] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:13.654129  9833 raft_consensus.cc:2468] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:13.654129  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2d12d5566b4c82a5572035a797a82d" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:13.655066  9760 raft_consensus.cc:2468] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:13.652321 10036 consensus_queue.cc:237] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.657090 10033 tablet_bootstrap.cc:654] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.656752  9798 leader_election.cc:304] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:13.656795  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2d12d5566b4c82a5572035a797a82d" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:13.659219 10045 raft_consensus.cc:2804] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:13.662526 10030 raft_consensus.cc:359] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.663431 10030 raft_consensus.cc:385] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.659641 10045 raft_consensus.cc:493] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.638814  9982 raft_consensus.cc:2468] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:13.659214  9725 leader_election.cc:304] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:13.667155 10045 raft_consensus.cc:3060] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.672421 10029 raft_consensus.cc:359] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.677284 10029 raft_consensus.cc:385] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.677647 10029 raft_consensus.cc:740] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.678637 10029 consensus_queue.cc:260] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.677758 10038 raft_consensus.cc:2804] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:13.680956 10038 raft_consensus.cc:493] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.681339 10038 raft_consensus.cc:3060] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.670022 10091 raft_consensus.cc:515] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.690987  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af25fe6172fa4032a6570d9a3c7757c6" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:13.692235  9981 raft_consensus.cc:3060] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.696584 10039 raft_consensus.cc:493] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.696782 10045 raft_consensus.cc:515] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.699154 10039 raft_consensus.cc:515] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.702701 10039 leader_election.cc:290] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:13.703096  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af25fe6172fa4032a6570d9a3c7757c6" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:13.704097  9760 raft_consensus.cc:3060] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.674520 10028 raft_consensus.cc:359] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.663793 10030 raft_consensus.cc:740] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.707355 10028 raft_consensus.cc:385] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.709208 10028 raft_consensus.cc:740] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.708875 10029 ts_tablet_manager.cc:1434] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.000s
I20260709 13:05:13.711575 10044 consensus_queue.cc:1048] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:13.711958  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2d12d5566b4c82a5572035a797a82d" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:13.714582 10029 tablet_bootstrap.cc:492] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:13.716158 10044 raft_consensus.cc:493] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.716632 10028 consensus_queue.cc:260] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.716759 10044 raft_consensus.cc:515] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.718412  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af25fe6172fa4032a6570d9a3c7757c6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:13.718881 10028 ts_tablet_manager.cc:1434] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.000s
I20260709 13:05:13.719588  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af25fe6172fa4032a6570d9a3c7757c6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:13.719900 10028 tablet_bootstrap.cc:492] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:13.720484  9908 raft_consensus.cc:2393] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 1: Already voted for candidate 6e10aab26c644389b8161974dcabc5d0 in this term.
I20260709 13:05:13.722433  9726 leader_election.cc:304] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0
I20260709 13:05:13.722862  9758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2d12d5566b4c82a5572035a797a82d" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:13.724730  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa9f688643024d419097e12e80ced0d2" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:13.723132 10045 leader_election.cc:290] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:13.708649 10091 leader_election.cc:290] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:13.725509  9908 raft_consensus.cc:2468] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:13.727608 10044 leader_election.cc:290] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:13.726269 10030 consensus_queue.cc:260] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.729167  9801 leader_election.cc:304] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:13.729202 10038 raft_consensus.cc:515] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.731189  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa9f688643024d419097e12e80ced0d2" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:13.735065 10044 raft_consensus.cc:2804] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:13.735328  9799 leader_election.cc:343] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:13.735533 10044 raft_consensus.cc:493] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.735991 10044 raft_consensus.cc:3060] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.736490  9979 raft_consensus.cc:2468] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:13.744689  9758 raft_consensus.cc:2393] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a1cbca39e34743d8a40816c3bc5f2030 in current term 1: Already voted for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in this term.
I20260709 13:05:13.743763  9760 raft_consensus.cc:2468] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:13.749377 10039 raft_consensus.cc:2749] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:13.751081 10038 leader_election.cc:290] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:13.751847  9798 leader_election.cc:304] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:13.753194 10103 raft_consensus.cc:2749] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:13.753604 10044 raft_consensus.cc:515] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.754810  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2d12d5566b4c82a5572035a797a82d" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:13.755764  9835 raft_consensus.cc:2393] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 1: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
I20260709 13:05:13.761678 10028 tablet_bootstrap.cc:654] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.760653  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2d12d5566b4c82a5572035a797a82d" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:13.765966 10029 tablet_bootstrap.cc:654] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.775743  9872 leader_election.cc:304] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:13.764802 10030 ts_tablet_manager.cc:1434] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.119s	user 0.009s	sys 0.000s
I20260709 13:05:13.765511  9979 raft_consensus.cc:3060] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:13.797048 10030 tablet_bootstrap.cc:492] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:13.737287  9981 raft_consensus.cc:2468] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:13.805193 10147 raft_consensus.cc:2804] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:13.805953 10147 raft_consensus.cc:697] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:13.806267 10044 leader_election.cc:290] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:13.806905 10147 consensus_queue.cc:237] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.830725  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa9f688643024d419097e12e80ced0d2" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:13.831606  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa9f688643024d419097e12e80ced0d2" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:13.832453  9909 raft_consensus.cc:3060] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:13.833364  9706 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:13.834215 10033 tablet_bootstrap.cc:492] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:13.834828 10033 ts_tablet_manager.cc:1403] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.202s	user 0.045s	sys 0.065s
W20260709 13:05:13.835278  9799 leader_election.cc:343] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:13.844833 10033 raft_consensus.cc:359] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.846020 10033 raft_consensus.cc:385] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.846452 10033 raft_consensus.cc:740] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.847523 10033 consensus_queue.cc:260] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.850189 10033 ts_tablet_manager.cc:1434] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:05:13.851420 10028 tablet_bootstrap.cc:492] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:13.851965 10028 ts_tablet_manager.cc:1403] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.132s	user 0.017s	sys 0.017s
I20260709 13:05:13.852326 10033 tablet_bootstrap.cc:492] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:13.855010 10028 raft_consensus.cc:359] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.855763 10028 raft_consensus.cc:385] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.855809  9760 raft_consensus.cc:1275] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Refusing update from remote peer a1cbca39e34743d8a40816c3bc5f2030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:13.856119 10028 raft_consensus.cc:740] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.857023 10028 consensus_queue.cc:260] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.857280  9909 raft_consensus.cc:2468] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:13.859488 10028 ts_tablet_manager.cc:1434] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:05:13.860711 10028 tablet_bootstrap.cc:492] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:13.857784 10044 consensus_queue.cc:1048] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:05:13.864439  9801 leader_election.cc:304] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:13.865492 10159 raft_consensus.cc:2804] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:13.866019 10159 raft_consensus.cc:697] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:13.866948 10159 consensus_queue.cc:237] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.868188 10033 tablet_bootstrap.cc:654] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.872540  9979 raft_consensus.cc:2468] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:13.880597  9724 leader_election.cc:304] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:13.881345 10029 tablet_bootstrap.cc:492] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:13.881942 10029 ts_tablet_manager.cc:1403] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.168s	user 0.016s	sys 0.003s
I20260709 13:05:13.884303 10038 raft_consensus.cc:2804] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:13.884907 10038 raft_consensus.cc:697] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:13.885164 10029 raft_consensus.cc:359] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.886018 10029 raft_consensus.cc:385] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.886409 10029 raft_consensus.cc:740] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.885892 10038 consensus_queue.cc:237] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.897076 10030 tablet_bootstrap.cc:654] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.896512 10029 consensus_queue.cc:260] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.902707 10028 tablet_bootstrap.cc:654] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.906555 10029 ts_tablet_manager.cc:1434] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 13:05:13.907371  9981 raft_consensus.cc:1275] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Refusing update from remote peer a1cbca39e34743d8a40816c3bc5f2030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:13.908062 10029 tablet_bootstrap.cc:492] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:13.911258 10044 consensus_queue.cc:1048] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:05:13.925642 10029 tablet_bootstrap.cc:654] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.927358  9657 catalog_manager.cc:5697] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.943377 10147 raft_consensus.cc:493] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.944259 10147 raft_consensus.cc:515] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:13.951043  9656 catalog_manager.cc:5697] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.959637  9656 catalog_manager.cc:5697] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.961373 10044 raft_consensus.cc:493] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.971195 10029 tablet_bootstrap.cc:492] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:13.971853 10029 ts_tablet_manager.cc:1403] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.064s	user 0.018s	sys 0.006s
I20260709 13:05:13.974889 10030 tablet_bootstrap.cc:492] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:13.974738 10029 raft_consensus.cc:359] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.975451 10030 ts_tablet_manager.cc:1403] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.180s	user 0.018s	sys 0.006s
I20260709 13:05:13.975636 10029 raft_consensus.cc:385] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.976042 10029 raft_consensus.cc:740] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:13.976936 10029 consensus_queue.cc:260] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:13.979239 10029 ts_tablet_manager.cc:1434] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:13.979591  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "755ff96760b04fd49a520c059f90b1fe" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:13.980085 10029 tablet_bootstrap.cc:492] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:13.981429  9873 leader_election.cc:343] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:13.968483 10044 raft_consensus.cc:515] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:13.984726 10072 consensus_queue.cc:1048] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:13.986904 10029 tablet_bootstrap.cc:654] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:13.987967 10147 leader_election.cc:290] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:13.989080 10030 raft_consensus.cc:359] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.989419  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23871dd822614a5da07c2289f80141c1" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:13.990350  9760 raft_consensus.cc:2468] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:13.986609 10044 leader_election.cc:290] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:13.991910 10030 raft_consensus.cc:385] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:13.993960  9798 leader_election.cc:304] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:13.995059 10159 raft_consensus.cc:2804] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:13.995431 10159 raft_consensus.cc:493] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.995872 10028 tablet_bootstrap.cc:492] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:13.996395 10028 ts_tablet_manager.cc:1403] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.136s	user 0.017s	sys 0.018s
I20260709 13:05:13.996582  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23871dd822614a5da07c2289f80141c1" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:13.993518 10030 raft_consensus.cc:740] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
W20260709 13:05:13.997962  9799 leader_election.cc:343] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:13.995692 10033 tablet_bootstrap.cc:492] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:13.999464 10033 ts_tablet_manager.cc:1403] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.147s	user 0.014s	sys 0.015s
I20260709 13:05:13.999373 10028 raft_consensus.cc:359] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.000172 10028 raft_consensus.cc:385] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.000490 10028 raft_consensus.cc:740] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.001143 10028 consensus_queue.cc:260] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.001654  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "755ff96760b04fd49a520c059f90b1fe" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:13.995777 10159 raft_consensus.cc:3060] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.004536 10033 raft_consensus.cc:359] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.005267 10033 raft_consensus.cc:385] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.005537 10033 raft_consensus.cc:740] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.006173 10028 ts_tablet_manager.cc:1434] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:05:14.006287 10033 consensus_queue.cc:260] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:13.998219 10030 consensus_queue.cc:260] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.007321 10028 tablet_bootstrap.cc:492] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:14.008989 10030 ts_tablet_manager.cc:1434] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
W20260709 13:05:14.008694  9872 leader_election.cc:343] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.019691  9872 leader_election.cc:304] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:14.020848 10091 raft_consensus.cc:2749] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:14.021749 10091 consensus_queue.cc:1048] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:14.026496 10033 ts_tablet_manager.cc:1434] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:05:14.010576 10030 tablet_bootstrap.cc:492] T 23871dd822614a5da07c2289f80141c1 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:14.018508  9657 catalog_manager.cc:5697] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:14.033504 10033 tablet_bootstrap.cc:492] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:14.036823 10159 raft_consensus.cc:515] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:14.040565 10159 leader_election.cc:290] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:14.041471  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23871dd822614a5da07c2289f80141c1" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:14.042755  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23871dd822614a5da07c2289f80141c1" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:14.043433 10028 tablet_bootstrap.cc:654] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.044596 10091 consensus_queue.cc:1048] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:14.043452  9759 raft_consensus.cc:3060] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:14.047667  9799 leader_election.cc:343] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.064328 10159 raft_consensus.cc:493] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.064877 10159 raft_consensus.cc:515] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.070843 10150 raft_consensus.cc:493] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.071606 10150 raft_consensus.cc:515] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.075287  9758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3505eea4224942d7a938f9eaf7427723" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:14.076011  9758 raft_consensus.cc:2468] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:14.076454 10150 leader_election.cc:290] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:14.076989 10033 tablet_bootstrap.cc:654] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.081063 10029 tablet_bootstrap.cc:492] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:14.081624 10029 ts_tablet_manager.cc:1403] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.102s	user 0.028s	sys 0.005s
I20260709 13:05:14.082942  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3505eea4224942d7a938f9eaf7427723" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:14.083652  9908 raft_consensus.cc:2468] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:14.084975  9801 leader_election.cc:304] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:14.085481  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716c25a1e46547c999c1c7c860e23f3b" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:14.085903 10044 raft_consensus.cc:2804] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:14.086313 10044 raft_consensus.cc:493] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:14.086800  9873 leader_election.cc:343] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.086669 10044 raft_consensus.cc:3060] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.095368  9758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716c25a1e46547c999c1c7c860e23f3b" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:14.096876 10159 leader_election.cc:290] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
W20260709 13:05:14.098248  9872 leader_election.cc:343] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.098994  9872 leader_election.cc:304] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:14.095083 10029 raft_consensus.cc:359] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.102775 10029 raft_consensus.cc:385] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.103194 10029 raft_consensus.cc:740] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.104291 10029 consensus_queue.cc:260] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.101825 10147 consensus_queue.cc:1048] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:05:14.113063 10029 ts_tablet_manager.cc:1434] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.031s	user 0.010s	sys 0.000s
I20260709 13:05:14.114429 10029 tablet_bootstrap.cc:492] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:14.106932 10091 raft_consensus.cc:2749] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:14.121585 10029 tablet_bootstrap.cc:654] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.124022 10028 tablet_bootstrap.cc:492] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:14.124356 10030 tablet_bootstrap.cc:654] T 23871dd822614a5da07c2289f80141c1 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.124581 10028 ts_tablet_manager.cc:1403] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.118s	user 0.010s	sys 0.015s
I20260709 13:05:14.111346  9759 raft_consensus.cc:2468] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:14.132673 10028 raft_consensus.cc:359] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.133741 10028 raft_consensus.cc:385] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.133548 10072 raft_consensus.cc:493] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.134248 10028 raft_consensus.cc:740] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.134588 10072 raft_consensus.cc:515] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.135031 10028 consensus_queue.cc:260] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.137229 10028 ts_tablet_manager.cc:1434] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:05:14.138178 10028 tablet_bootstrap.cc:492] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:14.138022 10044 raft_consensus.cc:515] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.138285  9798 leader_election.cc:304] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:14.143702  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa9f688643024d419097e12e80ced0d2" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:14.141251 10159 raft_consensus.cc:2804] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:14.145164 10159 raft_consensus.cc:697] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:14.145980 10159 consensus_queue.cc:237] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:14.149401 10033 tablet_bootstrap.cc:492] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:14.150075 10033 ts_tablet_manager.cc:1403] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.117s	user 0.023s	sys 0.018s
I20260709 13:05:14.150823 10044 leader_election.cc:290] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:14.153787 10033 raft_consensus.cc:359] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.154824 10033 raft_consensus.cc:385] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.155278 10033 raft_consensus.cc:740] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.156178 10033 consensus_queue.cc:260] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.157145 10028 tablet_bootstrap.cc:654] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.158023  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3505eea4224942d7a938f9eaf7427723" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:14.158898 10033 ts_tablet_manager.cc:1434] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:14.160574 10116 consensus_queue.cc:1048] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:14.162441 10033 tablet_bootstrap.cc:492] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:14.164295  9909 raft_consensus.cc:3060] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.173202 10030 tablet_bootstrap.cc:492] T 23871dd822614a5da07c2289f80141c1 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:14.181856 10030 ts_tablet_manager.cc:1403] T 23871dd822614a5da07c2289f80141c1 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.172s	user 0.015s	sys 0.014s
I20260709 13:05:14.178997 10033 tablet_bootstrap.cc:654] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.180675  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3505eea4224942d7a938f9eaf7427723" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:14.181039 10072 leader_election.cc:290] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:14.190347  9759 raft_consensus.cc:3060] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.202665  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa9f688643024d419097e12e80ced0d2" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:14.203684  9908 raft_consensus.cc:2393] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23cbd6234c5e43d89f22c235ed6bb96a in current term 1: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
I20260709 13:05:14.205433  9949 leader_election.cc:304] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:14.212913  9909 raft_consensus.cc:2468] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:14.214570  9801 leader_election.cc:304] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:14.217969 10029 tablet_bootstrap.cc:492] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:14.215512 10159 raft_consensus.cc:2804] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:14.219539 10159 raft_consensus.cc:697] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:14.220496 10159 consensus_queue.cc:237] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.222003 10028 tablet_bootstrap.cc:492] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:14.222779 10028 ts_tablet_manager.cc:1403] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.085s	user 0.024s	sys 0.003s
I20260709 13:05:14.222193 10178 consensus_queue.cc:1048] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:14.218684 10029 ts_tablet_manager.cc:1403] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.105s	user 0.023s	sys 0.011s
I20260709 13:05:14.211719 10072 raft_consensus.cc:3060] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.240159 10072 raft_consensus.cc:2749] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:14.228404 10028 raft_consensus.cc:359] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.240872 10028 raft_consensus.cc:385] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.241225 10028 raft_consensus.cc:740] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.241982 10028 consensus_queue.cc:260] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.241495 10030 raft_consensus.cc:359] T 23871dd822614a5da07c2289f80141c1 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:14.244289 10030 raft_consensus.cc:385] T 23871dd822614a5da07c2289f80141c1 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.244786 10030 raft_consensus.cc:740] T 23871dd822614a5da07c2289f80141c1 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.247378 10028 ts_tablet_manager.cc:1434] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.003s
I20260709 13:05:14.250089 10033 tablet_bootstrap.cc:492] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:14.250590 10033 ts_tablet_manager.cc:1403] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.088s	user 0.021s	sys 0.005s
I20260709 13:05:14.249563  9656 catalog_manager.cc:5697] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:14.256006  9759 raft_consensus.cc:2468] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:14.254711 10033 raft_consensus.cc:359] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:14.259740 10033 raft_consensus.cc:385] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.260047 10033 raft_consensus.cc:740] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.260782 10033 consensus_queue.cc:260] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:14.265615 10029 raft_consensus.cc:359] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:14.265914 10116 consensus_queue.cc:1048] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:14.267971 10028 tablet_bootstrap.cc:492] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:14.248404 10030 consensus_queue.cc:260] T 23871dd822614a5da07c2289f80141c1 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:14.266601 10029 raft_consensus.cc:385] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.269392 10029 raft_consensus.cc:740] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.267874 10033 ts_tablet_manager.cc:1434] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:05:14.276607 10033 tablet_bootstrap.cc:492] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:14.282251 10028 tablet_bootstrap.cc:654] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.283119 10033 tablet_bootstrap.cc:654] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.284529 10030 ts_tablet_manager.cc:1434] T 23871dd822614a5da07c2289f80141c1 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.102s	user 0.005s	sys 0.004s
I20260709 13:05:14.285537 10030 tablet_bootstrap.cc:492] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:14.290552 10029 consensus_queue.cc:260] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:14.306301 10029 ts_tablet_manager.cc:1434] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.081s	user 0.007s	sys 0.001s
I20260709 13:05:14.304322 10044 raft_consensus.cc:493] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.307957 10044 raft_consensus.cc:515] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.308699 10029 tablet_bootstrap.cc:492] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:14.310297 10044 leader_election.cc:290] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:14.311825  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dddadb853be408b8e6a1103a45ce0a1" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:14.312305  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dddadb853be408b8e6a1103a45ce0a1" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:14.312587  9909 raft_consensus.cc:2468] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:14.314272  9801 leader_election.cc:304] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
W20260709 13:05:14.315356  9798 leader_election.cc:343] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.315866 10082 consensus_queue.cc:1048] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:14.316726 10039 consensus_queue.cc:1048] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:14.316159 10159 raft_consensus.cc:2804] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:14.319486 10159 raft_consensus.cc:493] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.319943 10159 raft_consensus.cc:3060] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.322072 10044 consensus_queue.cc:1048] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:14.329097 10028 tablet_bootstrap.cc:492] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:14.329691 10028 ts_tablet_manager.cc:1403] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.062s	user 0.013s	sys 0.000s
I20260709 13:05:14.330899 10029 tablet_bootstrap.cc:654] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.333020 10028 raft_consensus.cc:359] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.333814 10028 raft_consensus.cc:385] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.334158 10028 raft_consensus.cc:740] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.337873  9655 catalog_manager.cc:5697] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:14.339821 10028 consensus_queue.cc:260] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.347448 10030 tablet_bootstrap.cc:654] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.366276 10159 raft_consensus.cc:515] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.422075 10157 consensus_queue.cc:1048] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:14.429747 10028 ts_tablet_manager.cc:1434] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.100s	user 0.010s	sys 0.000s
I20260709 13:05:14.431334 10028 tablet_bootstrap.cc:492] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:14.433293 10033 tablet_bootstrap.cc:492] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:14.433971 10033 ts_tablet_manager.cc:1403] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.158s	user 0.017s	sys 0.000s
I20260709 13:05:14.438938 10159 leader_election.cc:290] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:14.440153 10159 raft_consensus.cc:493] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.440344  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dddadb853be408b8e6a1103a45ce0a1" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:14.441174  9908 raft_consensus.cc:3060] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.440729 10159 raft_consensus.cc:515] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.451525  9908 raft_consensus.cc:2468] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:14.452658 10029 tablet_bootstrap.cc:492] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:14.476091 10029 ts_tablet_manager.cc:1403] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.168s	user 0.036s	sys 0.024s
I20260709 13:05:14.438789 10033 raft_consensus.cc:359] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.477497 10033 raft_consensus.cc:385] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.477778 10033 raft_consensus.cc:740] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.478456 10033 consensus_queue.cc:260] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.479271 10029 raft_consensus.cc:359] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.479564  9801 leader_election.cc:304] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:14.480010 10029 raft_consensus.cc:385] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.480319 10029 raft_consensus.cc:740] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.465907 10030 tablet_bootstrap.cc:492] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:14.465935  9757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dddadb853be408b8e6a1103a45ce0a1" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:14.466683 10159 leader_election.cc:290] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:14.482348 10197 raft_consensus.cc:493] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.467501 10091 consensus_queue.cc:1048] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:14.480718 10044 raft_consensus.cc:2804] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:14.493229 10030 ts_tablet_manager.cc:1403] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.208s	user 0.015s	sys 0.016s
I20260709 13:05:14.493362 10044 raft_consensus.cc:697] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:14.470197 10028 tablet_bootstrap.cc:654] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.470057  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:14.490196 10116 raft_consensus.cc:493] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.495779 10116 raft_consensus.cc:515] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.496273 10030 raft_consensus.cc:359] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.497025 10030 raft_consensus.cc:385] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.497268 10030 raft_consensus.cc:740] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.494340 10044 consensus_queue.cc:237] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.498062 10030 consensus_queue.cc:260] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.498746  9907 raft_consensus.cc:2468] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:14.498943  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:14.481033 10029 consensus_queue.cc:260] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.499864  9833 raft_consensus.cc:2468] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:14.467442  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:14.501478  9874 leader_election.cc:304] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:14.480029 10033 ts_tablet_manager.cc:1434] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.045s	user 0.005s	sys 0.000s
I20260709 13:05:14.502694 10091 raft_consensus.cc:2804] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:14.502875  9799 leader_election.cc:343] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.503183 10091 raft_consensus.cc:493] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.503546 10091 raft_consensus.cc:3060] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.504930 10116 leader_election.cc:290] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:14.505540 10030 ts_tablet_manager.cc:1434] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:05:14.503588 10033 tablet_bootstrap.cc:492] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:14.507102  9801 leader_election.cc:304] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
W20260709 13:05:14.507721  9798 leader_election.cc:343] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.508085 10199 raft_consensus.cc:2804] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:14.508522 10199 raft_consensus.cc:493] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.508903 10199 raft_consensus.cc:3060] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.509371 10082 raft_consensus.cc:493] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.509897 10082 raft_consensus.cc:515] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.513206 10030 tablet_bootstrap.cc:492] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:14.512997 10091 raft_consensus.cc:515] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.513706  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "755ff96760b04fd49a520c059f90b1fe" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:14.514189 10082 leader_election.cc:290] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:14.514459  9981 raft_consensus.cc:2468] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:14.515892  9724 leader_election.cc:304] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:14.516376  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "755ff96760b04fd49a520c059f90b1fe" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:14.507292 10197 raft_consensus.cc:515] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:14.520792  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:14.501567 10029 ts_tablet_manager.cc:1434] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.005s
I20260709 13:05:14.523049 10029 tablet_bootstrap.cc:492] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:14.508121  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:14.526692  9873 leader_election.cc:343] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.517647 10082 raft_consensus.cc:2804] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:14.527560 10082 raft_consensus.cc:493] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.528553 10082 raft_consensus.cc:3060] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.530184 10199 raft_consensus.cc:515] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.531602  9833 raft_consensus.cc:2393] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e10aab26c644389b8161974dcabc5d0 in current term 1: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
I20260709 13:05:14.536594  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
W20260709 13:05:14.538228  9799 leader_election.cc:343] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.540196  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:14.540482  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:14.541225  9908 raft_consensus.cc:2393] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a1cbca39e34743d8a40816c3bc5f2030 in current term 1: Already voted for candidate 6e10aab26c644389b8161974dcabc5d0 in this term.
W20260709 13:05:14.542002  9873 leader_election.cc:343] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.542528  9873 leader_election.cc:304] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:14.543025  9801 leader_election.cc:304] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0
I20260709 13:05:14.517052  9907 raft_consensus.cc:2468] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:14.524783 10033 tablet_bootstrap.cc:654] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.525111 10197 leader_election.cc:290] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:14.545974 10197 raft_consensus.cc:2749] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:14.546346 10116 raft_consensus.cc:2749] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:14.540267 10091 leader_election.cc:290] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:14.549846  9980 raft_consensus.cc:3060] T 23871dd822614a5da07c2289f80141c1 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.550678  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:14.555403  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:14.556248 10028 tablet_bootstrap.cc:492] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:14.556762 10028 ts_tablet_manager.cc:1403] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.126s	user 0.012s	sys 0.000s
W20260709 13:05:14.557258  9799 leader_election.cc:343] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.560916 10029 tablet_bootstrap.cc:654] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:14.562392  9798 leader_election.cc:343] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.562938  9798 leader_election.cc:304] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:14.563162 10157 consensus_queue.cc:1048] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:14.564252 10198 raft_consensus.cc:2749] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:14.565099 10082 raft_consensus.cc:515] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.565905 10030 tablet_bootstrap.cc:654] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.567999  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "755ff96760b04fd49a520c059f90b1fe" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:14.570776 10082 leader_election.cc:290] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:14.570986  9907 raft_consensus.cc:3060] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.571815 10199 leader_election.cc:290] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:14.579274 10091 raft_consensus.cc:493] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.579857 10091 raft_consensus.cc:515] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.582188 10091 leader_election.cc:290] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:14.583503  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c62d82ebe5420bb298d787cda221aa" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:14.584440  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c62d82ebe5420bb298d787cda221aa" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
W20260709 13:05:14.585731  9874 leader_election.cc:343] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:14.587028  9873 leader_election.cc:343] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.588762  9873 leader_election.cc:304] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:14.589416 10091 raft_consensus.cc:2749] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:14.591702 10028 raft_consensus.cc:359] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.592305 10028 raft_consensus.cc:385] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.592588 10028 raft_consensus.cc:740] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.594183 10028 consensus_queue.cc:260] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.597502 10033 tablet_bootstrap.cc:492] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:14.603328  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "755ff96760b04fd49a520c059f90b1fe" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:14.608057  9979 raft_consensus.cc:3060] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.602053  9657 catalog_manager.cc:5697] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:14.611474 10199 consensus_queue.cc:1048] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:14.616982  9907 raft_consensus.cc:2468] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:14.622345 10033 ts_tablet_manager.cc:1403] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.119s	user 0.024s	sys 0.009s
I20260709 13:05:14.625299 10028 ts_tablet_manager.cc:1434] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 13:05:14.626405 10028 tablet_bootstrap.cc:492] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:14.628545  9726 leader_election.cc:304] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:14.629422 10082 raft_consensus.cc:2804] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:14.629905 10082 raft_consensus.cc:697] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:14.632752 10033 raft_consensus.cc:359] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.630738 10082 consensus_queue.cc:237] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.635701  9979 raft_consensus.cc:2468] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:14.643772 10029 tablet_bootstrap.cc:492] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:14.646390 10029 ts_tablet_manager.cc:1403] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.124s	user 0.022s	sys 0.008s
I20260709 13:05:14.646960 10033 raft_consensus.cc:385] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.647517 10033 raft_consensus.cc:740] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.648559 10033 consensus_queue.cc:260] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.653061 10029 raft_consensus.cc:359] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.654078 10033 ts_tablet_manager.cc:1434] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.004s
I20260709 13:05:14.653854 10029 raft_consensus.cc:385] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.654796 10029 raft_consensus.cc:740] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.658207 10029 consensus_queue.cc:260] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.659443 10033 tablet_bootstrap.cc:492] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:14.671242 10033 tablet_bootstrap.cc:654] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.671456 10038 raft_consensus.cc:493] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.672068 10038 raft_consensus.cc:515] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:14.674670 10038 leader_election.cc:290] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:14.675732  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b189ef5046cd47ebb3d431e2ef24d6fb" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:14.678035  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b189ef5046cd47ebb3d431e2ef24d6fb" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
W20260709 13:05:14.680418  9725 leader_election.cc:343] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:14.681442  9724 leader_election.cc:343] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.682153 10029 ts_tablet_manager.cc:1434] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:05:14.682029  9724 leader_election.cc:304] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:14.696798 10039 raft_consensus.cc:2749] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:14.691908 10033 tablet_bootstrap.cc:492] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:14.698186 10033 ts_tablet_manager.cc:1403] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.015s
I20260709 13:05:14.695819 10028 tablet_bootstrap.cc:654] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.696923 10029 tablet_bootstrap.cc:492] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:14.701864 10033 raft_consensus.cc:359] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.702768 10033 raft_consensus.cc:385] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.686321 10030 tablet_bootstrap.cc:492] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:14.703241 10033 raft_consensus.cc:740] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.704276 10030 ts_tablet_manager.cc:1403] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.191s	user 0.013s	sys 0.025s
I20260709 13:05:14.704686 10033 consensus_queue.cc:260] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.705698  9657 catalog_manager.cc:5697] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:14.707182 10030 raft_consensus.cc:359] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.707850 10030 raft_consensus.cc:385] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.708099 10030 raft_consensus.cc:740] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.708956 10029 tablet_bootstrap.cc:654] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.708878 10030 consensus_queue.cc:260] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.712569 10116 raft_consensus.cc:493] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.712746 10033 ts_tablet_manager.cc:1434] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.002s
I20260709 13:05:14.713161 10116 raft_consensus.cc:515] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.714577 10033 tablet_bootstrap.cc:492] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:14.716733 10030 ts_tablet_manager.cc:1434] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:05:14.717056  9757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db11dbab2c874e55b70a46aba53095e0" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:14.717757 10030 tablet_bootstrap.cc:492] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:14.717703  9757 raft_consensus.cc:2468] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:14.719240  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db11dbab2c874e55b70a46aba53095e0" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:14.719923  9833 raft_consensus.cc:2468] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:14.721230  9872 leader_election.cc:304] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:14.722224 10091 raft_consensus.cc:2804] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:14.722647 10091 raft_consensus.cc:493] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.723042 10091 raft_consensus.cc:3060] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.725271 10030 tablet_bootstrap.cc:654] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.732728 10028 tablet_bootstrap.cc:492] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:14.733278 10028 ts_tablet_manager.cc:1403] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.107s	user 0.015s	sys 0.003s
I20260709 13:05:14.734200 10116 leader_election.cc:290] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:14.737811 10028 raft_consensus.cc:359] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.738581 10028 raft_consensus.cc:385] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.738865 10028 raft_consensus.cc:740] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.739562 10028 consensus_queue.cc:260] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.741855 10082 raft_consensus.cc:493] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.742288 10082 raft_consensus.cc:515] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.744470 10082 leader_election.cc:290] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:14.747094 10033 tablet_bootstrap.cc:654] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.750630  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716c25a1e46547c999c1c7c860e23f3b" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:14.751334 10028 ts_tablet_manager.cc:1434] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:05:14.752409 10028 tablet_bootstrap.cc:492] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:14.753480  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716c25a1e46547c999c1c7c860e23f3b" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:14.754419  9907 raft_consensus.cc:2468] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:14.755578 10091 raft_consensus.cc:515] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
W20260709 13:05:14.756595  9724 leader_election.cc:343] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.760403  9726 leader_election.cc:304] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:14.761627 10082 raft_consensus.cc:2804] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:14.762161 10082 raft_consensus.cc:493] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.762686 10082 raft_consensus.cc:3060] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.763023 10029 tablet_bootstrap.cc:492] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:14.765503  9757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db11dbab2c874e55b70a46aba53095e0" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:14.764501 10028 tablet_bootstrap.cc:654] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.766148  9757 raft_consensus.cc:3060] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.768476  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db11dbab2c874e55b70a46aba53095e0" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:14.776340  9833 raft_consensus.cc:3060] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.780889 10028 tablet_bootstrap.cc:492] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:14.781440 10028 ts_tablet_manager.cc:1403] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.001s
I20260709 13:05:14.784323 10082 raft_consensus.cc:515] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.786374 10033 tablet_bootstrap.cc:492] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:14.786870 10033 ts_tablet_manager.cc:1403] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.073s	user 0.010s	sys 0.012s
I20260709 13:05:14.787149 10082 leader_election.cc:290] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:14.788331  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716c25a1e46547c999c1c7c860e23f3b" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:14.784339 10028 raft_consensus.cc:359] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.789004 10028 raft_consensus.cc:385] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.789410 10028 raft_consensus.cc:740] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.765893 10029 ts_tablet_manager.cc:1403] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.069s	user 0.010s	sys 0.002s
I20260709 13:05:14.790407 10028 consensus_queue.cc:260] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
W20260709 13:05:14.791100  9724 leader_election.cc:343] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.790023  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716c25a1e46547c999c1c7c860e23f3b" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:14.792095  9907 raft_consensus.cc:3060] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.793004 10029 raft_consensus.cc:359] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.793709 10029 raft_consensus.cc:385] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.793960 10029 raft_consensus.cc:740] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.794590 10091 leader_election.cc:290] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:14.795576 10028 ts_tablet_manager.cc:1434] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 13:05:14.795800 10029 consensus_queue.cc:260] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.796859 10028 tablet_bootstrap.cc:492] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:14.797726 10029 ts_tablet_manager.cc:1434] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.004s
I20260709 13:05:14.798728 10029 tablet_bootstrap.cc:492] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:14.789824 10033 raft_consensus.cc:359] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:14.799641 10033 raft_consensus.cc:385] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.799917 10033 raft_consensus.cc:740] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.800664 10033 consensus_queue.cc:260] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:14.801947 10198 consensus_queue.cc:1048] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:14.804337 10033 ts_tablet_manager.cc:1434] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.002s
I20260709 13:05:14.809141 10033 tablet_bootstrap.cc:492] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:14.827054  9907 raft_consensus.cc:2468] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:14.842414 10029 tablet_bootstrap.cc:654] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.842525  9726 leader_election.cc:304] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:14.844419  9833 raft_consensus.cc:2468] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:14.849428 10028 tablet_bootstrap.cc:654] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.849586 10039 raft_consensus.cc:2804] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:14.851359 10039 raft_consensus.cc:697] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:14.852319 10039 consensus_queue.cc:237] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.852623 10199 consensus_queue.cc:1048] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:14.854370 10033 tablet_bootstrap.cc:654] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.854511  9874 leader_election.cc:304] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:14.845950  9757 raft_consensus.cc:2468] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:14.857614 10091 raft_consensus.cc:2804] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:14.858196 10091 raft_consensus.cc:697] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:14.862134 10091 consensus_queue.cc:237] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.880666 10030 tablet_bootstrap.cc:492] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
W20260709 13:05:14.879081  9706 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:14.881415 10030 ts_tablet_manager.cc:1403] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.164s	user 0.040s	sys 0.044s
I20260709 13:05:14.883594 10033 tablet_bootstrap.cc:492] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:14.884119 10033 ts_tablet_manager.cc:1403] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.078s	user 0.015s	sys 0.002s
I20260709 13:05:14.887356 10033 raft_consensus.cc:359] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:14.888082 10033 raft_consensus.cc:385] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.888357 10033 raft_consensus.cc:740] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.889199 10033 consensus_queue.cc:260] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:14.891726 10033 ts_tablet_manager.cc:1434] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:05:14.892318 10028 tablet_bootstrap.cc:492] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:14.892832 10028 ts_tablet_manager.cc:1403] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.096s	user 0.010s	sys 0.006s
I20260709 13:05:14.895983 10028 raft_consensus.cc:359] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.896692 10028 raft_consensus.cc:385] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.896956 10028 raft_consensus.cc:740] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.897308 10033 tablet_bootstrap.cc:492] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:14.897753 10028 consensus_queue.cc:260] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:14.899991 10028 ts_tablet_manager.cc:1434] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:14.901232 10028 tablet_bootstrap.cc:492] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:14.907307 10028 tablet_bootstrap.cc:654] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.918557 10028 tablet_bootstrap.cc:492] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:14.919687 10028 ts_tablet_manager.cc:1403] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:05:14.923341 10028 raft_consensus.cc:359] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.924041 10028 raft_consensus.cc:385] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.924357 10028 raft_consensus.cc:740] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.925154 10028 consensus_queue.cc:260] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.930038 10028 ts_tablet_manager.cc:1434] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:05:14.930480 10197 raft_consensus.cc:493] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.929570 10030 raft_consensus.cc:359] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.930987 10033 tablet_bootstrap.cc:654] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.931394 10030 raft_consensus.cc:385] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.931089 10197 raft_consensus.cc:515] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.931809 10030 raft_consensus.cc:740] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.925170  9657 catalog_manager.cc:5697] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:14.932711 10030 consensus_queue.cc:260] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.937263  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d7bb9a0b174b168df9040b02c70ab9" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:14.938059  9907 raft_consensus.cc:2468] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:14.938097  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d7bb9a0b174b168df9040b02c70ab9" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:14.940460  9799 leader_election.cc:343] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.941668 10197 leader_election.cc:290] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:14.943964  9801 leader_election.cc:304] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:14.945313 10197 raft_consensus.cc:2804] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:14.945803 10197 raft_consensus.cc:493] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.946229 10197 raft_consensus.cc:3060] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.946784 10030 ts_tablet_manager.cc:1434] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.065s	user 0.009s	sys 0.000s
I20260709 13:05:14.954501  9654 catalog_manager.cc:5697] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:14.957360 10030 tablet_bootstrap.cc:492] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:14.960170 10029 tablet_bootstrap.cc:492] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:14.960872 10029 ts_tablet_manager.cc:1403] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.162s	user 0.034s	sys 0.024s
I20260709 13:05:14.964092 10029 raft_consensus.cc:359] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.966005 10029 raft_consensus.cc:385] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:14.965103 10030 tablet_bootstrap.cc:654] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.966449 10029 raft_consensus.cc:740] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:14.965332 10197 raft_consensus.cc:515] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.969918  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d7bb9a0b174b168df9040b02c70ab9" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:14.970651  9907 raft_consensus.cc:3060] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.971697 10198 raft_consensus.cc:493] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.972265 10198 raft_consensus.cc:515] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.976382  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c62d82ebe5420bb298d787cda221aa" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:14.931036 10028 tablet_bootstrap.cc:492] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:14.977355  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d7bb9a0b174b168df9040b02c70ab9" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:14.978302  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c62d82ebe5420bb298d787cda221aa" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:14.979049  9908 raft_consensus.cc:2468] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
W20260709 13:05:14.979288  9799 leader_election.cc:343] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:14.974728 10198 leader_election.cc:290] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:14.977173  9980 raft_consensus.cc:2468] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:14.980648  9801 leader_election.cc:304] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:14.982640 10198 raft_consensus.cc:2804] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:14.983076 10198 raft_consensus.cc:493] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.983426 10198 raft_consensus.cc:3060] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:14.987210  9907 raft_consensus.cc:2468] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:14.990087 10028 tablet_bootstrap.cc:654] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:14.993121 10197 leader_election.cc:290] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:14.993912  9801 leader_election.cc:304] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:14.995795 10159 raft_consensus.cc:2804] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:14.995326 10029 consensus_queue.cc:260] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:14.996392 10159 raft_consensus.cc:697] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:14.997341 10159 consensus_queue.cc:237] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:14.998062 10029 ts_tablet_manager.cc:1434] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:05:14.999265 10029 tablet_bootstrap.cc:492] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:15.001892 10033 tablet_bootstrap.cc:492] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:15.002497 10033 ts_tablet_manager.cc:1403] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.105s	user 0.034s	sys 0.029s
I20260709 13:05:15.004087 10091 raft_consensus.cc:493] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.004679 10091 raft_consensus.cc:515] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:15.005590 10033 raft_consensus.cc:359] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:15.006292 10033 raft_consensus.cc:385] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.006552 10033 raft_consensus.cc:740] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.007385 10033 consensus_queue.cc:260] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:15.008285  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e044b992b684d52b54e3e8a4aa9720a" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:15.008946 10033 ts_tablet_manager.cc:1434] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:05:15.008970  9833 raft_consensus.cc:2468] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:15.009934 10033 tablet_bootstrap.cc:492] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:15.010772  9874 leader_election.cc:304] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:15.011830 10116 raft_consensus.cc:2804] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:15.012260 10116 raft_consensus.cc:493] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.012159 10198 raft_consensus.cc:515] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:15.012759 10116 raft_consensus.cc:3060] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.016000 10029 tablet_bootstrap.cc:654] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.016551  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e044b992b684d52b54e3e8a4aa9720a" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:15.018363  9873 leader_election.cc:343] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.019120 10091 leader_election.cc:290] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:15.019361 10033 tablet_bootstrap.cc:654] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.026198 10116 raft_consensus.cc:515] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:15.019363  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c62d82ebe5420bb298d787cda221aa" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:15.028220 10091 raft_consensus.cc:493] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.028620  9907 raft_consensus.cc:3060] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.028860 10091 raft_consensus.cc:515] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.030033 10198 leader_election.cc:290] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:15.031648  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c62d82ebe5420bb298d787cda221aa" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:15.032438  9980 raft_consensus.cc:3060] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.032737  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbee4fe35264c5aa3ea32adc034b9c7" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:15.034469  9873 leader_election.cc:343] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.042011 10028 tablet_bootstrap.cc:492] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:15.042234  9980 raft_consensus.cc:2468] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:15.042603 10028 ts_tablet_manager.cc:1403] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.112s	user 0.019s	sys 0.008s
I20260709 13:05:15.045749 10028 raft_consensus.cc:359] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.046422 10028 raft_consensus.cc:385] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.046746 10028 raft_consensus.cc:740] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.046530 10039 consensus_queue.cc:1048] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:15.047456 10028 consensus_queue.cc:260] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.047684  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbee4fe35264c5aa3ea32adc034b9c7" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:15.048519  9760 raft_consensus.cc:2468] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:15.049633 10028 ts_tablet_manager.cc:1434] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:05:15.051843  9907 raft_consensus.cc:2468] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:15.051757  9872 leader_election.cc:304] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:15.053542 10147 raft_consensus.cc:2804] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:15.053640  9801 leader_election.cc:304] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:15.053984 10147 raft_consensus.cc:493] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.054423 10147 raft_consensus.cc:3060] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.056540 10039 raft_consensus.cc:493] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.057032 10039 raft_consensus.cc:515] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.057520 10028 tablet_bootstrap.cc:492] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:15.057869 10197 raft_consensus.cc:2804] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:15.058475 10197 raft_consensus.cc:697] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:15.059401 10197 consensus_queue.cc:237] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:15.060009  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dddadb853be408b8e6a1103a45ce0a1" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:15.063040 10039 leader_election.cc:290] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:15.064353  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dddadb853be408b8e6a1103a45ce0a1" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:15.065496  9908 raft_consensus.cc:2393] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 1: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
I20260709 13:05:15.070822  9726 leader_election.cc:304] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:15.072000 10082 raft_consensus.cc:3060] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.073349 10028 tablet_bootstrap.cc:654] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.074194 10091 leader_election.cc:290] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:15.077517 10116 leader_election.cc:290] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:15.079028  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e044b992b684d52b54e3e8a4aa9720a" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:15.079730 10134 raft_consensus.cc:493] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.087440 10134 raft_consensus.cc:515] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:15.088964 10082 raft_consensus.cc:2749] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:15.093780 10030 tablet_bootstrap.cc:492] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:15.094363 10030 ts_tablet_manager.cc:1403] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.137s	user 0.011s	sys 0.006s
I20260709 13:05:15.081697 10147 raft_consensus.cc:515] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.098090 10030 raft_consensus.cc:359] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.100564 10030 raft_consensus.cc:385] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.101203 10030 raft_consensus.cc:740] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.102200 10030 consensus_queue.cc:260] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.108671 10030 ts_tablet_manager.cc:1434] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 13:05:15.109158 10134 leader_election.cc:290] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:15.111083 10030 tablet_bootstrap.cc:492] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:15.111729  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:15.113263 10159 consensus_queue.cc:1048] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:15.125597 10033 tablet_bootstrap.cc:492] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:15.126303 10033 ts_tablet_manager.cc:1403] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.117s	user 0.007s	sys 0.027s
I20260709 13:05:15.109786  9654 catalog_manager.cc:5697] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:15.101717  9873 leader_election.cc:343] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.133715  9907 raft_consensus.cc:2393] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23cbd6234c5e43d89f22c235ed6bb96a in current term 1: Already voted for candidate 6e10aab26c644389b8161974dcabc5d0 in this term.
I20260709 13:05:15.148942 10033 raft_consensus.cc:359] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.149857 10033 raft_consensus.cc:385] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.150174 10033 raft_consensus.cc:740] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.150902 10147 leader_election.cc:290] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:15.151124 10033 consensus_queue.cc:260] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.154774 10029 tablet_bootstrap.cc:492] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:15.156157 10029 ts_tablet_manager.cc:1403] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.157s	user 0.021s	sys 0.018s
I20260709 13:05:15.160696  9757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbee4fe35264c5aa3ea32adc034b9c7" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:15.161439  9757 raft_consensus.cc:3060] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.163384 10029 raft_consensus.cc:359] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.168113 10029 raft_consensus.cc:385] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.171283 10029 raft_consensus.cc:740] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.188367 10033 ts_tablet_manager.cc:1434] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.062s	user 0.008s	sys 0.003s
I20260709 13:05:15.190016 10033 tablet_bootstrap.cc:492] T c23fd23d082a4c58ab8404e8b9895ad4 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:15.177593  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:15.193682 10029 consensus_queue.cc:260] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.198364 10049 raft_consensus.cc:493] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.199368 10030 tablet_bootstrap.cc:654] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.200716  9757 raft_consensus.cc:2468] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:15.202314  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbee4fe35264c5aa3ea32adc034b9c7" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:15.206185 10049 raft_consensus.cc:515] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.209620 10198 consensus_queue.cc:1048] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:15.212466 10049 leader_election.cc:290] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:15.230757 10028 tablet_bootstrap.cc:492] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:15.231415 10028 ts_tablet_manager.cc:1403] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.174s	user 0.024s	sys 0.043s
I20260709 13:05:15.226584  9833 raft_consensus.cc:2393] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23cbd6234c5e43d89f22c235ed6bb96a in current term 1: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
I20260709 13:05:15.233788 10029 ts_tablet_manager.cc:1434] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.077s	user 0.010s	sys 0.001s
I20260709 13:05:15.234773 10029 tablet_bootstrap.cc:492] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:15.238721 10033 tablet_bootstrap.cc:654] T c23fd23d082a4c58ab8404e8b9895ad4 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.227337 10082 consensus_queue.cc:1048] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:05:15.244863  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c23fd23d082a4c58ab8404e8b9895ad4" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:15.245419 10199 raft_consensus.cc:493] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.246057  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e044b992b684d52b54e3e8a4aa9720a" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
W20260709 13:05:15.235476  9873 leader_election.cc:343] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.250011  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c23fd23d082a4c58ab8404e8b9895ad4" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:15.249794 10199 raft_consensus.cc:515] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.268563 10028 raft_consensus.cc:359] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.270526 10028 raft_consensus.cc:385] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.250902 10235 raft_consensus.cc:493] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:15.254323  9874 leader_election.cc:343] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.254498  9949 leader_election.cc:304] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:15.271266 10199 leader_election.cc:290] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:15.273379 10134 raft_consensus.cc:3060] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.271250 10235 raft_consensus.cc:515] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.276683 10235 leader_election.cc:290] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:15.280083  9872 leader_election.cc:304] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:15.266805 10029 tablet_bootstrap.cc:654] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.267588  9760 raft_consensus.cc:2468] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:15.270938 10028 raft_consensus.cc:740] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.285028 10028 consensus_queue.cc:260] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.289143 10030 tablet_bootstrap.cc:492] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:15.289934 10030 ts_tablet_manager.cc:1403] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.179s	user 0.014s	sys 0.010s
I20260709 13:05:15.282212  9833 raft_consensus.cc:3060] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.293639 10234 consensus_queue.cc:1048] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:15.282109 10147 raft_consensus.cc:2804] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:15.294283 10134 raft_consensus.cc:2749] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:15.301681 10028 ts_tablet_manager.cc:1434] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.001s
I20260709 13:05:15.303555 10028 tablet_bootstrap.cc:492] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:15.294726 10147 raft_consensus.cc:697] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:15.306157 10147 consensus_queue.cc:237] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.307319  9758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b189ef5046cd47ebb3d431e2ef24d6fb" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:15.308138  9758 raft_consensus.cc:2468] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:15.309237  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9579648f7487465aa44ecf73e23731d8" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:15.310184 10028 tablet_bootstrap.cc:654] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.310392  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b189ef5046cd47ebb3d431e2ef24d6fb" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:15.311043  9981 raft_consensus.cc:2468] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:15.312167  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9579648f7487465aa44ecf73e23731d8" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:15.339810 10030 raft_consensus.cc:359] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.340615 10030 raft_consensus.cc:385] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.340873 10030 raft_consensus.cc:740] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.341543 10030 consensus_queue.cc:260] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.350363 10030 ts_tablet_manager.cc:1434] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.000s
I20260709 13:05:15.351266 10030 tablet_bootstrap.cc:492] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:15.352947 10033 tablet_bootstrap.cc:492] T c23fd23d082a4c58ab8404e8b9895ad4 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:15.353626 10033 ts_tablet_manager.cc:1403] T c23fd23d082a4c58ab8404e8b9895ad4 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.164s	user 0.009s	sys 0.017s
I20260709 13:05:15.355708  9833 raft_consensus.cc:2468] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:15.358047  9872 leader_election.cc:304] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:15.363775 10049 raft_consensus.cc:2804] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:15.364408 10049 raft_consensus.cc:493] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.356477  9799 leader_election.cc:304] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:15.364571 10038 consensus_queue.cc:1048] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:15.364982 10049 raft_consensus.cc:3060] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:15.365592  9799 leader_election.cc:343] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.367300 10198 raft_consensus.cc:2804] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:15.367739 10198 raft_consensus.cc:493] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.368178 10198 raft_consensus.cc:3060] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.369666 10033 raft_consensus.cc:359] T c23fd23d082a4c58ab8404e8b9895ad4 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.370631 10033 raft_consensus.cc:385] T c23fd23d082a4c58ab8404e8b9895ad4 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.372761 10033 raft_consensus.cc:740] T c23fd23d082a4c58ab8404e8b9895ad4 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.376402 10033 consensus_queue.cc:260] T c23fd23d082a4c58ab8404e8b9895ad4 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.379570  9874 leader_election.cc:304] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:15.380796 10230 raft_consensus.cc:2804] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:05:15.380858  9798 leader_election.cc:343] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.381422 10230 raft_consensus.cc:697] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:15.395350  9798 leader_election.cc:304] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:15.384344 10033 ts_tablet_manager.cc:1434] T c23fd23d082a4c58ab8404e8b9895ad4 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 13:05:15.395797 10230 consensus_queue.cc:237] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:15.396585 10235 raft_consensus.cc:2749] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:15.385664 10198 raft_consensus.cc:515] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.392802 10234 consensus_queue.cc:1048] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:15.397467 10033 tablet_bootstrap.cc:492] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:15.415827 10029 tablet_bootstrap.cc:492] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:15.416445 10029 ts_tablet_manager.cc:1403] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.182s	user 0.010s	sys 0.004s
I20260709 13:05:15.420099 10028 tablet_bootstrap.cc:492] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:15.420706 10028 ts_tablet_manager.cc:1403] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.117s	user 0.016s	sys 0.010s
I20260709 13:05:15.425024 10029 raft_consensus.cc:359] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.425863 10029 raft_consensus.cc:385] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.426156 10029 raft_consensus.cc:740] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.426908  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b189ef5046cd47ebb3d431e2ef24d6fb" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:15.427745 10028 raft_consensus.cc:359] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.431090 10028 raft_consensus.cc:385] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.431465 10028 raft_consensus.cc:740] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.427616  9981 raft_consensus.cc:3060] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.432209 10028 consensus_queue.cc:260] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.435194 10029 consensus_queue.cc:260] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.434880 10028 ts_tablet_manager.cc:1434] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:05:15.443528 10028 tablet_bootstrap.cc:492] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:15.375908 10049 raft_consensus.cc:515] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.428200  9979 raft_consensus.cc:3060] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.445370 10234 raft_consensus.cc:493] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.462409 10029 ts_tablet_manager.cc:1434] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.001s
I20260709 13:05:15.464898 10029 tablet_bootstrap.cc:492] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:15.460127 10198 leader_election.cc:290] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:15.498697 10030 tablet_bootstrap.cc:654] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.467835 10234 raft_consensus.cc:515] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.503814  9758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b189ef5046cd47ebb3d431e2ef24d6fb" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:15.504598  9758 raft_consensus.cc:3060] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.507328 10033 tablet_bootstrap.cc:654] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.505947 10234 leader_election.cc:290] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:15.524816  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f99aa4491c4458bb5ebc03bc0cd813" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:15.527264 10230 raft_consensus.cc:493] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.527813 10230 raft_consensus.cc:515] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.530150 10230 leader_election.cc:290] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:15.532848  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:15.526660  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f99aa4491c4458bb5ebc03bc0cd813" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:15.535424  9799 consensus_peers.cc:597] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Couldn't send request to peer 23cbd6234c5e43d89f22c235ed6bb96a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:15.538599  9873 leader_election.cc:343] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:15.539335  9873 leader_election.cc:343] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.546743  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:15.547272  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c23fd23d082a4c58ab8404e8b9895ad4" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:15.547924  9760 raft_consensus.cc:3060] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.550305 10159 consensus_queue.cc:1048] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:15.551453  9872 leader_election.cc:343] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.552066  9872 leader_election.cc:304] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:15.552999 10147 raft_consensus.cc:2749] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:15.554381  9872 leader_election.cc:343] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.554538  9981 raft_consensus.cc:2468] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:15.554841  9872 leader_election.cc:304] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:15.556775 10159 consensus_queue.cc:1048] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:15.560971  9799 leader_election.cc:304] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:15.563401 10250 raft_consensus.cc:2804] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:15.564258 10250 raft_consensus.cc:697] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:15.565320  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c23fd23d082a4c58ab8404e8b9895ad4" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:15.565282 10250 consensus_queue.cc:237] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.566068  9831 raft_consensus.cc:3060] T c23fd23d082a4c58ab8404e8b9895ad4 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.570259 10230 raft_consensus.cc:2749] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:15.572662  9656 catalog_manager.cc:5697] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:15.566721 10028 tablet_bootstrap.cc:654] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.584859 10235 consensus_queue.cc:1048] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:15.593281 10029 tablet_bootstrap.cc:654] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.592922 10049 leader_election.cc:290] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:15.601068 10250 raft_consensus.cc:493] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.602803 10250 raft_consensus.cc:515] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.591492  9760 raft_consensus.cc:2468] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:15.604704  9872 leader_election.cc:304] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:15.605643 10049 raft_consensus.cc:2804] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:15.606235 10049 raft_consensus.cc:697] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:15.606894 10250 leader_election.cc:290] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:15.607648  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "196922ea9dc14e078a3da6af1206f79a" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
W20260709 13:05:15.608968  9798 leader_election.cc:343] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.607206 10049 consensus_queue.cc:237] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.611712 10028 tablet_bootstrap.cc:492] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:15.612478  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "196922ea9dc14e078a3da6af1206f79a" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:15.613250 10028 ts_tablet_manager.cc:1403] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.170s	user 0.012s	sys 0.005s
I20260709 13:05:15.613375  9906 raft_consensus.cc:2468] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:15.617951  9801 leader_election.cc:304] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:15.619211 10159 raft_consensus.cc:2804] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:15.619990 10028 raft_consensus.cc:359] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:15.620755 10028 raft_consensus.cc:385] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.621035 10028 raft_consensus.cc:740] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.621765 10028 consensus_queue.cc:260] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:15.622138  9831 raft_consensus.cc:2468] T c23fd23d082a4c58ab8404e8b9895ad4 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:15.597790  9758 raft_consensus.cc:2468] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:15.624325 10033 tablet_bootstrap.cc:492] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:15.624910 10033 ts_tablet_manager.cc:1403] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.228s	user 0.010s	sys 0.009s
I20260709 13:05:15.619727 10159 raft_consensus.cc:493] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.629340 10159 raft_consensus.cc:3060] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.631500 10028 ts_tablet_manager.cc:1434] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.002s
I20260709 13:05:15.632925 10028 tablet_bootstrap.cc:492] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:15.634325 10029 tablet_bootstrap.cc:492] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:15.635004 10029 ts_tablet_manager.cc:1403] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.170s	user 0.013s	sys 0.002s
I20260709 13:05:15.637590 10033 raft_consensus.cc:359] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.637841 10029 raft_consensus.cc:359] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.638443 10033 raft_consensus.cc:385] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.638597 10029 raft_consensus.cc:385] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.638833 10033 raft_consensus.cc:740] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.639489 10028 tablet_bootstrap.cc:654] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.639008 10029 raft_consensus.cc:740] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.641311 10134 raft_consensus.cc:493] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.641848 10134 raft_consensus.cc:515] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.642354 10029 consensus_queue.cc:260] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.646775 10029 ts_tablet_manager.cc:1434] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:05:15.648200 10134 leader_election.cc:290] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:15.649086  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716c25a1e46547c999c1c7c860e23f3b" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:15.649086  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716c25a1e46547c999c1c7c860e23f3b" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:15.639979 10033 consensus_queue.cc:260] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.651716 10039 raft_consensus.cc:493] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.651111 10029 tablet_bootstrap.cc:492] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:15.652904 10039 raft_consensus.cc:515] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.659456 10033 ts_tablet_manager.cc:1434] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.001s
I20260709 13:05:15.659584 10082 consensus_queue.cc:1048] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:15.660301 10030 tablet_bootstrap.cc:492] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:15.660583 10033 tablet_bootstrap.cc:492] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:15.660925 10030 ts_tablet_manager.cc:1403] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.310s	user 0.025s	sys 0.006s
I20260709 13:05:15.663597  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "196922ea9dc14e078a3da6af1206f79a" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:15.664212  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "196922ea9dc14e078a3da6af1206f79a" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:15.664923  9908 raft_consensus.cc:2468] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:15.665130 10030 raft_consensus.cc:359] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.666157  9726 leader_election.cc:304] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:15.663708 10039 leader_election.cc:290] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:15.668277 10039 raft_consensus.cc:2804] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:15.668715 10039 raft_consensus.cc:493] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.668550  9949 leader_election.cc:304] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:15.669282 10039 raft_consensus.cc:3060] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.671689 10033 tablet_bootstrap.cc:654] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.678515 10030 raft_consensus.cc:385] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.679616 10030 raft_consensus.cc:740] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.681816 10030 consensus_queue.cc:260] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.678906  9655 catalog_manager.cc:5697] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:15.688571 10030 ts_tablet_manager.cc:1434] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.002s
I20260709 13:05:15.689675 10030 tablet_bootstrap.cc:492] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:15.691514 10039 raft_consensus.cc:515] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.692222 10230 consensus_queue.cc:1048] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:15.695892  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "196922ea9dc14e078a3da6af1206f79a" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:15.687568  9655 catalog_manager.cc:5697] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:15.703305 10030 tablet_bootstrap.cc:654] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.708134 10159 raft_consensus.cc:515] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.714169 10028 tablet_bootstrap.cc:492] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:15.717374 10028 ts_tablet_manager.cc:1403] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.085s	user 0.014s	sys 0.005s
I20260709 13:05:15.721325  9831 raft_consensus.cc:2393] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 1: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
W20260709 13:05:15.722033  9726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45019, user_credentials={real_user=slave}} blocked reactor thread for 55952.1us
I20260709 13:05:15.723650 10028 raft_consensus.cc:359] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.725167 10028 raft_consensus.cc:385] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.729743 10039 leader_election.cc:290] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:15.734701 10029 tablet_bootstrap.cc:654] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.725466 10028 raft_consensus.cc:740] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.737423  9981 raft_consensus.cc:3060] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.737633 10028 consensus_queue.cc:260] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.742591 10159 leader_election.cc:290] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:15.743795  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "196922ea9dc14e078a3da6af1206f79a" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:15.745956  9906 raft_consensus.cc:3060] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.745877  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "196922ea9dc14e078a3da6af1206f79a" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:15.748150  9760 raft_consensus.cc:2393] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a1cbca39e34743d8a40816c3bc5f2030 in current term 1: Already voted for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in this term.
I20260709 13:05:15.751230 10033 tablet_bootstrap.cc:492] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:15.751772 10033 ts_tablet_manager.cc:1403] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.091s	user 0.023s	sys 0.013s
I20260709 13:05:15.751713 10147 raft_consensus.cc:493] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.753046 10147 raft_consensus.cc:515] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:15.754261  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "196922ea9dc14e078a3da6af1206f79a" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:15.756100  9726 leader_election.cc:304] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:15.756438 10147 leader_election.cc:290] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:15.758275  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01c76363b154905982dc75dbdead3be" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:15.759073  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01c76363b154905982dc75dbdead3be" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:15.759595  9874 leader_election.cc:343] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:15.760681  9873 leader_election.cc:343] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.761134  9873 leader_election.cc:304] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:15.762686 10233 raft_consensus.cc:2749] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:15.764317 10262 raft_consensus.cc:2749] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:15.764544 10134 raft_consensus.cc:2749] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:15.767556 10033 raft_consensus.cc:359] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.768314 10033 raft_consensus.cc:385] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.768608 10033 raft_consensus.cc:740] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.769296 10033 consensus_queue.cc:260] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.771734 10033 ts_tablet_manager.cc:1434] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:05:15.772761 10033 tablet_bootstrap.cc:492] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:15.744086 10028 ts_tablet_manager.cc:1434] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 13:05:15.777842 10028 tablet_bootstrap.cc:492] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:15.778337 10033 tablet_bootstrap.cc:654] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.778822  9656 catalog_manager.cc:5697] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:15.781064 10262 raft_consensus.cc:493] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.792297 10262 raft_consensus.cc:515] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.797758 10134 raft_consensus.cc:493] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.798162  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:15.798538 10134 raft_consensus.cc:515] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
W20260709 13:05:15.800423  9724 leader_election.cc:343] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.802640 10134 leader_election.cc:290] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:15.806535  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbee4fe35264c5aa3ea32adc034b9c7" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:15.807495  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:15.807684  9906 raft_consensus.cc:2468] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:15.808244  9833 raft_consensus.cc:2468] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:15.810474  9801 leader_election.cc:304] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:15.812572 10233 raft_consensus.cc:493] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.812914 10159 raft_consensus.cc:2804] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:15.813138 10233 raft_consensus.cc:515] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.813891 10159 raft_consensus.cc:697] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:15.816124 10233 leader_election.cc:290] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:15.817035  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82b4b9bc0c546adac703578be161959" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:15.817868 10028 tablet_bootstrap.cc:654] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:15.818774  9872 leader_election.cc:343] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.820165 10233 consensus_queue.cc:1048] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:15.823405  9725 leader_election.cc:304] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:15.824553 10039 raft_consensus.cc:2804] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:15.825032 10039 raft_consensus.cc:493] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.825464 10039 raft_consensus.cc:3060] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.826617  9655 catalog_manager.cc:5697] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:15.804896  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbee4fe35264c5aa3ea32adc034b9c7" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:15.830780  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82b4b9bc0c546adac703578be161959" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:15.831105  9949 leader_election.cc:304] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:15.831964  9831 raft_consensus.cc:2468] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:15.833662 10134 raft_consensus.cc:2749] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:15.833294 10159 consensus_queue.cc:237] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.840374 10233 consensus_queue.cc:1048] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:15.844488  9874 leader_election.cc:304] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
W20260709 13:05:15.847237  9873 consensus_peers.cc:597] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 -> Peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Couldn't send request to peer 23cbd6234c5e43d89f22c235ed6bb96a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:15.852618 10233 raft_consensus.cc:2804] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:15.853152 10233 raft_consensus.cc:493] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.852607 10028 tablet_bootstrap.cc:492] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:15.857612 10030 tablet_bootstrap.cc:492] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:15.857755 10028 ts_tablet_manager.cc:1403] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.080s	user 0.008s	sys 0.004s
I20260709 13:05:15.858263 10030 ts_tablet_manager.cc:1403] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.169s	user 0.015s	sys 0.010s
I20260709 13:05:15.858616 10262 leader_election.cc:290] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:15.860904 10028 raft_consensus.cc:359] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.861722 10028 raft_consensus.cc:385] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.862036 10028 raft_consensus.cc:740] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.853601 10233 raft_consensus.cc:3060] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.852607 10029 tablet_bootstrap.cc:492] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:15.863310 10029 ts_tablet_manager.cc:1403] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.216s	user 0.022s	sys 0.007s
I20260709 13:05:15.866673 10029 raft_consensus.cc:359] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.867440 10029 raft_consensus.cc:385] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.874136 10039 raft_consensus.cc:515] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.876638 10039 leader_election.cc:290] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:15.877362  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:15.877759  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:15.878481  9831 raft_consensus.cc:3060] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:15.878750  9724 leader_election.cc:343] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.880550 10030 raft_consensus.cc:359] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.882153 10233 raft_consensus.cc:515] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:15.885629  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82b4b9bc0c546adac703578be161959" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
W20260709 13:05:15.887297  9872 leader_election.cc:343] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.888228 10028 consensus_queue.cc:260] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.890805 10028 ts_tablet_manager.cc:1434] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.001s
I20260709 13:05:15.891845 10028 tablet_bootstrap.cc:492] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:15.867733 10029 raft_consensus.cc:740] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.896925 10029 consensus_queue.cc:260] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:15.899322 10029 ts_tablet_manager.cc:1434] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.001s
I20260709 13:05:15.881317 10030 raft_consensus.cc:385] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.902177 10030 raft_consensus.cc:740] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.902313  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82b4b9bc0c546adac703578be161959" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:15.903048  9834 raft_consensus.cc:3060] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:15.903064 10030 consensus_queue.cc:260] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.910523 10029 tablet_bootstrap.cc:492] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:15.912124 10030 ts_tablet_manager.cc:1434] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.003s
I20260709 13:05:15.913367 10030 tablet_bootstrap.cc:492] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:15.913715 10233 leader_election.cc:290] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:15.914431  9834 raft_consensus.cc:2468] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:15.930037 10197 raft_consensus.cc:493] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.930707 10197 raft_consensus.cc:515] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.933209 10029 tablet_bootstrap.cc:654] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.934621 10197 leader_election.cc:290] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:15.935842 10030 tablet_bootstrap.cc:654] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.936982  9874 leader_election.cc:304] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
W20260709 13:05:15.939414  9706 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:15.939342  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:15.939764  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:15.941278  9799 leader_election.cc:343] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:15.941771  9798 leader_election.cc:343] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:15.942215  9798 leader_election.cc:304] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:15.944571 10233 raft_consensus.cc:2804] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:15.945744 10250 raft_consensus.cc:2749] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:15.945583 10233 raft_consensus.cc:697] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:15.950412 10030 tablet_bootstrap.cc:492] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:15.951112 10030 ts_tablet_manager.cc:1403] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.001s
W20260709 13:05:15.951638 10269 log.cc:927] Time spent T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:05:15.953675 10028 tablet_bootstrap.cc:654] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:15.954592 10030 raft_consensus.cc:359] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.962036 10030 raft_consensus.cc:385] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:15.962414 10030 raft_consensus.cc:740] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:15.963255 10030 consensus_queue.cc:260] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:15.969468 10030 ts_tablet_manager.cc:1434] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:05:15.970389 10030 tablet_bootstrap.cc:492] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:15.975167 10197 consensus_queue.cc:1048] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:15.985060 10030 tablet_bootstrap.cc:654] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:16.002611 10030 tablet_bootstrap.cc:492] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:16.004361 10030 ts_tablet_manager.cc:1403] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.000s
I20260709 13:05:16.008605 10030 raft_consensus.cc:359] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.009532 10030 raft_consensus.cc:385] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:16.016368 10030 raft_consensus.cc:740] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:16.017226 10030 consensus_queue.cc:260] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.022024 10030 ts_tablet_manager.cc:1434] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 13:05:16.025362  9831 raft_consensus.cc:2468] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:16.025699 10030 tablet_bootstrap.cc:492] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:16.027393  9725 leader_election.cc:304] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:16.029939 10038 raft_consensus.cc:2804] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:16.030659 10038 raft_consensus.cc:697] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:16.031880 10038 consensus_queue.cc:237] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:16.040674 10072 raft_consensus.cc:493] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.041291 10072 raft_consensus.cc:515] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.043558 10072 leader_election.cc:290] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:16.045843  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e044b992b684d52b54e3e8a4aa9720a" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:15.947549 10233 consensus_queue.cc:237] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:16.052274  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e044b992b684d52b54e3e8a4aa9720a" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:16.058717  9948 leader_election.cc:304] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:16.060398 10072 raft_consensus.cc:2749] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:16.066006 10030 tablet_bootstrap.cc:654] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:16.065510 10147 consensus_queue.cc:1048] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:15.990502  9656 catalog_manager.cc:5697] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:16.087613 10072 raft_consensus.cc:493] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.088279 10072 raft_consensus.cc:515] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:16.085170 10039 raft_consensus.cc:493] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.091934 10039 raft_consensus.cc:515] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:16.093160  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:16.094050  9833 raft_consensus.cc:2393] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23cbd6234c5e43d89f22c235ed6bb96a in current term 1: Already voted for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in this term.
I20260709 13:05:16.095055  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:16.096410  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:16.097466  9906 raft_consensus.cc:2468] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:16.097885  9948 leader_election.cc:304] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:16.098865 10134 raft_consensus.cc:3060] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:16.101090  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:16.103042 10072 leader_election.cc:290] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
W20260709 13:05:16.104632  9724 leader_election.cc:343] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:16.108980 10134 raft_consensus.cc:2749] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:16.136251  9656 catalog_manager.cc:5697] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:16.122110 10039 leader_election.cc:290] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:16.154564 10049 raft_consensus.cc:493] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.155259 10049 raft_consensus.cc:515] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:16.157853 10049 leader_election.cc:290] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:16.184075 10197 consensus_queue.cc:1048] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:16.187356  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "168ffc86a17d47e8aba438779717eab6" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:16.143532  9726 leader_election.cc:304] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
W20260709 13:05:16.201844  9726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45019, user_credentials={real_user=slave}} blocked reactor thread for 65307.5us
I20260709 13:05:16.203365 10262 raft_consensus.cc:2804] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:16.203915 10262 raft_consensus.cc:493] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.204305 10262 raft_consensus.cc:3060] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:16.231060 10250 consensus_queue.cc:1048] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:16.257140 10028 tablet_bootstrap.cc:492] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:16.257850 10028 ts_tablet_manager.cc:1403] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.366s	user 0.008s	sys 0.011s
W20260709 13:05:16.259781  9873 leader_election.cc:343] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:16.296963 10029 tablet_bootstrap.cc:492] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:16.304003 10029 ts_tablet_manager.cc:1403] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.394s	user 0.053s	sys 0.032s
I20260709 13:05:16.288519 10028 raft_consensus.cc:359] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:16.305101 10028 raft_consensus.cc:385] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:16.302635  9654 catalog_manager.cc:5697] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:16.320688 10262 raft_consensus.cc:515] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:16.305392 10028 raft_consensus.cc:740] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:16.347015 10028 consensus_queue.cc:260] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:16.352993 10029 raft_consensus.cc:359] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:16.353811 10029 raft_consensus.cc:385] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:16.354094 10029 raft_consensus.cc:740] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:16.354774 10029 consensus_queue.cc:260] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:16.365854 10028 ts_tablet_manager.cc:1434] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.108s	user 0.009s	sys 0.003s
I20260709 13:05:16.367051 10028 tablet_bootstrap.cc:492] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
W20260709 13:05:16.374723 10033 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.148s	sys 0.293s
I20260709 13:05:16.376821 10029 ts_tablet_manager.cc:1434] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.073s	user 0.008s	sys 0.000s
I20260709 13:05:16.378280 10029 tablet_bootstrap.cc:492] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:16.378562 10033 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.148s	sys 0.293s
W20260709 13:05:16.375739  9874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 179199us
I20260709 13:05:16.390190 10134 raft_consensus.cc:493] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.412384 10030 tablet_bootstrap.cc:492] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:16.412513  9757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "168ffc86a17d47e8aba438779717eab6" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:16.413030 10030 ts_tablet_manager.cc:1403] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.390s	user 0.061s	sys 0.038s
I20260709 13:05:16.412397 10033 tablet_bootstrap.cc:492] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:16.414938 10033 ts_tablet_manager.cc:1403] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.642s	user 0.157s	sys 0.293s
I20260709 13:05:16.410343 10134 raft_consensus.cc:515] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:16.425217 10262 leader_election.cc:290] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:16.425319 10033 raft_consensus.cc:359] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.427462 10033 raft_consensus.cc:385] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:16.427667 10033 raft_consensus.cc:740] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:16.428386 10033 consensus_queue.cc:260] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.434717 10030 raft_consensus.cc:359] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.435616 10030 raft_consensus.cc:385] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:16.435890 10030 raft_consensus.cc:740] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:16.436633 10030 consensus_queue.cc:260] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.437273  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:16.434816 10134 leader_election.cc:290] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
W20260709 13:05:16.440230  9724 leader_election.cc:343] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:16.440866  9979 raft_consensus.cc:3060] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:16.444422  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:16.445101  9908 raft_consensus.cc:3060] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:16.447235  9872 leader_election.cc:343] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:16.447733  9872 leader_election.cc:304] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:16.449446 10116 raft_consensus.cc:2749] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:16.449815 10147 consensus_queue.cc:1048] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:16.452986 10033 ts_tablet_manager.cc:1434] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 13:05:16.455066  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9579648f7487465aa44ecf73e23731d8" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:16.455873  9835 raft_consensus.cc:2468] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 0.
I20260709 13:05:16.457654  9757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9579648f7487465aa44ecf73e23731d8" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:16.458377  9757 raft_consensus.cc:2468] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 0.
I20260709 13:05:16.468948 10030 ts_tablet_manager.cc:1434] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.000s
I20260709 13:05:16.472890 10033 tablet_bootstrap.cc:492] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:16.472791  9948 leader_election.cc:304] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:16.473820 10134 raft_consensus.cc:2804] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:16.479082 10262 consensus_queue.cc:1048] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:16.474445 10134 raft_consensus.cc:493] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.482875 10285 consensus_queue.cc:1048] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:16.484521 10134 raft_consensus.cc:3060] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:16.487347 10030 tablet_bootstrap.cc:492] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:16.493636 10029 tablet_bootstrap.cc:654] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:16.515296 10030 tablet_bootstrap.cc:654] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:16.515676 10028 tablet_bootstrap.cc:654] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:16.525523  9982 raft_consensus.cc:3060] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:16.552985 10197 raft_consensus.cc:493] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.555524 10197 raft_consensus.cc:515] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.557058 10033 tablet_bootstrap.cc:654] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:16.568204 10233 raft_consensus.cc:493] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.568989 10233 raft_consensus.cc:515] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.572188 10197 leader_election.cc:290] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:16.575610 10233 leader_election.cc:290] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:16.540874 10134 raft_consensus.cc:515] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:16.606845 10030 tablet_bootstrap.cc:492] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:16.607419 10030 ts_tablet_manager.cc:1403] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.120s	user 0.021s	sys 0.007s
I20260709 13:05:16.610060  9908 raft_consensus.cc:2468] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:16.634780 10250 raft_consensus.cc:493] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.635555 10134 leader_election.cc:290] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:16.640605 10028 tablet_bootstrap.cc:492] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:16.641218 10028 ts_tablet_manager.cc:1403] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.274s	user 0.013s	sys 0.008s
I20260709 13:05:16.629599  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01c76363b154905982dc75dbdead3be" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:16.626217  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:16.658381 10030 raft_consensus.cc:359] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:16.660264 10030 raft_consensus.cc:385] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:16.660591 10030 raft_consensus.cc:740] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:16.635589 10250 raft_consensus.cc:515] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.657377  9726 leader_election.cc:304] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:16.665683  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9579648f7487465aa44ecf73e23731d8" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:16.669844  9760 raft_consensus.cc:3060] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:16.670014 10038 raft_consensus.cc:2804] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:16.670657 10038 raft_consensus.cc:697] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:16.671563 10038 consensus_queue.cc:237] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
W20260709 13:05:16.663622  9799 leader_election.cc:343] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:16.671814 10250 leader_election.cc:290] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:16.682787  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9579648f7487465aa44ecf73e23731d8" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:16.675192 10049 raft_consensus.cc:493] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.657017  9978 raft_consensus.cc:2468] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:16.687978  9831 raft_consensus.cc:3060] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:16.688055 10049 raft_consensus.cc:515] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:16.688970 10072 raft_consensus.cc:493] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a1cbca39e34743d8a40816c3bc5f2030)
I20260709 13:05:16.689565 10072 raft_consensus.cc:515] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.690560 10049 leader_election.cc:290] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:16.665104 10028 raft_consensus.cc:359] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.684299 10030 consensus_queue.cc:260] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:16.675956 10033 tablet_bootstrap.cc:492] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:16.692299 10033 ts_tablet_manager.cc:1403] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.220s	user 0.023s	sys 0.008s
I20260709 13:05:16.691908 10028 raft_consensus.cc:385] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:16.692757 10028 raft_consensus.cc:740] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:16.695616 10028 consensus_queue.cc:260] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.696167  9873 leader_election.cc:304] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 
I20260709 13:05:16.697062 10049 raft_consensus.cc:2804] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:16.697495 10049 raft_consensus.cc:493] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.697880 10049 raft_consensus.cc:3060] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:16.698040 10028 ts_tablet_manager.cc:1434] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.057s	user 0.005s	sys 0.000s
I20260709 13:05:16.698578 10033 raft_consensus.cc:359] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:16.699354 10028 tablet_bootstrap.cc:492] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:16.699328 10033 raft_consensus.cc:385] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:16.699798 10033 raft_consensus.cc:740] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:16.700866 10033 consensus_queue.cc:260] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:16.704329 10033 ts_tablet_manager.cc:1434] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:05:16.705379 10033 tablet_bootstrap.cc:492] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:16.710769  9757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1064d1c0e3ba4ab68fb20fd5f093abef" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:16.716392 10299 raft_consensus.cc:493] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.716974 10299 raft_consensus.cc:515] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:16.704368  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01c76363b154905982dc75dbdead3be" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:16.718339  9908 raft_consensus.cc:2468] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:16.720275 10299 leader_election.cc:290] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:16.721472  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:16.722404  9908 raft_consensus.cc:2393] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23cbd6234c5e43d89f22c235ed6bb96a in current term 1: Already voted for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in this term.
I20260709 13:05:16.718166  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:16.723881  9978 raft_consensus.cc:2468] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:16.727806  9799 leader_election.cc:304] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:16.729043 10029 tablet_bootstrap.cc:492] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:16.729635 10029 ts_tablet_manager.cc:1403] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.352s	user 0.015s	sys 0.017s
I20260709 13:05:16.729502  9757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:16.731206 10250 raft_consensus.cc:2804] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:16.731420  9946 leader_election.cc:304] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:16.731695 10250 raft_consensus.cc:493] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.732776 10250 raft_consensus.cc:3060] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:16.732920 10029 raft_consensus.cc:359] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:16.734378 10299 raft_consensus.cc:3060] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:16.731448  9872 leader_election.cc:343] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:16.733821 10029 raft_consensus.cc:385] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:16.736964 10030 ts_tablet_manager.cc:1434] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.129s	user 0.001s	sys 0.007s
I20260709 13:05:16.737768 10029 raft_consensus.cc:740] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:16.739038 10030 tablet_bootstrap.cc:492] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:16.738714 10029 consensus_queue.cc:260] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:16.751348 10028 tablet_bootstrap.cc:654] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:16.710803  9760 raft_consensus.cc:2468] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:16.760222  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d7bb9a0b174b168df9040b02c70ab9" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:16.765098 10029 ts_tablet_manager.cc:1434] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.001s
I20260709 13:05:16.766135 10029 tablet_bootstrap.cc:492] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:16.767951  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:16.768179 10072 leader_election.cc:290] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:16.769071 10250 raft_consensus.cc:515] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.770167  9835 raft_consensus.cc:2393] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e10aab26c644389b8161974dcabc5d0 in current term 2: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
I20260709 13:05:16.772071 10033 tablet_bootstrap.cc:654] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:16.776221 10250 leader_election.cc:290] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:16.775444  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d7bb9a0b174b168df9040b02c70ab9" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:16.779654 10030 tablet_bootstrap.cc:654] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:16.781164  9949 leader_election.cc:304] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:16.782054 10072 raft_consensus.cc:2749] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:16.787596  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:16.793516  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:16.795872 10029 tablet_bootstrap.cc:654] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:16.790046 10049 raft_consensus.cc:515] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:16.798652  9906 raft_consensus.cc:2393] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a1cbca39e34743d8a40816c3bc5f2030 in current term 2: Already voted for candidate 6e10aab26c644389b8161974dcabc5d0 in this term.
I20260709 13:05:16.799225 10033 tablet_bootstrap.cc:492] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:16.800676 10033 ts_tablet_manager.cc:1403] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.095s	user 0.021s	sys 0.006s
I20260709 13:05:16.801528 10049 leader_election.cc:290] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:16.805743 10030 tablet_bootstrap.cc:492] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:16.806288 10030 ts_tablet_manager.cc:1403] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.003s
I20260709 13:05:16.807341 10299 raft_consensus.cc:2749] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:16.892413 10033 raft_consensus.cc:359] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:16.894979 10033 raft_consensus.cc:385] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:16.895377 10033 raft_consensus.cc:740] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:16.896512 10029 tablet_bootstrap.cc:492] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:16.896997  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:16.897957 10029 ts_tablet_manager.cc:1403] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.132s	user 0.009s	sys 0.012s
I20260709 13:05:16.898267  9978 raft_consensus.cc:3060] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:16.896704 10033 consensus_queue.cc:260] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:16.901800  9946 leader_election.cc:304] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:16.902823 10028 tablet_bootstrap.cc:492] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:16.909845  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:16.916620 10197 consensus_queue.cc:1048] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:16.935633 10299 raft_consensus.cc:2804] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:16.936596 10299 raft_consensus.cc:697] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:16.938374 10033 ts_tablet_manager.cc:1434] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.137s	user 0.010s	sys 0.001s
I20260709 13:05:16.947367 10028 ts_tablet_manager.cc:1403] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.248s	user 0.009s	sys 0.016s
I20260709 13:05:16.952792 10033 tablet_bootstrap.cc:492] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:16.955523  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1064d1c0e3ba4ab68fb20fd5f093abef" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:16.956511  9834 raft_consensus.cc:2468] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:16.960342  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:16.962215  9831 raft_consensus.cc:2468] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:16.963256  9835 raft_consensus.cc:2393] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e10aab26c644389b8161974dcabc5d0 in current term 2: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
I20260709 13:05:16.980332 10262 raft_consensus.cc:493] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.981129 10262 raft_consensus.cc:515] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:16.987351 10049 consensus_queue.cc:1048] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:16.995743 10029 raft_consensus.cc:359] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:16.996717 10029 raft_consensus.cc:385] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:16.997061 10029 raft_consensus.cc:740] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:16.997941 10029 consensus_queue.cc:260] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:16.992730 10028 raft_consensus.cc:359] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:16.999296 10028 raft_consensus.cc:385] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:16.999655 10028 raft_consensus.cc:740] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.004001  9874 leader_election.cc:304] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:17.007501 10147 raft_consensus.cc:2804] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.008122 10147 raft_consensus.cc:493] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.008608 10147 raft_consensus.cc:3060] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.011538  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:17.013028  9872 consensus_peers.cc:597] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:17.014443  9724 leader_election.cc:343] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.016552 10262 leader_election.cc:290] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:16.972157  9801 leader_election.cc:304] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:16.938665 10299 consensus_queue.cc:237] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
W20260709 13:05:17.023530  9801 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45019, user_credentials={real_user=slave}} blocked reactor thread for 51693.8us
I20260709 13:05:17.027168 10250 raft_consensus.cc:2804] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.027805 10250 raft_consensus.cc:493] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.028182 10250 raft_consensus.cc:3060] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:16.968883 10033 tablet_bootstrap.cc:654] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.000411 10028 consensus_queue.cc:260] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.009341  9654 catalog_manager.cc:5697] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.041114  9874 leader_election.cc:304] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:17.042336 10283 raft_consensus.cc:2749] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:16.989928 10030 raft_consensus.cc:359] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.048144 10030 raft_consensus.cc:385] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.048427 10030 raft_consensus.cc:740] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.018203  9978 raft_consensus.cc:2468] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 2.
I20260709 13:05:17.057991 10197 raft_consensus.cc:493] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:17.058635  9726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45019, user_credentials={real_user=slave}} blocked reactor thread for 51191.6us
I20260709 13:05:17.058775 10197 raft_consensus.cc:515] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.063251  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0e147704a7413397055b8d3a812a8e" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
W20260709 13:05:17.065219  9798 leader_election.cc:343] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.075161 10029 ts_tablet_manager.cc:1434] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.177s	user 0.000s	sys 0.013s
I20260709 13:05:17.076968 10029 tablet_bootstrap.cc:492] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:17.078233  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:17.079708  9835 raft_consensus.cc:2468] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:17.067689 10197 leader_election.cc:290] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:17.088209 10030 consensus_queue.cc:260] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.108476  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0e147704a7413397055b8d3a812a8e" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:17.109460 10147 raft_consensus.cc:515] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:17.113507 10147 leader_election.cc:290] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:17.117049  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1064d1c0e3ba4ab68fb20fd5f093abef" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
W20260709 13:05:17.119854  9872 leader_election.cc:343] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.120942 10028 ts_tablet_manager.cc:1434] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.173s	user 0.005s	sys 0.005s
I20260709 13:05:17.122568 10028 tablet_bootstrap.cc:492] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:17.128865 10030 ts_tablet_manager.cc:1434] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.322s	user 0.003s	sys 0.006s
I20260709 13:05:17.130318 10030 tablet_bootstrap.cc:492] T e01c76363b154905982dc75dbdead3be P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:17.141912 10029 tablet_bootstrap.cc:654] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.131809  9799 leader_election.cc:304] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 6e10aab26c644389b8161974dcabc5d0
W20260709 13:05:17.143921  9706 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:17.144204 10033 tablet_bootstrap.cc:492] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:17.149729 10033 ts_tablet_manager.cc:1403] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.198s	user 0.009s	sys 0.021s
I20260709 13:05:17.153373 10235 raft_consensus.cc:2804] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:17.154001 10235 raft_consensus.cc:697] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:17.154150 10030 tablet_bootstrap.cc:654] T e01c76363b154905982dc75dbdead3be P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.155046 10235 consensus_queue.cc:237] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.162899 10028 tablet_bootstrap.cc:654] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.163630 10250 raft_consensus.cc:515] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.175539  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01c76363b154905982dc75dbdead3be" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:17.176369  9906 raft_consensus.cc:3060] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.176040 10033 raft_consensus.cc:359] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.177083 10033 raft_consensus.cc:385] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.177448 10033 raft_consensus.cc:740] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.179679 10250 leader_election.cc:290] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:17.178367 10033 consensus_queue.cc:260] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.187649 10033 ts_tablet_manager.cc:1434] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 13:05:17.189208 10033 tablet_bootstrap.cc:492] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:17.189438 10072 raft_consensus.cc:493] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.189982 10072 raft_consensus.cc:515] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.191046 10147 raft_consensus.cc:493] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.191746 10147 raft_consensus.cc:515] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.194752  9906 raft_consensus.cc:2468] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:17.195190  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b228f0642a242db93a7d11c1afab0ea" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:17.196324  9757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f99aa4491c4458bb5ebc03bc0cd813" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:17.198264 10072 leader_election.cc:290] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:17.196659  9801 leader_election.cc:304] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:17.203794 10029 tablet_bootstrap.cc:492] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:17.205866 10029 ts_tablet_manager.cc:1403] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.129s	user 0.009s	sys 0.009s
I20260709 13:05:17.197342 10147 leader_election.cc:290] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
W20260709 13:05:17.202847  9946 leader_election.cc:343] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.201843 10250 raft_consensus.cc:2804] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.216579 10033 tablet_bootstrap.cc:654] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.216861 10250 raft_consensus.cc:697] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:17.200284 10262 raft_consensus.cc:493] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.219309 10262 raft_consensus.cc:515] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:17.221737 10029 raft_consensus.cc:359] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.221643 10262 leader_election.cc:290] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
W20260709 13:05:17.200430  9873 leader_election.cc:343] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.222442 10029 raft_consensus.cc:385] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.218261 10250 consensus_queue.cc:237] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.226424 10028 tablet_bootstrap.cc:492] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:17.226995 10028 ts_tablet_manager.cc:1403] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.105s	user 0.014s	sys 0.005s
I20260709 13:05:17.227788  9655 catalog_manager.cc:5697] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 2, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.223085  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82b4b9bc0c546adac703578be161959" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:17.224545 10029 raft_consensus.cc:740] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.231344 10029 consensus_queue.cc:260] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
W20260709 13:05:17.198542  9799 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 44989.6us
I20260709 13:05:17.213481  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f99aa4491c4458bb5ebc03bc0cd813" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:17.236125  9906 raft_consensus.cc:2468] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 0.
I20260709 13:05:17.237488  9949 leader_election.cc:304] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:17.238405 10072 raft_consensus.cc:2804] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.238801 10072 raft_consensus.cc:493] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.239163 10072 raft_consensus.cc:3060] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.240833  9658 catalog_manager.cc:5697] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 0 to 1, leader changed from <none> to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 1 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.237437 10028 raft_consensus.cc:359] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.243640 10028 raft_consensus.cc:385] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.243916 10028 raft_consensus.cc:740] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
W20260709 13:05:17.251178  9799 leader_election.cc:343] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.252214 10030 tablet_bootstrap.cc:492] T e01c76363b154905982dc75dbdead3be P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:17.252856 10030 ts_tablet_manager.cc:1403] T e01c76363b154905982dc75dbdead3be P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.123s	user 0.018s	sys 0.020s
I20260709 13:05:17.251727  9799 leader_election.cc:304] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:17.256439 10030 raft_consensus.cc:359] T e01c76363b154905982dc75dbdead3be P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.256247  9725 leader_election.cc:304] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:17.255218 10159 raft_consensus.cc:2749] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:17.260785 10285 raft_consensus.cc:2804] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.261227 10285 raft_consensus.cc:493] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.261585 10285 raft_consensus.cc:3060] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.257395 10072 raft_consensus.cc:515] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.267510  9757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f99aa4491c4458bb5ebc03bc0cd813" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
W20260709 13:05:17.270391  9946 leader_election.cc:343] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.247364 10028 consensus_queue.cc:260] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.271231 10038 consensus_queue.cc:1048] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:17.254855 10029 ts_tablet_manager.cc:1434] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.001s
I20260709 13:05:17.275383 10029 tablet_bootstrap.cc:492] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:17.281019  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e01c76363b154905982dc75dbdead3be" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:17.271170  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f99aa4491c4458bb5ebc03bc0cd813" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:17.282722  9906 raft_consensus.cc:3060] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.287665 10038 raft_consensus.cc:493] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.294761 10038 raft_consensus.cc:515] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.298714  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0e147704a7413397055b8d3a812a8e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:17.299114 10262 consensus_queue.cc:1048] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:17.300302  9724 leader_election.cc:343] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.301802 10028 ts_tablet_manager.cc:1434] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.074s	user 0.005s	sys 0.002s
I20260709 13:05:17.302318 10038 leader_election.cc:290] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:17.303220 10028 tablet_bootstrap.cc:492] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:17.304073  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82b4b9bc0c546adac703578be161959" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:17.267936 10033 tablet_bootstrap.cc:492] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:17.309478 10033 ts_tablet_manager.cc:1403] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.121s	user 0.012s	sys 0.005s
I20260709 13:05:17.313227 10030 raft_consensus.cc:385] T e01c76363b154905982dc75dbdead3be P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.313750 10030 raft_consensus.cc:740] T e01c76363b154905982dc75dbdead3be P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.311179 10072 leader_election.cc:290] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:17.318851 10029 tablet_bootstrap.cc:654] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.325017  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0e147704a7413397055b8d3a812a8e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:17.326038  9831 raft_consensus.cc:2468] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:17.326603 10033 raft_consensus.cc:359] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.327488 10033 raft_consensus.cc:385] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.327764 10033 raft_consensus.cc:740] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.328442 10033 consensus_queue.cc:260] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.339264 10033 ts_tablet_manager.cc:1434] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:05:17.339159  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1064d1c0e3ba4ab68fb20fd5f093abef" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:17.341672 10033 tablet_bootstrap.cc:492] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:17.343566  9833 raft_consensus.cc:3060] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.353008 10250 raft_consensus.cc:493] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.353860 10250 raft_consensus.cc:515] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.354771 10033 tablet_bootstrap.cc:654] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.358899  9757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6f94cfb8a734a83872ed245aeea11e6" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
W20260709 13:05:17.361100  9798 leader_election.cc:343] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.361954 10250 leader_election.cc:290] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:17.363090 10028 tablet_bootstrap.cc:654] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.367053  9757 raft_consensus.cc:3060] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.373169  9654 catalog_manager.cc:5697] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.392341  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b228f0642a242db93a7d11c1afab0ea" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:17.399770  9906 raft_consensus.cc:2468] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:17.400926 10030 consensus_queue.cc:260] T e01c76363b154905982dc75dbdead3be P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.401939 10285 raft_consensus.cc:515] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.409997  9978 raft_consensus.cc:3060] T e01c76363b154905982dc75dbdead3be P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.413784  9725 leader_election.cc:304] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:17.417248  9725 leader_election.cc:304] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:17.418203 10262 raft_consensus.cc:2804] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.418635 10262 raft_consensus.cc:493] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.419080 10262 raft_consensus.cc:3060] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.419137 10285 leader_election.cc:290] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
W20260709 13:05:17.421955  9874 leader_election.cc:343] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.422566  9874 leader_election.cc:304] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:17.423919 10049 raft_consensus.cc:2749] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:17.426265  9833 raft_consensus.cc:2468] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:17.426698 10049 raft_consensus.cc:493] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.427277 10049 raft_consensus.cc:515] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.431491  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b799cecf522841009ed9a2b1b421be50" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:17.431608  9978 raft_consensus.cc:2468] T e01c76363b154905982dc75dbdead3be P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
W20260709 13:05:17.433301  9873 leader_election.cc:343] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.434412 10049 leader_election.cc:290] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:17.434445  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:17.436018  9834 raft_consensus.cc:3060] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.437902  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b799cecf522841009ed9a2b1b421be50" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:17.438582  9831 raft_consensus.cc:2468] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:17.444440  9874 leader_election.cc:304] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:17.454171 10028 tablet_bootstrap.cc:492] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:17.454622 10147 raft_consensus.cc:2804] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.454761 10028 ts_tablet_manager.cc:1403] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.152s	user 0.015s	sys 0.004s
I20260709 13:05:17.455202 10147 raft_consensus.cc:493] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.455668 10147 raft_consensus.cc:3060] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.455498  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:17.454793  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6f94cfb8a734a83872ed245aeea11e6" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:17.457319 10030 ts_tablet_manager.cc:1434] T e01c76363b154905982dc75dbdead3be P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.204s	user 0.006s	sys 0.002s
I20260709 13:05:17.458103 10028 raft_consensus.cc:359] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:17.464275 10028 raft_consensus.cc:385] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.464522  9874 leader_election.cc:304] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:17.464598 10028 raft_consensus.cc:740] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.453625  9949 leader_election.cc:304] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:17.466830 10049 raft_consensus.cc:2804] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.468822 10299 raft_consensus.cc:2804] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.469478 10299 raft_consensus.cc:697] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:17.462589 10029 tablet_bootstrap.cc:492] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:17.469779 10049 raft_consensus.cc:697] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:17.470541 10029 ts_tablet_manager.cc:1403] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.196s	user 0.016s	sys 0.016s
I20260709 13:05:17.470741 10299 consensus_queue.cc:237] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.471186 10049 consensus_queue.cc:237] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:17.474241 10072 consensus_queue.cc:1048] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:17.469750 10028 consensus_queue.cc:260] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
W20260709 13:05:17.477680  9724 leader_election.cc:343] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.479475 10028 ts_tablet_manager.cc:1434] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.005s
I20260709 13:05:17.462417 10033 tablet_bootstrap.cc:492] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:17.480587 10033 ts_tablet_manager.cc:1403] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.139s	user 0.018s	sys 0.009s
I20260709 13:05:17.481475 10028 tablet_bootstrap.cc:492] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
W20260709 13:05:17.482275  9799 leader_election.cc:343] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.487187  9799 leader_election.cc:304] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:17.488679 10159 raft_consensus.cc:2749] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:17.460579 10030 tablet_bootstrap.cc:492] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:17.504550 10147 raft_consensus.cc:515] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.506230 10029 raft_consensus.cc:359] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.518522 10029 raft_consensus.cc:385] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.518900 10029 raft_consensus.cc:740] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.469341 10262 raft_consensus.cc:515] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.506639 10033 raft_consensus.cc:359] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.519874 10033 raft_consensus.cc:385] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.520205 10033 raft_consensus.cc:740] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.521003 10033 consensus_queue.cc:260] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.522543  9834 raft_consensus.cc:2468] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:17.523995 10033 ts_tablet_manager.cc:1434] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.043s	user 0.004s	sys 0.003s
I20260709 13:05:17.528452 10033 tablet_bootstrap.cc:492] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:17.528903 10262 leader_election.cc:290] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:17.529907  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b799cecf522841009ed9a2b1b421be50" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:17.523734 10029 consensus_queue.cc:260] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.513588 10028 tablet_bootstrap.cc:654] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.535298 10147 leader_election.cc:290] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:17.524839  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b799cecf522841009ed9a2b1b421be50" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:17.539142  9831 raft_consensus.cc:3060] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.543511 10038 raft_consensus.cc:2749] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:17.552420  9658 catalog_manager.cc:5697] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.556780  9725 leader_election.cc:304] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:17.559031 10038 raft_consensus.cc:2804] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.559804 10038 raft_consensus.cc:697] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:17.560861 10038 consensus_queue.cc:237] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.561400 10033 tablet_bootstrap.cc:654] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.559746  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0e147704a7413397055b8d3a812a8e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:17.567502  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0e147704a7413397055b8d3a812a8e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:17.568290  9834 raft_consensus.cc:3060] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:17.573117  9724 leader_election.cc:343] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.574131 10030 tablet_bootstrap.cc:654] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:17.576894  9873 leader_election.cc:343] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.538978 10029 ts_tablet_manager.cc:1434] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.002s
I20260709 13:05:17.579510 10029 tablet_bootstrap.cc:492] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:17.588776 10028 tablet_bootstrap.cc:492] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:17.589412 10028 ts_tablet_manager.cc:1403] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.108s	user 0.012s	sys 0.010s
I20260709 13:05:17.592404 10029 tablet_bootstrap.cc:654] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.597291 10030 tablet_bootstrap.cc:492] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:17.597915 10030 ts_tablet_manager.cc:1403] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.140s	user 0.012s	sys 0.001s
I20260709 13:05:17.597985 10033 tablet_bootstrap.cc:492] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:17.598528 10033 ts_tablet_manager.cc:1403] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.070s	user 0.013s	sys 0.000s
W20260709 13:05:17.603767  9874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 83220us
I20260709 13:05:17.591102  9654 catalog_manager.cc:5697] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 0 to 1, leader changed from <none> to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 1 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.609627  9831 raft_consensus.cc:2468] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:17.613070 10033 raft_consensus.cc:359] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:17.613849 10033 raft_consensus.cc:385] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.614157 10033 raft_consensus.cc:740] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.614825 10033 consensus_queue.cc:260] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:17.617691 10033 ts_tablet_manager.cc:1434] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:05:17.617554 10030 raft_consensus.cc:359] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.618501 10030 raft_consensus.cc:385] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.620100 10328 consensus_queue.cc:1048] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:17.621935 10313 raft_consensus.cc:493] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.622689 10313 raft_consensus.cc:515] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:17.625285 10028 raft_consensus.cc:359] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.625835 10049 raft_consensus.cc:493] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.626160 10028 raft_consensus.cc:385] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.626570 10028 raft_consensus.cc:740] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.626508 10049 raft_consensus.cc:515] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.627516  9757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "748a545c945244afa32bec8019157d0c" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:17.627813 10033 tablet_bootstrap.cc:492] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:17.627604 10028 consensus_queue.cc:260] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
W20260709 13:05:17.629240  9798 leader_election.cc:343] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.630383 10028 ts_tablet_manager.cc:1434] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 13:05:17.631318  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:17.630395 10049 leader_election.cc:290] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
W20260709 13:05:17.632938  9873 leader_election.cc:343] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.633530 10028 tablet_bootstrap.cc:492] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:17.626746 10313 leader_election.cc:290] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:17.634879 10029 tablet_bootstrap.cc:492] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:17.635375 10029 ts_tablet_manager.cc:1403] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.056s	user 0.015s	sys 0.004s
I20260709 13:05:17.639523  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
W20260709 13:05:17.640868  9872 leader_election.cc:343] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.641331  9872 leader_election.cc:304] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:17.642022 10033 tablet_bootstrap.cc:654] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.643297 10029 raft_consensus.cc:359] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.644033 10029 raft_consensus.cc:385] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.644307 10029 raft_consensus.cc:740] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.644387 10147 raft_consensus.cc:2749] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:17.647058 10029 consensus_queue.cc:260] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.618861 10030 raft_consensus.cc:740] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.649523 10029 ts_tablet_manager.cc:1434] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:05:17.649950 10030 consensus_queue.cc:260] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.650854 10029 tablet_bootstrap.cc:492] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:17.652264 10030 ts_tablet_manager.cc:1434] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.000s
I20260709 13:05:17.653398 10030 tablet_bootstrap.cc:492] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:17.646070  9874 leader_election.cc:304] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:17.660027 10116 raft_consensus.cc:2804] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.660605 10116 raft_consensus.cc:697] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:17.661509 10116 consensus_queue.cc:237] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.663456 10029 tablet_bootstrap.cc:654] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.667886 10033 tablet_bootstrap.cc:492] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:17.668380 10033 ts_tablet_manager.cc:1403] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.000s
I20260709 13:05:17.674046 10033 raft_consensus.cc:359] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.674769 10033 raft_consensus.cc:385] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.675176 10028 tablet_bootstrap.cc:654] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.675329 10033 raft_consensus.cc:740] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.676095 10033 consensus_queue.cc:260] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.677088 10038 raft_consensus.cc:493] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.677580 10038 raft_consensus.cc:515] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.678202  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "748a545c945244afa32bec8019157d0c" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:17.678881  9909 raft_consensus.cc:2468] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:17.679255 10030 tablet_bootstrap.cc:654] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.681102 10029 tablet_bootstrap.cc:492] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:17.682963  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "168ffc86a17d47e8aba438779717eab6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:17.683671  9909 raft_consensus.cc:2468] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:17.687047  9726 leader_election.cc:304] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:17.687714  9834 raft_consensus.cc:2468] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:17.688206 10285 raft_consensus.cc:2804] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.688757 10285 raft_consensus.cc:493] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.689131 10285 raft_consensus.cc:3060] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.692680 10030 tablet_bootstrap.cc:492] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:17.693241 10030 ts_tablet_manager.cc:1403] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.001s
I20260709 13:05:17.694240  9725 leader_election.cc:304] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:17.674592 10147 raft_consensus.cc:493] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.695473 10262 raft_consensus.cc:2804] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.695364 10147 raft_consensus.cc:515] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.696026 10262 raft_consensus.cc:697] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:17.696187 10030 raft_consensus.cc:359] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.696879 10030 raft_consensus.cc:385] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.697238 10030 raft_consensus.cc:740] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.696959 10262 consensus_queue.cc:237] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.697942 10030 consensus_queue.cc:260] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.698866  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46065991d847dfa3d934961da0830a" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:17.700218  9873 leader_election.cc:343] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.705260 10033 ts_tablet_manager.cc:1434] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 13:05:17.701327 10147 leader_election.cc:290] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:17.705770  9654 catalog_manager.cc:5697] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.709028 10028 tablet_bootstrap.cc:492] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:17.709571 10028 ts_tablet_manager.cc:1403] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.076s	user 0.004s	sys 0.008s
I20260709 13:05:17.711062  9801 leader_election.cc:304] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:17.712436 10313 raft_consensus.cc:2804] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:17.712469 10028 raft_consensus.cc:359] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.712975  9757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46065991d847dfa3d934961da0830a" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:17.713416 10028 raft_consensus.cc:385] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.713765 10028 raft_consensus.cc:740] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.713097 10313 raft_consensus.cc:493] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.714277 10313 raft_consensus.cc:3060] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:17.714469  9872 leader_election.cc:343] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.715047  9872 leader_election.cc:304] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:17.717806 10029 ts_tablet_manager.cc:1403] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.067s	user 0.007s	sys 0.005s
I20260709 13:05:17.717895 10049 raft_consensus.cc:2749] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:17.706568 10033 tablet_bootstrap.cc:492] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:17.723421 10159 consensus_queue.cc:1048] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:17.725023 10028 consensus_queue.cc:260] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.727753 10029 raft_consensus.cc:359] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:17.728834 10029 raft_consensus.cc:385] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.729218 10029 raft_consensus.cc:740] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.732354 10033 tablet_bootstrap.cc:654] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.732897 10082 raft_consensus.cc:493] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.733422 10082 raft_consensus.cc:515] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.735313 10030 ts_tablet_manager.cc:1434] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.000s
I20260709 13:05:17.736493 10030 tablet_bootstrap.cc:492] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:17.736541 10082 leader_election.cc:290] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:17.727888  9981 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "168ffc86a17d47e8aba438779717eab6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:17.737862  9981 raft_consensus.cc:2468] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:17.737701  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f99aa4491c4458bb5ebc03bc0cd813" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:17.738481  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f99aa4491c4458bb5ebc03bc0cd813" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:17.726567 10038 leader_election.cc:290] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:17.742373  9906 raft_consensus.cc:2393] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 1: Already voted for candidate 23cbd6234c5e43d89f22c235ed6bb96a in this term.
I20260709 13:05:17.714525 10285 raft_consensus.cc:515] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.745144 10029 consensus_queue.cc:260] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:17.747376 10028 ts_tablet_manager.cc:1434] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.001s
I20260709 13:05:17.748728  9726 leader_election.cc:304] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0
I20260709 13:05:17.749447 10028 tablet_bootstrap.cc:492] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:17.753392 10313 raft_consensus.cc:515] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:17.756567  9656 catalog_manager.cc:5697] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.759527  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "168ffc86a17d47e8aba438779717eab6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:17.760236  9978 raft_consensus.cc:3060] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.760944  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "168ffc86a17d47e8aba438779717eab6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:17.761566  9906 raft_consensus.cc:3060] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.763063 10285 leader_election.cc:290] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:17.756753 10262 raft_consensus.cc:3060] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.764106  9907 raft_consensus.cc:1275] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer a1cbca39e34743d8a40816c3bc5f2030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:17.766686 10235 consensus_queue.cc:1048] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:17.766872 10030 tablet_bootstrap.cc:654] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.775812 10028 tablet_bootstrap.cc:654] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.784130 10029 ts_tablet_manager.cc:1434] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.000s
I20260709 13:05:17.785132  9982 raft_consensus.cc:1275] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Refusing update from remote peer a1cbca39e34743d8a40816c3bc5f2030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:17.788154 10159 consensus_queue.cc:1048] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:17.792539 10262 raft_consensus.cc:2749] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:17.793147 10313 leader_election.cc:290] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:17.794134  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "748a545c945244afa32bec8019157d0c" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
W20260709 13:05:17.795903  9798 leader_election.cc:343] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:17.796283 10033 tablet_bootstrap.cc:492] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:17.796792 10033 ts_tablet_manager.cc:1403] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.091s	user 0.016s	sys 0.006s
I20260709 13:05:17.797035 10029 tablet_bootstrap.cc:492] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:17.799857  9906 raft_consensus.cc:2468] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:17.801568  9726 leader_election.cc:304] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:17.802475 10038 raft_consensus.cc:2804] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:17.803077 10038 raft_consensus.cc:697] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:17.803964 10038 consensus_queue.cc:237] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.815119 10159 consensus_queue.cc:1048] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:17.816349  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "748a545c945244afa32bec8019157d0c" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:17.818516  9906 raft_consensus.cc:3060] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:17.894555 10350 log.cc:927] Time spent T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:05:17.904348 10029 tablet_bootstrap.cc:654] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.906491  9978 raft_consensus.cc:2468] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:17.912822 10033 raft_consensus.cc:359] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.922891 10033 raft_consensus.cc:385] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.921810 10030 tablet_bootstrap.cc:492] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:17.923352 10033 raft_consensus.cc:740] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.924026 10030 ts_tablet_manager.cc:1403] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.188s	user 0.021s	sys 0.029s
I20260709 13:05:17.918668  9655 catalog_manager.cc:5697] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.924396 10033 consensus_queue.cc:260] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.927358 10030 raft_consensus.cc:359] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.928274 10030 raft_consensus.cc:385] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.928633 10030 raft_consensus.cc:740] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.929402 10030 consensus_queue.cc:260] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.931536 10072 raft_consensus.cc:493] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.931828 10030 ts_tablet_manager.cc:1434] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:05:17.932081 10072 raft_consensus.cc:515] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:17.933074 10030 tablet_bootstrap.cc:492] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:17.934458 10072 leader_election.cc:290] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:17.935367 10033 ts_tablet_manager.cc:1434] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.138s	user 0.001s	sys 0.016s
W20260709 13:05:17.913631  9726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45019, user_credentials={real_user=slave}} blocked reactor thread for 90626.6us
I20260709 13:05:17.944427 10033 tablet_bootstrap.cc:492] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:17.952916 10250 raft_consensus.cc:493] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.953140  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:17.954115  9831 raft_consensus.cc:2393] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23cbd6234c5e43d89f22c235ed6bb96a in current term 1: Already voted for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in this term.
I20260709 13:05:17.953641 10250 raft_consensus.cc:515] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:17.956754 10250 leader_election.cc:290] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:17.960929  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:17.971668  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b228f0642a242db93a7d11c1afab0ea" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:17.985854  9980 raft_consensus.cc:2468] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:17.968642 10028 tablet_bootstrap.cc:492] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:17.987485 10028 ts_tablet_manager.cc:1403] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.238s	user 0.027s	sys 0.020s
I20260709 13:05:17.978490  9654 catalog_manager.cc:5697] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.990666 10028 raft_consensus.cc:359] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.991521 10028 raft_consensus.cc:385] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:17.991835 10028 raft_consensus.cc:740] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:17.992573 10028 consensus_queue.cc:260] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:17.994380 10028 ts_tablet_manager.cc:1434] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:05:17.996025 10028 tablet_bootstrap.cc:492] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:17.996948 10033 tablet_bootstrap.cc:654] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:17.978807  9946 leader_election.cc:304] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:17.999549 10030 tablet_bootstrap.cc:654] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.000185  9757 raft_consensus.cc:3060] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:17.999995 10072 raft_consensus.cc:3060] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.006633 10028 tablet_bootstrap.cc:654] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.008698 10072 raft_consensus.cc:2749] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:17.977587  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b228f0642a242db93a7d11c1afab0ea" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:18.010423  9907 raft_consensus.cc:2468] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:17.981825  9906 raft_consensus.cc:2468] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:18.012897 10049 consensus_queue.cc:1048] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.020040 10328 consensus_queue.cc:1048] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:18.020741  9799 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 47072us
I20260709 13:05:18.022859  9799 leader_election.cc:304] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:18.026750 10033 tablet_bootstrap.cc:492] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:18.027343 10033 ts_tablet_manager.cc:1403] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.083s	user 0.012s	sys 0.002s
I20260709 13:05:18.027455 10313 raft_consensus.cc:2804] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:18.028028 10313 raft_consensus.cc:493] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.028388 10313 raft_consensus.cc:3060] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.030586 10033 raft_consensus.cc:359] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:18.031381 10033 raft_consensus.cc:385] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.031648 10033 raft_consensus.cc:740] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.037091 10033 consensus_queue.cc:260] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:18.049170 10033 ts_tablet_manager.cc:1434] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 13:05:18.051537 10033 tablet_bootstrap.cc:492] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:18.063230 10033 tablet_bootstrap.cc:654] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.066391 10030 tablet_bootstrap.cc:492] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:18.067132 10030 ts_tablet_manager.cc:1403] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.134s	user 0.022s	sys 0.008s
I20260709 13:05:18.072670 10313 raft_consensus.cc:515] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
W20260709 13:05:18.071448  9946 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 92684.3us
I20260709 13:05:18.074937  9801 leader_election.cc:304] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:18.076136 10030 raft_consensus.cc:359] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:18.076357 10082 consensus_queue.cc:1048] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:18.073236 10029 tablet_bootstrap.cc:492] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:18.084201 10030 raft_consensus.cc:385] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.085152 10029 ts_tablet_manager.cc:1403] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.288s	user 0.011s	sys 0.009s
I20260709 13:05:18.085282 10030 raft_consensus.cc:740] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.090415 10028 tablet_bootstrap.cc:492] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:18.091960 10028 ts_tablet_manager.cc:1403] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.096s	user 0.015s	sys 0.004s
I20260709 13:05:18.083537 10159 raft_consensus.cc:2804] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:18.094821 10030 consensus_queue.cc:260] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:18.103175 10030 ts_tablet_manager.cc:1434] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.003s
I20260709 13:05:18.105365  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b228f0642a242db93a7d11c1afab0ea" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:18.113113  9980 raft_consensus.cc:3060] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.105152 10159 raft_consensus.cc:697] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:18.117230 10159 consensus_queue.cc:237] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:18.120201 10283 consensus_queue.cc:1048] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.106020 10030 tablet_bootstrap.cc:492] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:18.122853 10283 raft_consensus.cc:493] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.109437 10029 raft_consensus.cc:359] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:18.127748 10283 raft_consensus.cc:515] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:18.128460 10029 raft_consensus.cc:385] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.128937 10029 raft_consensus.cc:740] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.130088 10029 consensus_queue.cc:260] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:18.134194 10029 ts_tablet_manager.cc:1434] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.001s
I20260709 13:05:18.142323 10028 raft_consensus.cc:359] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:18.143708 10028 raft_consensus.cc:385] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.144176 10028 raft_consensus.cc:740] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.104269 10313 leader_election.cc:290] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
W20260709 13:05:18.140342  9946 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 45614.1us
I20260709 13:05:18.155835 10028 consensus_queue.cc:260] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:18.143208 10072 consensus_queue.cc:1048] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:05:18.163909  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b228f0642a242db93a7d11c1afab0ea" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:18.164767  9908 raft_consensus.cc:3060] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.167773 10313 raft_consensus.cc:493] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.168398 10313 raft_consensus.cc:515] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:18.163902 10029 tablet_bootstrap.cc:492] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:18.181344 10033 tablet_bootstrap.cc:492] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:18.182004 10033 ts_tablet_manager.cc:1403] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.131s	user 0.038s	sys 0.035s
I20260709 13:05:18.183928  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd54dca51434fcf8e5a473955555bfa" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:18.184741  9831 raft_consensus.cc:2468] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:18.188524 10033 raft_consensus.cc:359] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:18.189357 10033 raft_consensus.cc:385] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.189662 10033 raft_consensus.cc:740] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.190347 10033 consensus_queue.cc:260] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:18.197430 10033 ts_tablet_manager.cc:1434] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.001s
I20260709 13:05:18.198552 10033 tablet_bootstrap.cc:492] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:18.203215  9982 raft_consensus.cc:3060] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.207782 10028 ts_tablet_manager.cc:1434] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.115s	user 0.007s	sys 0.001s
I20260709 13:05:18.228473 10030 tablet_bootstrap.cc:654] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.232475  9874 leader_election.cc:304] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:18.240092 10033 tablet_bootstrap.cc:654] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.241000 10353 raft_consensus.cc:493] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.241557 10353 raft_consensus.cc:515] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:18.244423  9980 raft_consensus.cc:2468] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:18.246443 10313 leader_election.cc:290] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:18.244488 10353 leader_election.cc:290] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:18.247941 10049 raft_consensus.cc:2804] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:18.248353 10049 raft_consensus.cc:493] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.248677 10049 raft_consensus.cc:3060] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.250197  9908 raft_consensus.cc:2468] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:18.254227 10028 tablet_bootstrap.cc:492] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:18.258716 10283 leader_election.cc:290] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:18.263463 10049 raft_consensus.cc:515] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:18.265794  9799 leader_election.cc:304] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:18.269670 10368 raft_consensus.cc:2804] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:18.270386 10368 raft_consensus.cc:697] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:18.271607 10368 consensus_queue.cc:237] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:18.274816 10028 tablet_bootstrap.cc:654] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.290652 10049 leader_election.cc:290] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:18.307780 10029 tablet_bootstrap.cc:654] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.322384  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd54dca51434fcf8e5a473955555bfa" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:18.328266  9977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd54dca51434fcf8e5a473955555bfa" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:18.329705  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a930896e984848a0516f17a1213ab5" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:18.330593  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d69b492d298d42e18ab6b82be92774ca" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:18.331647 10038 consensus_queue.cc:1048] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:18.342660  9873 leader_election.cc:343] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:18.343933  9873 consensus_peers.cc:597] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Couldn't send request to peer 23cbd6234c5e43d89f22c235ed6bb96a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:18.344561  9873 leader_election.cc:343] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:18.345289  9873 leader_election.cc:343] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.345645 10030 tablet_bootstrap.cc:492] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:18.346191 10030 ts_tablet_manager.cc:1403] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.240s	user 0.013s	sys 0.006s
I20260709 13:05:18.351118 10030 raft_consensus.cc:359] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:18.351728 10030 raft_consensus.cc:385] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.351970 10030 raft_consensus.cc:740] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.354161 10030 consensus_queue.cc:260] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:18.359050 10082 raft_consensus.cc:493] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.360250 10082 raft_consensus.cc:515] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:18.363118 10082 leader_election.cc:290] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:18.365026 10030 ts_tablet_manager.cc:1434] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:05:18.366307 10030 tablet_bootstrap.cc:492] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:18.368330 10033 tablet_bootstrap.cc:492] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:18.368966 10033 ts_tablet_manager.cc:1403] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.171s	user 0.034s	sys 0.019s
I20260709 13:05:18.371714  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:18.374711 10285 consensus_queue.cc:1048] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.375133 10033 raft_consensus.cc:359] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:18.376089 10033 raft_consensus.cc:385] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.376451 10033 raft_consensus.cc:740] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.376255 10285 consensus_queue.cc:1048] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.377358 10033 consensus_queue.cc:260] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
W20260709 13:05:18.379441  9724 leader_election.cc:343] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.380164 10033 ts_tablet_manager.cc:1434] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:05:18.384912 10033 tablet_bootstrap.cc:492] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:18.392565 10033 tablet_bootstrap.cc:654] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.421573 10029 tablet_bootstrap.cc:492] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:18.422109 10029 ts_tablet_manager.cc:1403] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.259s	user 0.016s	sys 0.000s
I20260709 13:05:18.426443 10029 raft_consensus.cc:359] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:18.427397 10029 raft_consensus.cc:385] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.427767 10029 raft_consensus.cc:740] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
W20260709 13:05:18.468050  9706 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:18.516172 10368 raft_consensus.cc:493] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.516999 10368 raft_consensus.cc:515] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
W20260709 13:05:18.683894  9798 consensus_peers.cc:597] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:18.796191 10368 leader_election.cc:290] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:18.797351  9752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6bd3b28a03c44fda0c31e4198a86726" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
W20260709 13:05:18.799317  9798 leader_election.cc:343] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.824271  9757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d69b492d298d42e18ab6b82be92774ca" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:18.841114 10376 consensus_queue.cc:1048] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.859555  9832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd54dca51434fcf8e5a473955555bfa" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
W20260709 13:05:18.859840  9799 leader_election.cc:343] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.860431  9832 raft_consensus.cc:3060] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:18.875808  9832 raft_consensus.cc:2468] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
W20260709 13:05:18.881415  9872 leader_election.cc:343] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.881948  9872 leader_election.cc:304] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:18.884233 10049 raft_consensus.cc:493] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.884773 10049 raft_consensus.cc:515] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:18.890969 10378 raft_consensus.cc:2749] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:18.893874 10049 leader_election.cc:290] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
W20260709 13:05:18.898761 10033 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.014s	sys 0.029s
W20260709 13:05:18.899240 10033 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.014s	sys 0.029s
I20260709 13:05:18.909291  9758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a930896e984848a0516f17a1213ab5" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:18.927345 10033 tablet_bootstrap.cc:492] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
W20260709 13:05:18.928310  9798 leader_election.cc:343] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:18.928851  9798 leader_election.cc:304] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:18.930634 10366 raft_consensus.cc:2749] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:18.928869 10033 ts_tablet_manager.cc:1403] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.548s	user 0.028s	sys 0.029s
I20260709 13:05:18.955678 10033 raft_consensus.cc:359] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:18.956398 10033 raft_consensus.cc:385] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:18.956664 10033 raft_consensus.cc:740] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:18.957307 10033 consensus_queue.cc:260] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:18.958864 10033 ts_tablet_manager.cc:1434] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.000s
I20260709 13:05:18.964138 10382 consensus_queue.cc:1048] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.965699  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:18.974742  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912bd6d0efac4042884ce6823f7e8e73" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:18.975651  9834 raft_consensus.cc:2468] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:18.978224 10033 tablet_bootstrap.cc:492] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:18.981503 10030 tablet_bootstrap.cc:654] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.984994 10033 tablet_bootstrap.cc:654] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:18.976056  9906 raft_consensus.cc:2468] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:18.993022 10049 consensus_queue.cc:1048] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:18.993000  9874 leader_election.cc:304] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:18.994804 10147 raft_consensus.cc:2804] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:18.995414 10147 raft_consensus.cc:697] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:18.996261 10147 consensus_queue.cc:237] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:18.990489  9726 leader_election.cc:304] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
W20260709 13:05:18.998538  9801 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.021s	sys 0.044s
W20260709 13:05:18.999105  9801 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.022s	sys 0.045s
I20260709 13:05:18.999218 10082 raft_consensus.cc:2804] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:18.999428  9801 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45019, user_credentials={real_user=slave}} blocked reactor thread for 798067us
I20260709 13:05:18.999747 10082 raft_consensus.cc:493] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.000177 10082 raft_consensus.cc:3060] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:19.002861  9874 leader_election.cc:304] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:19.027943 10049 raft_consensus.cc:2804] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:19.028532 10049 raft_consensus.cc:493] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.028945 10049 raft_consensus.cc:3060] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:19.053289  9874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 50494.3us
I20260709 13:05:19.069931 10299 raft_consensus.cc:493] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.071345 10082 raft_consensus.cc:515] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:19.071774 10299 raft_consensus.cc:515] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:19.074121 10299 leader_election.cc:290] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:19.073144 10082 leader_election.cc:290] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:19.074865  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6f94cfb8a734a83872ed245aeea11e6" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:19.075542 10029 consensus_queue.cc:260] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
W20260709 13:05:19.076543  9946 leader_election.cc:343] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.077152 10029 ts_tablet_manager.cc:1434] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.655s	user 0.014s	sys 0.001s
I20260709 13:05:19.077237 10368 raft_consensus.cc:493] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:19.079723 10368 raft_consensus.cc:515] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:19.077952 10377 raft_consensus.cc:493] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.087388 10377 raft_consensus.cc:515] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:19.092962 10030 tablet_bootstrap.cc:492] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:19.094684 10030 ts_tablet_manager.cc:1403] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.729s	user 0.018s	sys 0.002s
I20260709 13:05:19.086328 10049 raft_consensus.cc:515] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:19.097668 10049 leader_election.cc:290] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
W20260709 13:05:19.077520 10029 ts_tablet_manager.cc:1467] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4: Tablet startup took 655ms
I20260709 13:05:19.105480  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912bd6d0efac4042884ce6823f7e8e73" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:19.106222  9833 raft_consensus.cc:3060] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:19.106647 10368 leader_election.cc:290] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:19.107583  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41e6a20c04a4d99ba9b1ea99106480a" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:19.109977 10030 raft_consensus.cc:359] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:19.110733 10030 raft_consensus.cc:385] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:19.111018 10030 raft_consensus.cc:740] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.111721 10030 consensus_queue.cc:260] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
W20260709 13:05:19.100898 10029 ts_tablet_manager.cc:1469] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4: Trace:
0709 13:05:12.596525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.596685 (+   160us) service_pool.cc:224] Handling call
0709 13:05:12.598284 (+  1599us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:05:12.598366 (+    82us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:05:12.617859 (+ 19493us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:12.631187 (+ 13328us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:12.631714 (+   527us) inbound_call.cc:177] Queueing success response
0709 13:05:18.161573 (+5529859us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:05:18.422398 (+260825us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:05:18.423068 (+   670us) tablet_replica.cc:261] Starting instrumentation
0709 13:05:18.423801 (+   733us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":454,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"tablet-open.queue_time_us":5519139}
I20260709 13:05:19.115605 10029 tablet_bootstrap.cc:492] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:19.116078 10368 consensus_queue.cc:1048] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:19.118582 10030 ts_tablet_manager.cc:1434] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:05:19.119817 10030 tablet_bootstrap.cc:492] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:19.122397  9978 raft_consensus.cc:3060] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:19.122550  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41e6a20c04a4d99ba9b1ea99106480a" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:19.123730 10029 tablet_bootstrap.cc:654] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:19.124804  9830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6f94cfb8a734a83872ed245aeea11e6" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:19.125525  9830 raft_consensus.cc:2468] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 0.
I20260709 13:05:19.125568  9801 leader_election.cc:304] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:19.131203 10313 raft_consensus.cc:2749] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:19.144748 10028 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.005s	sys 0.010s
W20260709 13:05:19.145292 10028 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.005s	sys 0.010s
I20260709 13:05:19.155714  9948 leader_election.cc:304] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
W20260709 13:05:19.165288  9759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.964s	user 0.010s	sys 0.024s
W20260709 13:05:19.165791  9759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.964s	user 0.010s	sys 0.024s
I20260709 13:05:19.167873  9904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6bd3b28a03c44fda0c31e4198a86726" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:19.167014 10033 tablet_bootstrap.cc:492] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:19.170056  9903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:19.170810  9903 raft_consensus.cc:3060] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:19.170989  9975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912bd6d0efac4042884ce6823f7e8e73" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:19.175012 10033 ts_tablet_manager.cc:1403] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.202s	user 0.023s	sys 0.009s
I20260709 13:05:19.175777 10028 tablet_bootstrap.cc:492] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:19.176273 10028 ts_tablet_manager.cc:1403] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.922s	user 0.016s	sys 0.012s
I20260709 13:05:19.178548 10030 tablet_bootstrap.cc:654] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:19.175206 10313 raft_consensus.cc:493] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.180493 10368 consensus_queue.cc:1048] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
W20260709 13:05:19.180888  9801 leader_election.cc:343] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.188859  9801 leader_election.cc:304] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:19.192812 10328 raft_consensus.cc:2804] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:19.193256 10328 raft_consensus.cc:493] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.193594 10328 raft_consensus.cc:3060] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:19.190968 10235 raft_consensus.cc:2749] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:19.184844 10313 raft_consensus.cc:515] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:19.201294 10313 leader_election.cc:290] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
W20260709 13:05:19.206476  9948 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 51010.4us
W20260709 13:05:19.209497  9873 leader_election.cc:343] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.211468  9759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "622d51c8b2124715a814796353745c9b" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
W20260709 13:05:19.214047  9798 leader_election.cc:343] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.213877 10033 raft_consensus.cc:359] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:19.169199  9833 raft_consensus.cc:2468] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:19.214890 10033 raft_consensus.cc:385] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:19.215931  9874 leader_election.cc:304] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:19.217978 10029 tablet_bootstrap.cc:492] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:19.218739 10033 raft_consensus.cc:740] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.219198 10029 ts_tablet_manager.cc:1403] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.104s	user 0.033s	sys 0.023s
I20260709 13:05:19.219758 10033 consensus_queue.cc:260] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:19.190860  9903 raft_consensus.cc:2468] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
W20260709 13:05:19.261101  9874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 45279.3us
I20260709 13:05:19.261320 10354 raft_consensus.cc:2804] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:19.261967 10354 raft_consensus.cc:697] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:19.262807 10354 consensus_queue.cc:237] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:19.271807 10028 raft_consensus.cc:359] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:19.272931 10028 raft_consensus.cc:385] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:19.273290 10028 raft_consensus.cc:740] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.237901  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912bd6d0efac4042884ce6823f7e8e73" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:19.282116 10029 raft_consensus.cc:359] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:19.285579 10029 raft_consensus.cc:385] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:19.286161 10029 raft_consensus.cc:740] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.287194 10029 consensus_queue.cc:260] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:19.230821 10328 raft_consensus.cc:515] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:19.300697 10028 consensus_queue.cc:260] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:19.312809 10029 ts_tablet_manager.cc:1434] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.093s	user 0.010s	sys 0.000s
I20260709 13:05:19.293761  9658 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47620 (request call id 68) took 1099 ms. Trace:
I20260709 13:05:19.328775 10033 ts_tablet_manager.cc:1434] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.153s	user 0.006s	sys 0.002s
I20260709 13:05:19.331483 10030 tablet_bootstrap.cc:492] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:19.332319 10030 ts_tablet_manager.cc:1403] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.213s	user 0.015s	sys 0.015s
I20260709 13:05:19.313896  9658 rpcz_store.cc:276] 0709 13:05:18.194655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.495132 (+300477us) service_pool.cc:224] Handling call
0709 13:05:19.293695 (+798563us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1247,"reported_tablets":1}
I20260709 13:05:19.345322  9759 raft_consensus.cc:3060] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:19.359615 10033 tablet_bootstrap.cc:492] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:19.373096 10030 raft_consensus.cc:359] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:19.373910 10030 raft_consensus.cc:385] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:19.374173 10030 raft_consensus.cc:740] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.379496  9976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "622d51c8b2124715a814796353745c9b" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:19.366211  9873 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 76822.1us
I20260709 13:05:19.394518 10038 raft_consensus.cc:493] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.395396 10038 raft_consensus.cc:515] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:19.398134 10038 leader_election.cc:290] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:19.399084 10366 raft_consensus.cc:493] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.399664 10366 raft_consensus.cc:515] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:19.402765 10033 tablet_bootstrap.cc:654] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:19.424942  9655 catalog_manager.cc:5697] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:19.415443  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6f94cfb8a734a83872ed245aeea11e6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:19.422731  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:19.439546  9753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d8c823d37d4e9a864b46403d267028" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:19.440204  9754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6f94cfb8a734a83872ed245aeea11e6" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:19.441540 10377 leader_election.cc:290] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
W20260709 13:05:19.442389  9856 debug-util.cc:398] Leaking SignalData structure 0x7b08000c5f80 after lost signal to thread 434
I20260709 13:05:19.406262 10072 raft_consensus.cc:493] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e10aab26c644389b8161974dcabc5d0)
I20260709 13:05:19.449468  9726 leader_election.cc:304] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:19.410436 10030 consensus_queue.cc:260] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:19.425549 10328 leader_election.cc:290] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:19.456023  9655 catalog_manager.cc:5697] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:19.469611 10028 ts_tablet_manager.cc:1434] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.293s	user 0.011s	sys 0.021s
I20260709 13:05:19.486358 10029 tablet_bootstrap.cc:492] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:19.476269  9979 raft_consensus.cc:3060] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:19.486112 10377 raft_consensus.cc:2804] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:19.491607 10377 raft_consensus.cc:697] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:19.492475 10377 consensus_queue.cc:237] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:19.495596 10030 ts_tablet_manager.cc:1434] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.163s	user 0.005s	sys 0.003s
I20260709 13:05:19.477957  9754 raft_consensus.cc:3060] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:19.465514  9834 raft_consensus.cc:2468] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:19.484541  9903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "748a545c945244afa32bec8019157d0c" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
W20260709 13:05:19.498461 10394 log.cc:927] Time spent T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 13:05:19.497643  9856 debug-util.cc:398] Leaking SignalData structure 0x7b080020c0c0 after lost signal to thread 9692
I20260709 13:05:19.470366 10072 raft_consensus.cc:515] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
W20260709 13:05:19.521006  9798 leader_election.cc:343] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.494475  9830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6f94cfb8a734a83872ed245aeea11e6" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:19.538522 10030 tablet_bootstrap.cc:492] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:19.536083 10328 raft_consensus.cc:493] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:19.530241  9830 raft_consensus.cc:3060] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:19.543562 10328 raft_consensus.cc:515] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:19.544486  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6f94cfb8a734a83872ed245aeea11e6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:19.551048 10368 consensus_queue.cc:1048] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:19.533916  9725 leader_election.cc:304] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
W20260709 13:05:19.561254  9706 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:19.564702 10029 tablet_bootstrap.cc:654] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:19.552243  9856 debug-util.cc:398] Leaking SignalData structure 0x7b08002ba5a0 after lost signal to thread 9784
I20260709 13:05:19.581657 10328 leader_election.cc:290] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:19.583184  9980 raft_consensus.cc:2393] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 1: Already voted for candidate 23cbd6234c5e43d89f22c235ed6bb96a in this term.
I20260709 13:05:19.585547  9979 raft_consensus.cc:2468] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
W20260709 13:05:19.590812  9725 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 62310.5us
I20260709 13:05:19.594184  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "748a545c945244afa32bec8019157d0c" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:19.602736 10028 tablet_bootstrap.cc:492] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:19.603310 10387 raft_consensus.cc:493] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.603874 10387 raft_consensus.cc:515] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:19.605681 10072 leader_election.cc:290] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:19.606237 10387 leader_election.cc:290] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:19.609707 10028 tablet_bootstrap.cc:654] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:19.609795  9873 leader_election.cc:343] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.610046 10283 raft_consensus.cc:493] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:19.612259  9856 debug-util.cc:398] Leaking SignalData structure 0x7b08001899c0 after lost signal to thread 9859
I20260709 13:05:19.612043 10283 raft_consensus.cc:515] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:19.612792  9755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2684c424702483b92758b1173702381" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
W20260709 13:05:19.613572  9856 debug-util.cc:398] Leaking SignalData structure 0x7b08002aae00 after lost signal to thread 9933
I20260709 13:05:19.614118 10283 leader_election.cc:290] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:19.552129 10366 leader_election.cc:290] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:19.632992 10033 tablet_bootstrap.cc:492] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:19.633687 10033 ts_tablet_manager.cc:1403] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.275s	user 0.028s	sys 0.020s
I20260709 13:05:19.635147  9753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9623952612f4929825a90c043f1c6ba" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
W20260709 13:05:19.639572  9798 leader_election.cc:343] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:19.614877  9856 debug-util.cc:398] Leaking SignalData structure 0x7b08002a04e0 after lost signal to thread 10007
I20260709 13:05:19.656090  9830 raft_consensus.cc:2468] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
W20260709 13:05:19.659271 10403 log.cc:927] Time spent T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:05:19.672686 10033 raft_consensus.cc:359] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:19.702116  9754 raft_consensus.cc:2468] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:19.703122 10376 raft_consensus.cc:2764] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:05:19.673669 10033 raft_consensus.cc:385] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:19.706493 10033 raft_consensus.cc:740] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.707424 10033 consensus_queue.cc:260] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:19.677135  9948 leader_election.cc:304] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:19.683318  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b799cecf522841009ed9a2b1b421be50" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:19.711449 10299 raft_consensus.cc:2804] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:19.712174 10299 raft_consensus.cc:697] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:19.713284 10299 consensus_queue.cc:237] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
W20260709 13:05:19.725069  9872 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 225302us
I20260709 13:05:19.727038 10030 tablet_bootstrap.cc:654] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:19.739604  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9623952612f4929825a90c043f1c6ba" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:19.750093 10033 ts_tablet_manager.cc:1434] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.116s	user 0.008s	sys 0.002s
W20260709 13:05:19.763790  9799 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 200854us
W20260709 13:05:19.765005  9872 leader_election.cc:343] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:19.765065  9799 leader_election.cc:343] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.769097  9799 leader_election.cc:304] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:19.797546  9725 leader_election.cc:304] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:19.799269  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d8c823d37d4e9a864b46403d267028" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:19.801887  9903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b799cecf522841009ed9a2b1b421be50" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:19.805998 10029 tablet_bootstrap.cc:492] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:19.806473 10029 ts_tablet_manager.cc:1403] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.335s	user 0.012s	sys 0.008s
I20260709 13:05:19.806447 10028 tablet_bootstrap.cc:492] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:19.807525 10028 ts_tablet_manager.cc:1403] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.205s	user 0.016s	sys 0.004s
I20260709 13:05:19.805472 10376 raft_consensus.cc:2749] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:19.817631  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2684c424702483b92758b1173702381" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:19.818260 10029 raft_consensus.cc:359] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:19.823902 10029 raft_consensus.cc:385] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:19.824604 10029 raft_consensus.cc:740] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.824920  9949 leader_election.cc:304] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:19.803906 10250 raft_consensus.cc:2749] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:19.843683  9949 leader_election.cc:304] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:19.835366 10028 raft_consensus.cc:359] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:19.846585 10028 raft_consensus.cc:385] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:05:19.812897  9801 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45019, user_credentials={real_user=slave}} blocked reactor thread for 215275us
I20260709 13:05:19.847095 10028 raft_consensus.cc:740] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.844882 10029 consensus_queue.cc:260] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:19.851740 10033 tablet_bootstrap.cc:492] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:19.861306 10354 raft_consensus.cc:493] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.861969 10354 raft_consensus.cc:515] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:19.848807 10072 raft_consensus.cc:2749] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:19.857059 10028 consensus_queue.cc:260] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:19.867419 10028 ts_tablet_manager.cc:1434] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.000s
I20260709 13:05:19.868109 10415 raft_consensus.cc:2749] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:19.868721 10028 tablet_bootstrap.cc:492] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:19.864269 10354 leader_election.cc:290] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
W20260709 13:05:19.869583  9873 consensus_peers.cc:597] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 -> Peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Couldn't send request to peer 23cbd6234c5e43d89f22c235ed6bb96a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:19.857537 10029 ts_tablet_manager.cc:1434] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 13:05:19.880775 10033 tablet_bootstrap.cc:654] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:19.876791  9902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9623952612f4929825a90c043f1c6ba" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:19.890692 10029 tablet_bootstrap.cc:492] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:19.885629  9753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46065991d847dfa3d934961da0830a" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:19.894312  9902 raft_consensus.cc:2468] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:19.897596  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46065991d847dfa3d934961da0830a" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:19.899963  9873 leader_election.cc:343] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.906268 10028 tablet_bootstrap.cc:654] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:19.906576 10030 tablet_bootstrap.cc:492] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:19.907109 10030 ts_tablet_manager.cc:1403] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.384s	user 0.021s	sys 0.020s
W20260709 13:05:19.907018  9872 leader_election.cc:343] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.909881  9872 leader_election.cc:304] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:19.910066 10030 raft_consensus.cc:359] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:19.910795 10030 raft_consensus.cc:385] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:19.910732 10147 raft_consensus.cc:2749] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:19.919181  9830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9623952612f4929825a90c043f1c6ba" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:19.929023  9830 raft_consensus.cc:2468] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:19.915961 10030 raft_consensus.cc:740] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.944612 10049 consensus_queue.cc:1048] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:19.946678  9799 leader_election.cc:343] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:19.947188  9799 leader_election.cc:304] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:19.950263 10030 consensus_queue.cc:260] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:19.942714 10313 raft_consensus.cc:493] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.956425 10049 consensus_queue.cc:1048] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:19.960952 10413 raft_consensus.cc:2749] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:19.961681 10033 tablet_bootstrap.cc:492] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:19.962179 10033 ts_tablet_manager.cc:1403] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.150s	user 0.018s	sys 0.006s
I20260709 13:05:19.959189  9874 leader_election.cc:304] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:19.967280 10354 raft_consensus.cc:2804] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:19.967856 10354 raft_consensus.cc:493] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.968315 10354 raft_consensus.cc:3060] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:19.972518 10030 ts_tablet_manager.cc:1434] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.000s
I20260709 13:05:19.973811 10030 tablet_bootstrap.cc:492] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:19.980278 10030 tablet_bootstrap.cc:654] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:19.987058 10415 raft_consensus.cc:493] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.987641 10415 raft_consensus.cc:515] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:19.988067 10033 raft_consensus.cc:359] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:19.988850 10029 tablet_bootstrap.cc:654] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:19.988993 10033 raft_consensus.cc:385] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:19.994338 10033 raft_consensus.cc:740] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:19.958864 10313 raft_consensus.cc:515] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:19.996243 10033 consensus_queue.cc:260] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:19.993784  9753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a930896e984848a0516f17a1213ab5" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:19.999996  9753 raft_consensus.cc:2468] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 0.
I20260709 13:05:20.003003 10313 leader_election.cc:290] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:20.003976  9946 leader_election.cc:304] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:20.004910 10299 raft_consensus.cc:2804] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:20.005307 10299 raft_consensus.cc:493] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.007099 10299 raft_consensus.cc:3060] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.010546 10415 leader_election.cc:290] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:20.012157 10033 ts_tablet_manager.cc:1434] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.050s	user 0.009s	sys 0.001s
I20260709 13:05:20.011723  9654 catalog_manager.cc:5697] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:20.017541  9654 catalog_manager.cc:5697] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:20.019362  9801 leader_election.cc:304] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:20.020182  9830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a930896e984848a0516f17a1213ab5" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:20.020893  9830 raft_consensus.cc:2468] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 0.
I20260709 13:05:20.021322 10387 raft_consensus.cc:2804] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:20.021916 10387 raft_consensus.cc:493] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.022365 10387 raft_consensus.cc:3060] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.025905  9902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "140e604980a14cfe9b9ee5918dc45bb2" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
W20260709 13:05:20.045490  9801 leader_election.cc:343] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.051620 10354 raft_consensus.cc:515] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.053839 10354 leader_election.cc:290] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:20.054654  9753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9623952612f4929825a90c043f1c6ba" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
W20260709 13:05:20.056217  9872 leader_election.cc:343] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.057768 10028 tablet_bootstrap.cc:492] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:20.058274 10028 ts_tablet_manager.cc:1403] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.190s	user 0.008s	sys 0.021s
I20260709 13:05:20.065907 10377 consensus_queue.cc:1048] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:20.068521 10387 raft_consensus.cc:515] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.070315 10033 tablet_bootstrap.cc:492] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030: Bootstrap starting.
I20260709 13:05:20.093022 10033 tablet_bootstrap.cc:654] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:20.099802  9873 consensus_peers.cc:597] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Couldn't send request to peer 23cbd6234c5e43d89f22c235ed6bb96a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:20.106619 10028 raft_consensus.cc:359] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.109197 10028 raft_consensus.cc:385] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.110277 10028 raft_consensus.cc:740] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.145546  9832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9623952612f4929825a90c043f1c6ba" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:20.145850  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "140e604980a14cfe9b9ee5918dc45bb2" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:20.146507  9832 raft_consensus.cc:2393] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e10aab26c644389b8161974dcabc5d0 in current term 1: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
W20260709 13:05:20.171777  9799 leader_election.cc:343] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.172574  9799 leader_election.cc:304] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0
I20260709 13:05:20.179297  9874 leader_election.cc:304] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:20.180075 10028 consensus_queue.cc:260] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.187428 10029 tablet_bootstrap.cc:492] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:20.188870 10029 ts_tablet_manager.cc:1403] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.304s	user 0.022s	sys 0.020s
I20260709 13:05:20.188336 10283 raft_consensus.cc:2749] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:20.189054 10368 raft_consensus.cc:2749] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:20.224382  9753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9623952612f4929825a90c043f1c6ba" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:20.197026 10299 raft_consensus.cc:515] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:20.227077 10028 ts_tablet_manager.cc:1434] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.169s	user 0.011s	sys 0.001s
W20260709 13:05:20.228229  9798 leader_election.cc:343] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.236660 10033 tablet_bootstrap.cc:492] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030: No bootstrap required, opened a new log
I20260709 13:05:20.237289 10033 ts_tablet_manager.cc:1403] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030: Time spent bootstrapping tablet: real 0.221s	user 0.032s	sys 0.017s
I20260709 13:05:20.240984 10072 consensus_queue.cc:1048] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:20.242906 10030 tablet_bootstrap.cc:492] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:20.243448 10030 ts_tablet_manager.cc:1403] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.270s	user 0.031s	sys 0.027s
I20260709 13:05:20.246444 10033 raft_consensus.cc:359] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:20.247880 10033 raft_consensus.cc:385] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.248236 10033 raft_consensus.cc:740] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.249014 10033 consensus_queue.cc:260] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:20.251959 10082 consensus_queue.cc:1048] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:20.246373 10030 raft_consensus.cc:359] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:20.253758 10033 ts_tablet_manager.cc:1434] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:05:20.253770 10030 raft_consensus.cc:385] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.254808 10387 leader_election.cc:290] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:20.255473 10030 raft_consensus.cc:740] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.256381 10030 consensus_queue.cc:260] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:20.257556  9902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9623952612f4929825a90c043f1c6ba" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:20.258479  9902 raft_consensus.cc:2393] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a1cbca39e34743d8a40816c3bc5f2030 in current term 1: Already voted for candidate 6e10aab26c644389b8161974dcabc5d0 in this term.
I20260709 13:05:20.258603 10030 ts_tablet_manager.cc:1434] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:05:20.259655 10030 tablet_bootstrap.cc:492] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:20.261806  9801 leader_election.cc:304] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:20.265589 10030 tablet_bootstrap.cc:654] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.277531 10429 raft_consensus.cc:2749] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:20.287314 10028 tablet_bootstrap.cc:492] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:20.299562 10299 leader_election.cc:290] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:20.315261 10049 raft_consensus.cc:493] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.316021 10049 raft_consensus.cc:515] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:20.318327 10049 leader_election.cc:290] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:20.320967  9829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a930896e984848a0516f17a1213ab5" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:20.321686  9829 raft_consensus.cc:3060] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.323163  9656 catalog_manager.cc:5697] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:20.323839 10299 consensus_queue.cc:1048] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:20.325724  9758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a930896e984848a0516f17a1213ab5" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:20.326390  9758 raft_consensus.cc:3060] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.329105 10399 raft_consensus.cc:493] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.329999 10028 tablet_bootstrap.cc:654] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.329923 10399 raft_consensus.cc:515] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.332751 10399 leader_election.cc:290] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:20.336009  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a981502782314272a8a9bee8da734427" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:20.336194  9832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6bd3b28a03c44fda0c31e4198a86726" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:20.337109  9832 raft_consensus.cc:2468] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:20.342499  9760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a981502782314272a8a9bee8da734427" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:20.344141  9756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6bd3b28a03c44fda0c31e4198a86726" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:20.346617  9874 leader_election.cc:304] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:20.347724 10408 raft_consensus.cc:2804] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:20.348104 10408 raft_consensus.cc:493] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.348433 10408 raft_consensus.cc:3060] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.349819 10029 raft_consensus.cc:359] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:20.350595 10029 raft_consensus.cc:385] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.351016 10029 raft_consensus.cc:740] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.351693 10029 consensus_queue.cc:260] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:20.353878 10029 ts_tablet_manager.cc:1434] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.165s	user 0.005s	sys 0.001s
W20260709 13:05:20.355824  9873 leader_election.cc:343] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.369438  9829 raft_consensus.cc:2468] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
W20260709 13:05:20.370424  9872 leader_election.cc:343] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.370820  9948 leader_election.cc:304] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:20.371807 10299 raft_consensus.cc:2804] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:20.372318 10299 raft_consensus.cc:697] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:20.373165 10299 consensus_queue.cc:237] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:20.370883  9872 leader_election.cc:304] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:20.376220 10029 tablet_bootstrap.cc:492] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:20.388188 10049 raft_consensus.cc:2749] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:20.388988  9872 leader_election.cc:343] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.400408 10313 raft_consensus.cc:493] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.401386 10313 raft_consensus.cc:515] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.404453  9758 raft_consensus.cc:2468] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:20.499403 10387 raft_consensus.cc:493] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.500161 10387 raft_consensus.cc:515] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:20.470634  9755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecae0626a5c744f0ad9571147ccc239d" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:20.433583 10029 tablet_bootstrap.cc:654] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.453088 10313 leader_election.cc:290] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:20.461812  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecae0626a5c744f0ad9571147ccc239d" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:20.547016 10387 leader_election.cc:290] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:20.558470 10408 raft_consensus.cc:515] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.561017 10408 leader_election.cc:290] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
W20260709 13:05:20.568995  9798 leader_election.cc:343] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.569763 10262 raft_consensus.cc:493] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:20.569952  9801 leader_election.cc:343] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.570446  9801 leader_election.cc:304] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:20.569778  9758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ff13f827e7411d9a5782fb06e93c9b" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:20.570371 10262 raft_consensus.cc:515] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:20.571025  9658 catalog_manager.cc:5697] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 0 to 1, leader changed from <none> to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 1 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:20.574081 10262 leader_election.cc:290] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:20.575412 10235 raft_consensus.cc:2749] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:05:20.577518  9798 leader_election.cc:343] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.585125  9755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6bd3b28a03c44fda0c31e4198a86726" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:20.588253 10028 tablet_bootstrap.cc:492] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:20.588800 10028 ts_tablet_manager.cc:1403] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.314s	user 0.015s	sys 0.013s
I20260709 13:05:20.590200  9904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46065991d847dfa3d934961da0830a" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:20.591086  9904 raft_consensus.cc:2468] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:20.593284 10030 tablet_bootstrap.cc:492] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:20.593873 10030 ts_tablet_manager.cc:1403] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.334s	user 0.042s	sys 0.036s
I20260709 13:05:20.594472  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46065991d847dfa3d934961da0830a" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:20.595294  9978 raft_consensus.cc:2468] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
W20260709 13:05:20.597096  9872 leader_election.cc:343] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.605337 10028 raft_consensus.cc:359] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:20.606161 10028 raft_consensus.cc:385] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.606451 10028 raft_consensus.cc:740] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.597157  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ff13f827e7411d9a5782fb06e93c9b" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:20.609822 10028 consensus_queue.cc:260] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:20.611577 10028 ts_tablet_manager.cc:1434] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 13:05:20.612560 10028 tablet_bootstrap.cc:492] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:20.598160  9726 leader_election.cc:304] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:20.613976 10038 raft_consensus.cc:2804] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:20.614398 10038 raft_consensus.cc:493] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.614756 10038 raft_consensus.cc:3060] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.624662 10038 raft_consensus.cc:515] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:20.597158 10030 raft_consensus.cc:359] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:20.626257 10030 raft_consensus.cc:385] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.626605 10030 raft_consensus.cc:740] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.642371 10028 tablet_bootstrap.cc:654] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.627303 10038 leader_election.cc:290] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:20.627413 10030 consensus_queue.cc:260] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:20.648897 10030 ts_tablet_manager.cc:1434] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.000s
I20260709 13:05:20.650055 10030 tablet_bootstrap.cc:492] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:20.642297  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6bd3b28a03c44fda0c31e4198a86726" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:20.651692  9833 raft_consensus.cc:3060] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.654856  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46065991d847dfa3d934961da0830a" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:20.655598  9974 raft_consensus.cc:3060] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.657205 10028 tablet_bootstrap.cc:492] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:20.657733 10028 ts_tablet_manager.cc:1403] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.045s	user 0.008s	sys 0.002s
I20260709 13:05:20.660652 10028 raft_consensus.cc:359] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.661342 10028 raft_consensus.cc:385] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.661594 10028 raft_consensus.cc:740] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.662257 10028 consensus_queue.cc:260] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.664590 10028 ts_tablet_manager.cc:1434] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:05:20.665351 10029 tablet_bootstrap.cc:492] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:20.665962 10029 ts_tablet_manager.cc:1403] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.311s	user 0.009s	sys 0.014s
I20260709 13:05:20.669409 10029 raft_consensus.cc:359] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.670156 10029 raft_consensus.cc:385] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.670457 10029 raft_consensus.cc:740] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.672570 10049 raft_consensus.cc:493] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.673060 10049 raft_consensus.cc:515] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:20.673738 10029 consensus_queue.cc:260] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.678721  9833 raft_consensus.cc:2468] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:20.642833  9904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46065991d847dfa3d934961da0830a" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:20.683933  9904 raft_consensus.cc:3060] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.679816  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d69b492d298d42e18ab6b82be92774ca" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:20.680343 10402 raft_consensus.cc:493] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.674774 10049 leader_election.cc:290] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:20.686254 10402 raft_consensus.cc:515] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:20.688472 10028 tablet_bootstrap.cc:492] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0: Bootstrap starting.
I20260709 13:05:20.688694 10402 leader_election.cc:290] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:20.691329  9974 raft_consensus.cc:2468] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:20.682358 10030 tablet_bootstrap.cc:654] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:20.692546  9799 leader_election.cc:343] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.693017  9799 leader_election.cc:304] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:20.694816 10313 raft_consensus.cc:2749] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:20.699753  9904 raft_consensus.cc:2468] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:20.707386 10028 tablet_bootstrap.cc:654] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.707211  9755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d69b492d298d42e18ab6b82be92774ca" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:20.683219 10029 ts_tablet_manager.cc:1434] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:05:20.717538 10029 tablet_bootstrap.cc:492] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:20.718468  9872 leader_election.cc:343] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:20.729897  9873 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 47225.4us
W20260709 13:05:20.732549  9873 leader_election.cc:343] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.737244  9873 leader_election.cc:304] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:20.746644  9874 leader_election.cc:304] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:20.750485 10402 raft_consensus.cc:2749] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:20.754587 10353 raft_consensus.cc:2804] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:20.743892  9726 leader_election.cc:304] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:20.767222 10028 tablet_bootstrap.cc:492] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0: No bootstrap required, opened a new log
I20260709 13:05:20.780762 10028 ts_tablet_manager.cc:1403] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.018s
I20260709 13:05:20.783581 10038 raft_consensus.cc:2804] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:20.784101 10038 raft_consensus.cc:697] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:20.764230  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "140e604980a14cfe9b9ee5918dc45bb2" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:20.784792 10038 consensus_queue.cc:237] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:20.765986  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "140e604980a14cfe9b9ee5918dc45bb2" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:20.781095 10353 raft_consensus.cc:697] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:20.803149  9833 raft_consensus.cc:2468] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
W20260709 13:05:20.791647  9726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45019, user_credentials={real_user=slave}} blocked reactor thread for 53045.4us
W20260709 13:05:20.808862  9706 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:05:20.809900  9873 leader_election.cc:343] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.805225  9983 raft_consensus.cc:3060] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.816927  9874 leader_election.cc:304] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:20.817811 10283 raft_consensus.cc:2804] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:20.817476 10353 consensus_queue.cc:237] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.818217 10283 raft_consensus.cc:493] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.818584 10283 raft_consensus.cc:3060] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.818696 10030 tablet_bootstrap.cc:492] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:20.819334 10030 ts_tablet_manager.cc:1403] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.170s	user 0.031s	sys 0.017s
W20260709 13:05:20.807181  9801 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45019, user_credentials={real_user=slave}} blocked reactor thread for 51429.1us
I20260709 13:05:20.827661 10030 raft_consensus.cc:359] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:20.830188 10030 raft_consensus.cc:385] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.830591 10030 raft_consensus.cc:740] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.833091 10030 consensus_queue.cc:260] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:20.852418 10030 ts_tablet_manager.cc:1434] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.002s
I20260709 13:05:20.859406 10299 raft_consensus.cc:493] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.859988 10299 raft_consensus.cc:515] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:20.861122 10283 raft_consensus.cc:515] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:20.862181 10299 leader_election.cc:290] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:20.866742 10283 leader_election.cc:290] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:20.866818  9755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d8c823d37d4e9a864b46403d267028" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:20.867868  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "140e604980a14cfe9b9ee5918dc45bb2" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:20.868314  9830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "140e604980a14cfe9b9ee5918dc45bb2" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
W20260709 13:05:20.869683  9946 leader_election.cc:343] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.869194  9830 raft_consensus.cc:3060] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.876626 10030 tablet_bootstrap.cc:492] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:20.875231  9656 catalog_manager.cc:5697] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:20.886524  9873 leader_election.cc:343] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.882465 10354 raft_consensus.cc:493] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.889048  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d8c823d37d4e9a864b46403d267028" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:20.887488 10354 raft_consensus.cc:515] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.889902  9835 raft_consensus.cc:2468] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 0.
I20260709 13:05:20.892740  9948 leader_election.cc:304] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:20.893842 10328 raft_consensus.cc:2804] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:20.894280 10328 raft_consensus.cc:493] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.894665 10328 raft_consensus.cc:3060] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.897851 10029 tablet_bootstrap.cc:654] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:20.904415 10028 raft_consensus.cc:359] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:20.903699 10328 raft_consensus.cc:515] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:20.905730  9755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecae0626a5c744f0ad9571147ccc239d" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
W20260709 13:05:20.921933  9872 leader_election.cc:343] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.927006 10328 leader_election.cc:290] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:20.928157  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d8c823d37d4e9a864b46403d267028" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:20.929020  9835 raft_consensus.cc:3060] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.919595  9758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d8c823d37d4e9a864b46403d267028" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
W20260709 13:05:20.931682  9946 leader_election.cc:343] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:20.905269 10028 raft_consensus.cc:385] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:20.935783 10028 raft_consensus.cc:740] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:20.936671 10028 consensus_queue.cc:260] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:20.938469 10028 ts_tablet_manager.cc:1434] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0: Time spent starting tablet: real 0.157s	user 0.004s	sys 0.001s
I20260709 13:05:20.949230  9654 catalog_manager.cc:5697] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:20.956599  9829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecae0626a5c744f0ad9571147ccc239d" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:20.957418  9829 raft_consensus.cc:2468] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 0.
I20260709 13:05:20.959281 10328 consensus_queue.cc:1048] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:20.963980  9874 leader_election.cc:304] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:20.965157 10353 raft_consensus.cc:2804] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:20.965615 10353 raft_consensus.cc:493] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.966136 10353 raft_consensus.cc:3060] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:20.972569 10354 leader_election.cc:290] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:20.974727  9830 raft_consensus.cc:2468] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:20.976269  9874 leader_election.cc:304] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:20.978724 10354 raft_consensus.cc:2804] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:20.979777 10354 raft_consensus.cc:697] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:20.980947 10354 consensus_queue.cc:237] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:20.993995 10038 raft_consensus.cc:493] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.010121 10038 raft_consensus.cc:515] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.994035 10283 raft_consensus.cc:493] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.013799 10283 raft_consensus.cc:515] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.016737 10283 leader_election.cc:290] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:21.017244 10353 raft_consensus.cc:515] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:20.999174 10328 consensus_queue.cc:1048] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:21.022413  9902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6bd3b28a03c44fda0c31e4198a86726" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:21.027307  9758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a516bf1d774804b94074f9765006ad" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:21.028842  9753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecae0626a5c744f0ad9571147ccc239d" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:21.029585  9983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a516bf1d774804b94074f9765006ad" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:21.031580  9873 leader_election.cc:343] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:21.032377  9872 leader_election.cc:343] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:21.032778  9872 leader_election.cc:304] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
W20260709 13:05:21.033856  9872 leader_election.cc:343] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:21.034420 10283 raft_consensus.cc:2749] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:21.034524  9829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6bd3b28a03c44fda0c31e4198a86726" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:21.001044 10030 tablet_bootstrap.cc:654] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:21.036263  9654 catalog_manager.cc:5697] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 1 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } }
W20260709 13:05:21.002161  9724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 263126us
I20260709 13:05:21.037389  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecae0626a5c744f0ad9571147ccc239d" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:21.049261  9834 raft_consensus.cc:3060] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.036813 10038 leader_election.cc:290] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:21.058038  9834 raft_consensus.cc:2468] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:21.035557  9829 raft_consensus.cc:2393] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 1: Already voted for candidate 6e10aab26c644389b8161974dcabc5d0 in this term.
I20260709 13:05:21.072139 10353 leader_election.cc:290] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:21.059384  9835 raft_consensus.cc:2468] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:21.086527  9948 leader_election.cc:304] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:21.087513 10328 raft_consensus.cc:2804] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:21.088469 10328 raft_consensus.cc:697] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:21.089738 10328 consensus_queue.cc:237] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:21.096045  9874 leader_election.cc:304] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:21.105997 10030 tablet_bootstrap.cc:492] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:21.106594 10030 ts_tablet_manager.cc:1403] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.236s	user 0.010s	sys 0.005s
I20260709 13:05:21.126708 10353 raft_consensus.cc:2804] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:21.143842 10353 raft_consensus.cc:697] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:21.145095 10353 consensus_queue.cc:237] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:21.165333 10029 tablet_bootstrap.cc:492] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:21.166013 10029 ts_tablet_manager.cc:1403] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.449s	user 0.082s	sys 0.082s
I20260709 13:05:21.166307 10030 raft_consensus.cc:359] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:21.174017 10030 raft_consensus.cc:385] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:21.174468 10030 raft_consensus.cc:740] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:21.176592 10030 consensus_queue.cc:260] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:21.176314 10029 raft_consensus.cc:359] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:21.179554 10029 raft_consensus.cc:385] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:21.180028 10029 raft_consensus.cc:740] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:21.182657  9656 catalog_manager.cc:5697] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:21.181147 10029 consensus_queue.cc:260] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:21.210528 10030 ts_tablet_manager.cc:1434] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.104s	user 0.010s	sys 0.001s
I20260709 13:05:21.211766 10030 tablet_bootstrap.cc:492] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:21.219434 10029 ts_tablet_manager.cc:1434] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.007s
I20260709 13:05:21.220711 10029 tablet_bootstrap.cc:492] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:21.221843  9725 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 93368us
I20260709 13:05:21.222813  9725 leader_election.cc:304] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:21.223778 10038 raft_consensus.cc:3060] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.228356 10030 tablet_bootstrap.cc:654] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:21.228578 10463 consensus_queue.cc:1048] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:21.232182  9724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 72103.8us
I20260709 13:05:21.232406 10038 raft_consensus.cc:2749] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:21.239464 10376 consensus_queue.cc:1048] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:21.241240 10029 tablet_bootstrap.cc:654] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:21.247808 10299 raft_consensus.cc:493] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.248392 10299 raft_consensus.cc:515] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:21.252211  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912bd6d0efac4042884ce6823f7e8e73" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:21.254690 10147 consensus_queue.cc:1048] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:21.255254  9829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912bd6d0efac4042884ce6823f7e8e73" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:21.250722 10299 leader_election.cc:290] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:21.261849  9948 leader_election.cc:304] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:21.263373 10299 raft_consensus.cc:2749] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:21.267278 10030 tablet_bootstrap.cc:492] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:21.267879 10030 ts_tablet_manager.cc:1403] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.056s	user 0.012s	sys 0.005s
I20260709 13:05:21.270864 10030 raft_consensus.cc:359] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.288941 10030 raft_consensus.cc:385] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:21.289283 10030 raft_consensus.cc:740] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:21.290069 10030 consensus_queue.cc:260] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.292320 10030 ts_tablet_manager.cc:1434] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:05:21.293303 10030 tablet_bootstrap.cc:492] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:21.315294 10030 tablet_bootstrap.cc:654] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:21.290398  9654 catalog_manager.cc:5697] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 0 to 1, leader changed from <none> to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 1 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:21.330758  9656 catalog_manager.cc:5697] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:21.376143 10029 tablet_bootstrap.cc:492] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:21.380637 10029 ts_tablet_manager.cc:1403] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.160s	user 0.022s	sys 0.033s
I20260709 13:05:21.401341 10030 tablet_bootstrap.cc:492] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:21.401975 10030 ts_tablet_manager.cc:1403] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.109s	user 0.017s	sys 0.004s
I20260709 13:05:21.408283 10029 raft_consensus.cc:359] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.419190 10029 raft_consensus.cc:385] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:21.419730 10029 raft_consensus.cc:740] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:21.423074 10029 consensus_queue.cc:260] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.425365 10147 consensus_queue.cc:1048] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:21.417961 10030 raft_consensus.cc:359] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:21.434588 10029 ts_tablet_manager.cc:1434] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260709 13:05:21.439131 10029 tablet_bootstrap.cc:492] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:21.446799 10030 raft_consensus.cc:385] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:21.447669 10030 raft_consensus.cc:740] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:21.448709 10030 consensus_queue.cc:260] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:21.454958 10029 tablet_bootstrap.cc:654] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:21.480309 10030 ts_tablet_manager.cc:1434] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.078s	user 0.010s	sys 0.000s
I20260709 13:05:21.481771 10030 tablet_bootstrap.cc:492] T 140e604980a14cfe9b9ee5918dc45bb2 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:21.492298 10029 tablet_bootstrap.cc:492] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:21.492964 10029 ts_tablet_manager.cc:1403] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.054s	user 0.011s	sys 0.001s
I20260709 13:05:21.494683 10030 tablet_bootstrap.cc:654] T 140e604980a14cfe9b9ee5918dc45bb2 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:21.496152 10029 raft_consensus.cc:359] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:21.497017 10029 raft_consensus.cc:385] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:21.497321 10029 raft_consensus.cc:740] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:21.498173 10029 consensus_queue.cc:260] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:21.512692 10029 ts_tablet_manager.cc:1434] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:05:21.527242 10030 tablet_bootstrap.cc:492] T 140e604980a14cfe9b9ee5918dc45bb2 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:21.527242 10029 tablet_bootstrap.cc:492] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:21.527832 10030 ts_tablet_manager.cc:1403] T 140e604980a14cfe9b9ee5918dc45bb2 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.046s	user 0.020s	sys 0.003s
I20260709 13:05:21.530797 10030 raft_consensus.cc:359] T 140e604980a14cfe9b9ee5918dc45bb2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:21.531653 10030 raft_consensus.cc:385] T 140e604980a14cfe9b9ee5918dc45bb2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:21.531986 10030 raft_consensus.cc:740] T 140e604980a14cfe9b9ee5918dc45bb2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:21.532758 10030 consensus_queue.cc:260] T 140e604980a14cfe9b9ee5918dc45bb2 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:21.535491 10030 ts_tablet_manager.cc:1434] T 140e604980a14cfe9b9ee5918dc45bb2 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:05:21.538252 10029 tablet_bootstrap.cc:654] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:21.540016 10030 tablet_bootstrap.cc:492] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:21.549948 10030 tablet_bootstrap.cc:654] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:21.560408 10030 tablet_bootstrap.cc:492] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:21.560945 10030 ts_tablet_manager.cc:1403] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.000s
I20260709 13:05:21.561120 10262 raft_consensus.cc:493] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.561682 10262 raft_consensus.cc:515] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.562124 10376 raft_consensus.cc:493] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.562625 10376 raft_consensus.cc:515] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:21.564178 10262 leader_election.cc:290] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:21.565742 10029 tablet_bootstrap.cc:492] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:21.566273 10029 ts_tablet_manager.cc:1403] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.051s	user 0.009s	sys 0.005s
I20260709 13:05:21.566632  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d69b492d298d42e18ab6b82be92774ca" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:21.566813 10376 leader_election.cc:290] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:21.567306  9909 raft_consensus.cc:2468] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:21.568439  9829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d8c823d37d4e9a864b46403d267028" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:21.570043  9726 leader_election.cc:304] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:21.569254  9829 raft_consensus.cc:2393] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 1: Already voted for candidate 23cbd6234c5e43d89f22c235ed6bb96a in this term.
I20260709 13:05:21.572953 10376 raft_consensus.cc:2804] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:21.573482 10376 raft_consensus.cc:493] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.569248 10029 raft_consensus.cc:359] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.574057 10376 raft_consensus.cc:3060] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.564185 10030 raft_consensus.cc:359] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:21.574107 10029 raft_consensus.cc:385] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:21.584826 10030 raft_consensus.cc:385] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:21.585278 10030 raft_consensus.cc:740] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:21.573864 10299 raft_consensus.cc:493] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.585948 10029 raft_consensus.cc:740] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:21.586274 10299 raft_consensus.cc:515] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.587008 10029 consensus_queue.cc:260] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.588485 10299 leader_election.cc:290] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:21.590041 10376 raft_consensus.cc:515] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.591615  9758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d69b492d298d42e18ab6b82be92774ca" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:21.592615  9983 raft_consensus.cc:3060] T 140e604980a14cfe9b9ee5918dc45bb2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.595361  9904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d69b492d298d42e18ab6b82be92774ca" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:21.596053  9904 raft_consensus.cc:2468] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 0.
I20260709 13:05:21.584162  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d69b492d298d42e18ab6b82be92774ca" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:21.597451  9974 raft_consensus.cc:2468] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:21.594182  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d8c823d37d4e9a864b46403d267028" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:21.594458 10029 ts_tablet_manager.cc:1434] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:05:21.599421 10029 tablet_bootstrap.cc:492] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:21.604785 10399 consensus_queue.cc:1048] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:21.606227 10029 tablet_bootstrap.cc:654] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:21.607201 10030 consensus_queue.cc:260] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:21.608821  9758 raft_consensus.cc:2393] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23cbd6234c5e43d89f22c235ed6bb96a in current term 1: Already voted for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in this term.
I20260709 13:05:21.609979 10030 ts_tablet_manager.cc:1434] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.000s
I20260709 13:05:21.611100 10030 tablet_bootstrap.cc:492] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:21.613425 10376 leader_election.cc:290] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:21.622973  9724 leader_election.cc:304] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:21.626791 10030 tablet_bootstrap.cc:654] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:21.628381  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d69b492d298d42e18ab6b82be92774ca" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:21.651144  9909 raft_consensus.cc:3060] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.658447  9949 leader_election.cc:304] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:21.642364 10353 consensus_queue.cc:1048] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:21.646219  9974 raft_consensus.cc:3060] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.652667  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d69b492d298d42e18ab6b82be92774ca" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:21.660300  9979 raft_consensus.cc:3060] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.655517 10038 raft_consensus.cc:3060] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.638496 10029 tablet_bootstrap.cc:492] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:21.670069 10029 ts_tablet_manager.cc:1403] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.071s	user 0.006s	sys 0.012s
W20260709 13:05:21.679811  9872 consensus_peers.cc:597] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:21.683638 10038 raft_consensus.cc:2749] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:21.685158 10029 raft_consensus.cc:359] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:21.695590 10029 raft_consensus.cc:385] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:21.695950 10029 raft_consensus.cc:740] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:21.696760 10029 consensus_queue.cc:260] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:21.697739 10408 consensus_queue.cc:1048] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:21.698489 10029 ts_tablet_manager.cc:1434] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.028s	user 0.002s	sys 0.005s
I20260709 13:05:21.699720 10029 tablet_bootstrap.cc:492] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:21.720577 10299 consensus_queue.cc:1048] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:21.726037 10030 tablet_bootstrap.cc:492] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:21.726656 10030 ts_tablet_manager.cc:1403] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.116s	user 0.017s	sys 0.000s
I20260709 13:05:21.731945 10029 tablet_bootstrap.cc:654] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:21.735437 10072 consensus_queue.cc:1048] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:21.746662 10030 raft_consensus.cc:359] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.750808 10030 raft_consensus.cc:385] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:21.751164 10030 raft_consensus.cc:740] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:21.755471 10030 consensus_queue.cc:260] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.767962 10029 tablet_bootstrap.cc:492] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:21.774152 10030 ts_tablet_manager.cc:1434] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:05:21.775591 10030 tablet_bootstrap.cc:492] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a: Bootstrap starting.
I20260709 13:05:21.768703 10029 ts_tablet_manager.cc:1403] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.069s	user 0.012s	sys 0.000s
I20260709 13:05:21.786501 10376 raft_consensus.cc:493] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.786628 10029 raft_consensus.cc:359] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:21.787410 10029 raft_consensus.cc:385] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:21.690182  9909 raft_consensus.cc:2468] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:21.787705 10029 raft_consensus.cc:740] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:21.787071 10376 raft_consensus.cc:515] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.788484 10029 consensus_queue.cc:260] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:21.789079  9726 leader_election.cc:304] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:21.789979 10038 raft_consensus.cc:2804] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:21.688995  9979 raft_consensus.cc:2468] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:21.790599 10038 raft_consensus.cc:697] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:21.798012 10038 consensus_queue.cc:237] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.792867  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a981502782314272a8a9bee8da734427" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:21.799120  9909 raft_consensus.cc:2468] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:21.800343  9726 leader_election.cc:304] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:21.801189 10462 raft_consensus.cc:2804] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:21.802963 10462 raft_consensus.cc:493] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.803447 10462 raft_consensus.cc:3060] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.792611 10376 leader_election.cc:290] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:21.806610 10328 raft_consensus.cc:2764] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:05:21.810213 10029 ts_tablet_manager.cc:1434] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:05:21.811105 10030 tablet_bootstrap.cc:654] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:21.811367 10029 tablet_bootstrap.cc:492] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:21.814632 10462 raft_consensus.cc:515] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.817873  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a981502782314272a8a9bee8da734427" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:21.818517  9909 raft_consensus.cc:3060] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.829944 10328 raft_consensus.cc:493] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.830698 10328 raft_consensus.cc:515] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:21.835793  9909 raft_consensus.cc:2468] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:21.838289  9726 leader_election.cc:304] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:21.841684 10328 leader_election.cc:290] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:21.843605  9829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ff13f827e7411d9a5782fb06e93c9b" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:21.848675  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a981502782314272a8a9bee8da734427" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:21.849608  9829 raft_consensus.cc:2468] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 0.
I20260709 13:05:21.849781  9980 raft_consensus.cc:2468] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 0.
I20260709 13:05:21.851346  9948 leader_election.cc:304] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:21.853246 10328 raft_consensus.cc:2804] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:21.853853 10328 raft_consensus.cc:493] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.854308 10328 raft_consensus.cc:3060] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.855336  9754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ff13f827e7411d9a5782fb06e93c9b" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:21.856750 10376 raft_consensus.cc:2804] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:21.857493 10376 raft_consensus.cc:697] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
W20260709 13:05:21.864396  9946 leader_election.cc:343] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:21.875458  9977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a981502782314272a8a9bee8da734427" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:21.876157  9977 raft_consensus.cc:3060] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.865304 10462 leader_election.cc:290] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:21.858639 10376 consensus_queue.cc:237] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:21.897387 10235 raft_consensus.cc:493] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.898067 10235 raft_consensus.cc:515] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:21.901739 10235 leader_election.cc:290] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:21.902513  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "622d51c8b2124715a814796353745c9b" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:21.904048  9980 raft_consensus.cc:2468] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:21.902542  9754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "622d51c8b2124715a814796353745c9b" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:21.905287  9754 raft_consensus.cc:2468] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 0.
I20260709 13:05:21.905566  9799 leader_election.cc:304] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:21.909873 10429 raft_consensus.cc:2804] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:21.910354 10429 raft_consensus.cc:493] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.910697 10429 raft_consensus.cc:3060] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:21.934150 10029 tablet_bootstrap.cc:654] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:21.941982 10483 log.cc:927] Time spent T 140e604980a14cfe9b9ee5918dc45bb2 P 23cbd6234c5e43d89f22c235ed6bb96a: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:05:21.948745 10463 raft_consensus.cc:493] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.949321 10463 raft_consensus.cc:515] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:21.951633 10463 leader_election.cc:290] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:21.968627 10429 raft_consensus.cc:515] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:21.971458  9904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9623952612f4929825a90c043f1c6ba" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:22.007745  9904 raft_consensus.cc:2393] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 1: Already voted for candidate 6e10aab26c644389b8161974dcabc5d0 in this term.
I20260709 13:05:21.977687  9832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9623952612f4929825a90c043f1c6ba" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:22.009141  9832 raft_consensus.cc:2393] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 1: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
I20260709 13:05:21.981571 10328 raft_consensus.cc:515] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:22.004019 10030 tablet_bootstrap.cc:492] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a: No bootstrap required, opened a new log
I20260709 13:05:22.010737 10030 ts_tablet_manager.cc:1403] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent bootstrapping tablet: real 0.235s	user 0.026s	sys 0.028s
I20260709 13:05:22.012122 10328 leader_election.cc:290] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:22.014109 10030 raft_consensus.cc:359] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:22.014770 10030 raft_consensus.cc:385] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:22.015051 10030 raft_consensus.cc:740] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Initialized, Role: FOLLOWER
I20260709 13:05:22.019846  9754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ff13f827e7411d9a5782fb06e93c9b" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:22.023507  9726 leader_election.cc:304] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:22.023655 10030 consensus_queue.cc:260] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
W20260709 13:05:22.026557  9946 leader_election.cc:343] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:22.027640  9832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ff13f827e7411d9a5782fb06e93c9b" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:22.028354  9832 raft_consensus.cc:3060] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.029753 10030 ts_tablet_manager.cc:1434] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:05:22.034039 10262 raft_consensus.cc:3060] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.045867  9832 raft_consensus.cc:2468] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:22.055927  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "622d51c8b2124715a814796353745c9b" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:22.056747  9979 raft_consensus.cc:3060] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.056887  9758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "622d51c8b2124715a814796353745c9b" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:22.057600  9758 raft_consensus.cc:3060] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.061842  9948 leader_election.cc:304] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:22.063069 10488 raft_consensus.cc:2804] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:22.063601 10488 raft_consensus.cc:697] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:22.064388 10488 consensus_queue.cc:237] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:22.065001  9979 raft_consensus.cc:2468] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:22.055709  9977 raft_consensus.cc:2468] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 1.
I20260709 13:05:22.055342 10429 leader_election.cc:290] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:22.073077  9799 leader_election.cc:304] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
W20260709 13:05:22.066665  9706 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:22.078858 10429 raft_consensus.cc:2804] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:22.080338 10429 raft_consensus.cc:697] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:22.081403 10429 consensus_queue.cc:237] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:22.084759 10463 raft_consensus.cc:493] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.085460 10463 raft_consensus.cc:515] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:22.103685  9832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecae0626a5c744f0ad9571147ccc239d" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:22.146809  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecae0626a5c744f0ad9571147ccc239d" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:22.151206  9726 leader_election.cc:304] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:22.153416 10462 raft_consensus.cc:2749] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:22.163851 10262 raft_consensus.cc:2749] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:22.173336 10463 leader_election.cc:290] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:22.184497  9754 raft_consensus.cc:3060] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.202586 10496 consensus_queue.cc:1048] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:22.219920 10029 tablet_bootstrap.cc:492] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:22.220711 10029 ts_tablet_manager.cc:1403] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.410s	user 0.055s	sys 0.035s
I20260709 13:05:22.228063  9758 raft_consensus.cc:2468] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:22.231994  9658 catalog_manager.cc:5697] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 0 to 1, leader changed from <none> to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 1 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:22.233893 10029 raft_consensus.cc:359] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:22.234728 10029 raft_consensus.cc:385] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:22.235086 10029 raft_consensus.cc:740] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:22.235770 10029 consensus_queue.cc:260] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:22.238276 10029 ts_tablet_manager.cc:1434] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:05:22.242169  9654 catalog_manager.cc:5697] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:22.247316 10029 tablet_bootstrap.cc:492] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:22.251724  9724 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 166148us
I20260709 13:05:22.256762  9655 catalog_manager.cc:5697] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } }
I20260709 13:05:22.285140 10262 consensus_queue.cc:1048] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:22.339576 10328 raft_consensus.cc:493] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.340219 10328 raft_consensus.cc:515] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:22.344067 10328 leader_election.cc:290] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:22.346813  9758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a516bf1d774804b94074f9765006ad" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:22.347048  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a516bf1d774804b94074f9765006ad" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:22.347730  9909 raft_consensus.cc:2468] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 0.
W20260709 13:05:22.351641  9946 leader_election.cc:343] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:22.352589  9949 leader_election.cc:304] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:22.353621 10488 raft_consensus.cc:2804] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:22.354015 10488 raft_consensus.cc:493] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.354429 10488 raft_consensus.cc:3060] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.355444 10387 raft_consensus.cc:493] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.356019 10387 raft_consensus.cc:515] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:22.358240 10387 leader_election.cc:290] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:22.360304  9754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9623952612f4929825a90c043f1c6ba" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:22.360456  9902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9623952612f4929825a90c043f1c6ba" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:22.361028  9754 raft_consensus.cc:2468] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:22.361279  9902 raft_consensus.cc:2468] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:22.363112  9798 leader_election.cc:304] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:22.364362 10387 raft_consensus.cc:2804] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:22.364733 10387 raft_consensus.cc:493] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.365084 10387 raft_consensus.cc:3060] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.372845 10387 raft_consensus.cc:515] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:22.373150 10488 raft_consensus.cc:515] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:22.377276 10029 tablet_bootstrap.cc:654] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:22.378664 10494 raft_consensus.cc:493] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.396363 10494 raft_consensus.cc:515] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:22.399798 10494 leader_election.cc:290] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:22.383756 10488 leader_election.cc:290] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:22.384819  9754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a516bf1d774804b94074f9765006ad" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:22.384819  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a516bf1d774804b94074f9765006ad" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:22.402819  9909 raft_consensus.cc:3060] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.403295  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ff13f827e7411d9a5782fb06e93c9b" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:22.409693  9902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9623952612f4929825a90c043f1c6ba" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:22.410465  9902 raft_consensus.cc:3060] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:22.411541  9946 leader_election.cc:343] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:22.386119  9758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9623952612f4929825a90c043f1c6ba" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
I20260709 13:05:22.412490  9758 raft_consensus.cc:3060] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.421476  9758 raft_consensus.cc:2468] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 2.
I20260709 13:05:22.425647  9798 leader_election.cc:304] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:22.426608 10429 raft_consensus.cc:2804] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:22.427273 10429 raft_consensus.cc:697] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 2 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:22.428207 10429 consensus_queue.cc:237] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:22.433264  9909 raft_consensus.cc:2468] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:22.434731  9949 leader_election.cc:304] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:22.436189 10488 raft_consensus.cc:2804] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:22.437258 10488 raft_consensus.cc:697] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:22.438706 10488 consensus_queue.cc:237] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:22.443109 10262 consensus_queue.cc:1048] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:22.445779 10463 consensus_queue.cc:1048] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:22.454793 10029 tablet_bootstrap.cc:492] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4: No bootstrap required, opened a new log
I20260709 13:05:22.455509 10029 ts_tablet_manager.cc:1403] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.216s	user 0.007s	sys 0.006s
I20260709 13:05:22.457783  9832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ff13f827e7411d9a5782fb06e93c9b" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:22.460484  9832 raft_consensus.cc:2393] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 1: Already voted for candidate 23cbd6234c5e43d89f22c235ed6bb96a in this term.
I20260709 13:05:22.468294  9725 leader_election.cc:304] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:22.470803 10262 raft_consensus.cc:3060] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.462940 10387 leader_election.cc:290] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:22.487294 10262 raft_consensus.cc:2749] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:22.501294 10029 raft_consensus.cc:359] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:22.502105 10029 raft_consensus.cc:385] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:22.502655 10029 raft_consensus.cc:740] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:22.505669 10029 consensus_queue.cc:260] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:22.508850 10313 consensus_queue.cc:1048] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:05:22.516634  9902 raft_consensus.cc:2468] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 2.
I20260709 13:05:22.552919 10029 ts_tablet_manager.cc:1434] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.001s
I20260709 13:05:22.552430  9656 catalog_manager.cc:5697] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:22.563076  9799 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 52424.4us
W20260709 13:05:22.589713  9801 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45019, user_credentials={real_user=slave}} blocked reactor thread for 49644.3us
W20260709 13:05:22.596608  9798 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 53693.5us
I20260709 13:05:22.603438 10512 consensus_queue.cc:1048] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:22.608296  9658 catalog_manager.cc:5697] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 0 to 1, leader changed from <none> to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 1 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:22.624603  9654 catalog_manager.cc:5697] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 0 to 2, leader changed from <none> to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:22.636200 10299 consensus_queue.cc:1048] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:22.675894 10488 consensus_queue.cc:1048] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:22.813210 10496 raft_consensus.cc:493] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.813743 10496 raft_consensus.cc:515] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:22.815769 10496 leader_election.cc:290] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:22.816947  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a516bf1d774804b94074f9765006ad" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:22.816947  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a516bf1d774804b94074f9765006ad" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:22.818070  9909 raft_consensus.cc:2393] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 1: Already voted for candidate 23cbd6234c5e43d89f22c235ed6bb96a in this term.
I20260709 13:05:22.819195  9726 leader_election.cc:304] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0
I20260709 13:05:22.819909 10496 raft_consensus.cc:3060] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:22.825930 10496 raft_consensus.cc:2749] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:22.877707 10488 consensus_queue.cc:1048] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:22.905736  9909 raft_consensus.cc:1275] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer a1cbca39e34743d8a40816c3bc5f2030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:22.908803 10328 consensus_queue.cc:1048] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:22.927872  9758 raft_consensus.cc:1275] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Refusing update from remote peer a1cbca39e34743d8a40816c3bc5f2030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:05:22.941329 10512 consensus_queue.cc:1048] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:22.943209 10512 consensus_queue.cc:1048] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:23.300851   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:05:23.307904  9798 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20260709 13:05:23.329520  9798 consensus_peers.cc:597] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.333961  9798 consensus_peers.cc:597] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.348512  9872 consensus_peers.cc:597] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.351169  9872 consensus_peers.cc:597] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.364964  9872 consensus_peers.cc:597] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.378434  9872 consensus_peers.cc:597] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.389748   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:23.390781   431 tablet_replica.cc:333] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.392921   431 raft_consensus.cc:2243] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.393498   431 raft_consensus.cc:2272] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.397259  9798 consensus_peers.cc:597] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.398408   431 tablet_replica.cc:333] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.399448   431 raft_consensus.cc:2243] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.400434   431 raft_consensus.cc:2272] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.403048   431 tablet_replica.cc:333] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.403971   431 raft_consensus.cc:2243] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.404552   431 raft_consensus.cc:2272] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.407254   431 tablet_replica.cc:333] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.408097   431 raft_consensus.cc:2243] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.409040   431 raft_consensus.cc:2272] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.410985   431 tablet_replica.cc:333] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.411942   431 raft_consensus.cc:2243] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.412530   431 raft_consensus.cc:2272] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.415756   431 tablet_replica.cc:333] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.416671   431 raft_consensus.cc:2243] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.417272   431 raft_consensus.cc:2272] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.419776   431 tablet_replica.cc:333] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
W20260709 13:05:23.420527  9946 consensus_peers.cc:597] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.420679   431 raft_consensus.cc:2243] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:05:23.421357  9872 consensus_peers.cc:597] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.421789   431 raft_consensus.cc:2272] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.424041   431 tablet_replica.cc:333] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.424806   431 raft_consensus.cc:2243] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.425319   431 raft_consensus.cc:2272] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.427420   431 tablet_replica.cc:333] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.427989   431 raft_consensus.cc:2243] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.428409   431 raft_consensus.cc:2272] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.430485   431 tablet_replica.cc:333] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.431391   431 raft_consensus.cc:2243] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.432252   431 raft_consensus.cc:2272] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.434517   431 tablet_replica.cc:333] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.435374   431 raft_consensus.cc:2243] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.435809   431 raft_consensus.cc:2272] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.438081   431 tablet_replica.cc:333] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.438791   431 raft_consensus.cc:2243] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.439879   431 raft_consensus.cc:2272] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.442068   431 tablet_replica.cc:333] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.442943   431 raft_consensus.cc:2243] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.443987   431 raft_consensus.cc:2272] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.446732   431 tablet_replica.cc:333] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.447602   431 raft_consensus.cc:2243] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:23.447822  9798 consensus_peers.cc:597] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.448264   431 raft_consensus.cc:2272] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.450776   431 tablet_replica.cc:333] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.451428   431 raft_consensus.cc:2243] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.451848   431 raft_consensus.cc:2272] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.454325   431 tablet_replica.cc:333] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.455097   431 raft_consensus.cc:2243] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.455518   431 raft_consensus.cc:2272] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.458243   431 tablet_replica.cc:333] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.459069   431 raft_consensus.cc:2243] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.459842   431 raft_consensus.cc:2272] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.462792   431 tablet_replica.cc:333] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.463761   431 raft_consensus.cc:2243] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:05:23.463881  9946 consensus_peers.cc:597] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.464854   431 raft_consensus.cc:2272] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.467787   431 tablet_replica.cc:333] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.468616   431 raft_consensus.cc:2243] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.469213   431 raft_consensus.cc:2272] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.471613   431 tablet_replica.cc:333] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.472483   431 raft_consensus.cc:2243] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.473016   431 raft_consensus.cc:2272] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.475605   431 tablet_replica.cc:333] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.476476   431 raft_consensus.cc:2243] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:23.476931  9798 consensus_peers.cc:597] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.477033   431 raft_consensus.cc:2272] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.482484   431 tablet_replica.cc:333] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.483503   431 raft_consensus.cc:2243] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.484122   431 raft_consensus.cc:2272] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.485370  9798 consensus_peers.cc:597] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.487257   431 tablet_replica.cc:333] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.488502   431 raft_consensus.cc:2243] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.490001   431 raft_consensus.cc:2272] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.490265  9798 consensus_peers.cc:597] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.492985   431 tablet_replica.cc:333] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.493999   431 raft_consensus.cc:2243] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.494534   431 raft_consensus.cc:2272] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.497097   431 tablet_replica.cc:333] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.497961   431 raft_consensus.cc:2243] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.499118   431 raft_consensus.cc:2272] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.502002   431 tablet_replica.cc:333] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.502893   431 raft_consensus.cc:2243] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.503500   431 raft_consensus.cc:2272] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.506206   431 tablet_replica.cc:333] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.506999   431 raft_consensus.cc:2243] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.507550   431 raft_consensus.cc:2272] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.510462   431 tablet_replica.cc:333] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.511307   431 raft_consensus.cc:2243] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.512233   431 raft_consensus.cc:2272] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.515363   431 tablet_replica.cc:333] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.516249   431 raft_consensus.cc:2243] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.516842   431 raft_consensus.cc:2272] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.518018  9872 consensus_peers.cc:597] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.520036   431 tablet_replica.cc:333] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
W20260709 13:05:23.520741  9872 consensus_peers.cc:597] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.521051   431 raft_consensus.cc:2243] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.521773   431 raft_consensus.cc:2272] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.522781  9798 consensus_peers.cc:597] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.524139   431 tablet_replica.cc:333] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.525002   431 raft_consensus.cc:2243] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.526013   431 raft_consensus.cc:2272] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.528600   431 tablet_replica.cc:333] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.529138   431 raft_consensus.cc:2243] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:23.529517  9946 consensus_peers.cc:597] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.529580   431 raft_consensus.cc:2272] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.531934   431 tablet_replica.cc:333] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.532495   431 raft_consensus.cc:2243] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.532907   431 raft_consensus.cc:2272] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.534864   431 tablet_replica.cc:333] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.535454   431 raft_consensus.cc:2243] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.536235   431 raft_consensus.cc:2272] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.540117   431 tablet_replica.cc:333] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.540920   431 raft_consensus.cc:2243] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.542069   431 raft_consensus.cc:2272] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.545368   431 tablet_replica.cc:333] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.546162   431 raft_consensus.cc:2243] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.546651   431 raft_consensus.cc:2272] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.549187   431 tablet_replica.cc:333] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.549901   431 raft_consensus.cc:2243] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.550812   431 raft_consensus.cc:2272] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.552968  9798 consensus_peers.cc:597] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.553607   431 tablet_replica.cc:333] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.554441   431 raft_consensus.cc:2243] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.555016   431 raft_consensus.cc:2272] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.557809   431 tablet_replica.cc:333] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.558682   431 raft_consensus.cc:2243] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.559820   431 raft_consensus.cc:2272] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.563059   431 tablet_replica.cc:333] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.563664   431 raft_consensus.cc:2243] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.564109   431 raft_consensus.cc:2272] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.566186   431 tablet_replica.cc:333] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.566732   431 raft_consensus.cc:2243] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.567198   431 raft_consensus.cc:2272] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.569346   431 tablet_replica.cc:333] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.569905   431 raft_consensus.cc:2243] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:23.570775   431 raft_consensus.cc:2272] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.572754   431 tablet_replica.cc:333] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.573284   431 raft_consensus.cc:2243] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.573727   431 raft_consensus.cc:2272] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.575767   431 tablet_replica.cc:333] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.576279   431 raft_consensus.cc:2243] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.576740   431 raft_consensus.cc:2272] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:23.578963   431 tablet_replica.cc:333] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4: stopping tablet replica
I20260709 13:05:23.579434   431 raft_consensus.cc:2243] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:23.579833   431 raft_consensus.cc:2272] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:23.613883  9798 consensus_peers.cc:597] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.617790  9872 consensus_peers.cc:597] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.638901  9946 connection.cc:573] client connection to 127.0.107.193:41383 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:05:23.639664  9946 consensus_peers.cc:597] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.660526   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:05:23.709216  9946 consensus_peers.cc:597] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.729257  9872 consensus_peers.cc:597] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.731830  9946 consensus_peers.cc:597] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.764293  9798 consensus_peers.cc:597] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.768850  9946 consensus_peers.cc:597] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.784459   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:23.792618 10536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:23.796077 10537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:23.848945  9872 consensus_peers.cc:597] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:23.853015  9946 consensus_peers.cc:597] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:23.932296   431 server_base.cc:1061] running on GCE node
W20260709 13:05:23.937809 10539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:23.939688   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:23.940013   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:23.940285   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602323940265 us; error 0 us; skew 500 ppm
I20260709 13:05:23.941228   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:23.959713   431 webserver.cc:533] Webserver started at http://127.0.107.193:36667/ using document root <none> and password file <none>
I20260709 13:05:23.960402   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:23.960656   431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:05:23.966003   431 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:05:24.009934 10544 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:24.011412   431 fs_manager.cc:730] Time spent opening block manager: real 0.026s	user 0.009s	sys 0.012s
I20260709 13:05:24.011876   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-0-root
uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
format_stamp: "Formatted at 2026-07-09 13:05:09 on dist-test-slave-cq4t"
I20260709 13:05:24.012997   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602014632082-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:24.038187   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:24.040063   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:24.043841   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:24.074429 10551 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 13:05:24.424542   431 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:05:24.424903   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.368s	user 0.012s	sys 0.011s
I20260709 13:05:24.425242   431 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:05:24.435393 10551 tablet_bootstrap.cc:492] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:24.445689 10299 raft_consensus.cc:493] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.446336 10299 raft_consensus.cc:515] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:24.449287 10299 leader_election.cc:290] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:24.452056  9829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0e147704a7413397055b8d3a812a8e" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
W20260709 13:05:24.457084  9946 leader_election.cc:336] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.457702  9946 leader_election.cc:304] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:24.459120 10299 raft_consensus.cc:2749] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.489544 10235 raft_consensus.cc:493] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.490185 10235 raft_consensus.cc:515] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:24.492092 10235 leader_election.cc:290] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.494418  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2d12d5566b4c82a5572035a797a82d" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:24.503865 10531 raft_consensus.cc:493] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
W20260709 13:05:24.503922  9798 leader_election.cc:336] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.504478 10531 raft_consensus.cc:515] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.504794  9798 leader_election.cc:304] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.505919 10235 raft_consensus.cc:2749] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.506569 10531 leader_election.cc:290] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.508071  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716c25a1e46547c999c1c7c860e23f3b" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:24.516515  9872 leader_election.cc:336] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.517252  9872 leader_election.cc:304] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.517979 10551 tablet_bootstrap.cc:492] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:24.518066 10353 raft_consensus.cc:2749] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.519440 10551 tablet_bootstrap.cc:492] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:24.519493 10328 raft_consensus.cc:493] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.520236 10551 ts_tablet_manager.cc:1403] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.085s	user 0.046s	sys 0.028s
I20260709 13:05:24.520092 10328 raft_consensus.cc:515] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.522256 10328 leader_election.cc:290] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.523092 10551 raft_consensus.cc:359] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.523787 10551 raft_consensus.cc:740] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.524628 10551 consensus_queue.cc:260] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.528803  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716c25a1e46547c999c1c7c860e23f3b" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
W20260709 13:05:24.529368  9946 leader_election.cc:336] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.529701  9909 raft_consensus.cc:2468] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:24.534571 10299 raft_consensus.cc:493] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.535188 10299 raft_consensus.cc:515] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.537484 10299 leader_election.cc:290] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.539479  9949 leader_election.cc:304] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.540972 10488 raft_consensus.cc:2804] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:24.541322 10488 raft_consensus.cc:493] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.541637 10488 raft_consensus.cc:3060] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.548220 10328 raft_consensus.cc:493] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.549022 10328 raft_consensus.cc:515] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.552882  9902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778ba1e8c7b04438a6633db058c14508" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
W20260709 13:05:24.553965  9946 leader_election.cc:336] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.555356  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "168ffc86a17d47e8aba438779717eab6" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:24.557308  9949 leader_election.cc:304] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.560400 10299 raft_consensus.cc:2749] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.558578 10488 raft_consensus.cc:515] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.561939 10328 leader_election.cc:290] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
W20260709 13:05:24.571686  9946 leader_election.cc:336] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.572335  9946 leader_election.cc:304] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.573237 10328 raft_consensus.cc:2749] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.573558 10551 ts_tablet_manager.cc:1434] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.053s	user 0.020s	sys 0.031s
I20260709 13:05:24.574606 10551 tablet_bootstrap.cc:492] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:24.574792  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716c25a1e46547c999c1c7c860e23f3b" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:24.575410  9909 raft_consensus.cc:3060] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.583395  9909 raft_consensus.cc:2468] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:24.584707  9949 leader_election.cc:304] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 
I20260709 13:05:24.585635 10328 raft_consensus.cc:2804] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:24.586170 10328 raft_consensus.cc:697] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:24.587149 10328 consensus_queue.cc:237] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.592798 10299 raft_consensus.cc:493] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.593483 10299 raft_consensus.cc:515] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.595790 10488 leader_election.cc:290] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.595964 10299 leader_election.cc:290] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.599232  9902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:24.607339 10551 tablet_bootstrap.cc:492] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:24.608387 10551 tablet_bootstrap.cc:492] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:24.610739 10551 ts_tablet_manager.cc:1403] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.008s
I20260709 13:05:24.618762 10551 raft_consensus.cc:359] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:24.619508 10551 raft_consensus.cc:740] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.620277 10551 consensus_queue.cc:260] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:24.622224 10551 ts_tablet_manager.cc:1434] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:05:24.623701 10551 tablet_bootstrap.cc:492] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:24.631301 10235 raft_consensus.cc:493] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.631919 10235 raft_consensus.cc:515] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:24.633908 10235 leader_election.cc:290] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.635581  9977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:24.635046  9946 leader_election.cc:336] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:24.641882  9946 leader_election.cc:336] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:24.643916  9798 leader_election.cc:336] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.644593 10235 raft_consensus.cc:493] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.651263 10235 raft_consensus.cc:515] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:24.653605 10235 leader_election.cc:290] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.656410  9799 leader_election.cc:304] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.658118 10429 raft_consensus.cc:2749] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.658802 10429 raft_consensus.cc:493] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.659183  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0e147704a7413397055b8d3a812a8e" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:24.659317 10429 raft_consensus.cc:515] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:24.659920  9979 raft_consensus.cc:2468] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:24.660992  9946 leader_election.cc:304] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.662263  9799 leader_election.cc:304] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:24.663318 10429 leader_election.cc:290] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:24.665023  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41e6a20c04a4d99ba9b1ea99106480a" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:24.666287 10328 raft_consensus.cc:2749] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:24.669328  9798 leader_election.cc:336] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:24.669894  9798 leader_election.cc:336] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.670266 10429 raft_consensus.cc:2804] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:24.670291  9798 leader_election.cc:304] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.670855 10429 raft_consensus.cc:493] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.671293 10429 raft_consensus.cc:3060] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.671005  9658 catalog_manager.cc:5697] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 1 to 2, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:24.674680 10235 raft_consensus.cc:2749] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.687908 10353 raft_consensus.cc:493] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.688755 10353 raft_consensus.cc:515] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.690773 10429 raft_consensus.cc:515] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:24.691978 10408 raft_consensus.cc:493] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.692755 10408 raft_consensus.cc:515] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.695870  9977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d69b492d298d42e18ab6b82be92774ca" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:24.696658 10513 raft_consensus.cc:493] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.697216 10513 raft_consensus.cc:515] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
W20260709 13:05:24.698086  9798 leader_election.cc:336] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.707278  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0e147704a7413397055b8d3a812a8e" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:24.708190  9979 raft_consensus.cc:3060] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.711941 10353 leader_election.cc:290] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.703402 10429 leader_election.cc:290] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.714951 10513 leader_election.cc:290] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.724655  9977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:24.737154  9798 leader_election.cc:336] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:24.738075  9872 leader_election.cc:336] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.738579  9872 leader_election.cc:304] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.739980 10535 raft_consensus.cc:2749] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.741706 10408 leader_election.cc:290] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.750396 10408 raft_consensus.cc:493] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.751101 10408 raft_consensus.cc:515] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.807293 10513 raft_consensus.cc:493] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.808177 10513 raft_consensus.cc:515] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:24.816244 10299 raft_consensus.cc:493] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.816895 10299 raft_consensus.cc:515] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.819068 10299 leader_election.cc:290] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.836625 10551 tablet_bootstrap.cc:492] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:24.837208 10408 leader_election.cc:290] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.837785 10551 tablet_bootstrap.cc:492] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:24.838645 10551 ts_tablet_manager.cc:1403] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.215s	user 0.038s	sys 0.082s
W20260709 13:05:24.840086  9946 leader_election.cc:336] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.840018  9977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b0376b68ba34b8596a5ad96711a0c6e" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:24.843796 10299 raft_consensus.cc:493] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.843925 10531 raft_consensus.cc:493] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.844352 10299 raft_consensus.cc:515] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.845434  9799 leader_election.cc:304] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.846249 10235 raft_consensus.cc:2749] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.846793 10299 leader_election.cc:290] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.844553 10531 raft_consensus.cc:515] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
W20260709 13:05:24.847280  9872 leader_election.cc:336] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.849255 10531 leader_election.cc:290] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.851361 10513 leader_election.cc:290] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
W20260709 13:05:24.851981  9946 leader_election.cc:336] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:24.857447  9798 leader_election.cc:336] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.857992 10299 raft_consensus.cc:493] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.858596 10299 raft_consensus.cc:515] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:24.860805 10299 leader_election.cc:290] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:24.864624  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778ba1e8c7b04438a6633db058c14508" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:24.865283  9974 raft_consensus.cc:2468] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
W20260709 13:05:24.867458  9946 leader_election.cc:336] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.867830  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "755ff96760b04fd49a520c059f90b1fe" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:24.869958  9829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:24.870481 10328 raft_consensus.cc:493] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.870635  9829 raft_consensus.cc:2468] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:24.871189 10328 raft_consensus.cc:515] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:24.873504 10328 leader_election.cc:290] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:24.875140  9873 leader_election.cc:304] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.875923 10531 raft_consensus.cc:2749] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:24.880739  9946 leader_election.cc:336] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.888062  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:24.888831  9974 raft_consensus.cc:2468] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 1.
I20260709 13:05:24.888092 10328 raft_consensus.cc:493] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.890002 10328 raft_consensus.cc:515] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.890337  9799 leader_election.cc:304] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.858045 10535 raft_consensus.cc:493] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.891673 10429 raft_consensus.cc:2804] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:24.892066 10328 leader_election.cc:290] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.852658 10551 raft_consensus.cc:359] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.892119 10429 raft_consensus.cc:493] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.893173 10429 raft_consensus.cc:3060] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:24.894551  9874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 183821us
W20260709 13:05:24.897269  9946 leader_election.cc:336] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.899310  9979 raft_consensus.cc:2468] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 2.
I20260709 13:05:24.899154 10328 raft_consensus.cc:493] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.899996 10328 raft_consensus.cc:515] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:24.903493  9948 leader_election.cc:304] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.904176  9799 leader_election.cc:304] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
W20260709 13:05:24.904464  9872 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 56124.6us
I20260709 13:05:24.905450 10235 raft_consensus.cc:2804] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:05:24.908874  9872 leader_election.cc:336] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:24.909250  9946 leader_election.cc:336] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.910224 10488 raft_consensus.cc:2804] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:24.910686 10488 raft_consensus.cc:493] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.911182 10488 raft_consensus.cc:3060] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.892794 10551 raft_consensus.cc:740] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:24.913496 10551 consensus_queue.cc:260] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.914542  9873 leader_election.cc:304] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.916190 10353 raft_consensus.cc:2804] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:24.916580 10353 raft_consensus.cc:493] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.916925 10353 raft_consensus.cc:3060] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.917484 10299 raft_consensus.cc:493] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.918121 10299 raft_consensus.cc:515] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.892120 10535 raft_consensus.cc:515] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.921144 10531 raft_consensus.cc:493] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.921736 10531 raft_consensus.cc:515] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.923878 10531 leader_election.cc:290] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
W20260709 13:05:24.925848  9946 leader_election.cc:336] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:24.925969 10299 leader_election.cc:290] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.925853 10551 ts_tablet_manager.cc:1434] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.087s	user 0.007s	sys 0.000s
I20260709 13:05:24.927835  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46065991d847dfa3d934961da0830a" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:24.928166 10551 tablet_bootstrap.cc:492] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:24.928056  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2d12d5566b4c82a5572035a797a82d" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:24.928511  9909 raft_consensus.cc:2468] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:24.928762  9831 raft_consensus.cc:2468] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:24.931447  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:24.930639  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:24.932047  9909 raft_consensus.cc:2468] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:24.932510  9831 raft_consensus.cc:2468] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:24.935317  9904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d69b492d298d42e18ab6b82be92774ca" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:24.936036  9904 raft_consensus.cc:2468] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:24.937534  9904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "755ff96760b04fd49a520c059f90b1fe" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:24.939798  9949 leader_election.cc:304] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.941675 10353 raft_consensus.cc:515] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.948210 10353 leader_election.cc:290] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.937441 10429 raft_consensus.cc:515] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:24.938263  9904 raft_consensus.cc:2468] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:24.910667 10328 leader_election.cc:290] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:24.975842 10488 raft_consensus.cc:515] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:24.978180 10235 raft_consensus.cc:697] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 2 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:24.981805 10488 leader_election.cc:290] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:24.982839 10353 raft_consensus.cc:493] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.983554 10353 raft_consensus.cc:515] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:24.938254 10535 leader_election.cc:290] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.987767 10429 leader_election.cc:290] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.981858 10235 consensus_queue.cc:237] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:24.990523  9949 leader_election.cc:304] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.992111  9949 leader_election.cc:304] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:24.990523 10408 raft_consensus.cc:493] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:24.996345 10408 raft_consensus.cc:515] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.000762  9949 leader_election.cc:304] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.003613 10353 leader_election.cc:290] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:24.967193 10565 raft_consensus.cc:2804] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.006275 10328 raft_consensus.cc:2804] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.009084 10328 raft_consensus.cc:493] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.009490 10328 raft_consensus.cc:3060] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.007773 10488 raft_consensus.cc:2804] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.012986 10408 leader_election.cc:290] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:25.024264 10566 raft_consensus.cc:493] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.025085 10566 raft_consensus.cc:515] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
W20260709 13:05:25.017868  9948 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 75312.6us
I20260709 13:05:25.013028 10565 raft_consensus.cc:493] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.005064 10567 raft_consensus.cc:2804] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:05:25.042461  9798 leader_election.cc:336] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.045565 10567 raft_consensus.cc:493] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.046023 10567 raft_consensus.cc:3060] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.019680 10488 raft_consensus.cc:493] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.047256 10488 raft_consensus.cc:3060] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.029453 10408 raft_consensus.cc:493] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.052232 10535 raft_consensus.cc:493] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.052013 10408 raft_consensus.cc:515] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
W20260709 13:05:25.054528  9872 leader_election.cc:336] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.053090 10535 raft_consensus.cc:515] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:25.055075  9872 leader_election.cc:304] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
W20260709 13:05:25.058434  9946 leader_election.cc:336] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.059644 10566 leader_election.cc:290] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:25.060140 10408 leader_election.cc:290] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:25.064864 10328 raft_consensus.cc:515] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.065706 10535 leader_election.cc:290] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:25.044898 10565 raft_consensus.cc:3060] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.070758 10353 raft_consensus.cc:2749] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:25.078392  9872 leader_election.cc:336] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:25.078637  9946 leader_election.cc:336] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:25.081095  9874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 51996.9us
W20260709 13:05:25.088621  9872 leader_election.cc:336] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:25.089279  9872 leader_election.cc:336] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.092273 10299 raft_consensus.cc:493] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.092885  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:25.092847 10299 raft_consensus.cc:515] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.093703  9979 raft_consensus.cc:2393] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a1cbca39e34743d8a40816c3bc5f2030 in current term 2: Already voted for candidate 23cbd6234c5e43d89f22c235ed6bb96a in this term.
I20260709 13:05:25.096232 10299 leader_election.cc:290] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:25.096309  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2684c424702483b92758b1173702381" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:25.096995  9909 raft_consensus.cc:2468] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:25.097043 10328 leader_election.cc:290] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:25.090327 10488 raft_consensus.cc:515] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.098383  9904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d69b492d298d42e18ab6b82be92774ca" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:25.099057  9904 raft_consensus.cc:3060] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.100412 10567 raft_consensus.cc:515] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.102564 10488 leader_election.cc:290] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
W20260709 13:05:25.103178  9872 leader_election.cc:336] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:25.103751  9872 leader_election.cc:336] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.103804 10565 raft_consensus.cc:515] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
W20260709 13:05:25.105173  9946 leader_election.cc:336] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:25.105840  9946 consensus_peers.cc:597] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:25.105991 10565 leader_election.cc:290] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:25.108502  9658 catalog_manager.cc:5697] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 1 to 2, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.111320  9949 leader_election.cc:304] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 
I20260709 13:05:25.113309  9909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "755ff96760b04fd49a520c059f90b1fe" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:25.114107  9909 raft_consensus.cc:3060] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.110872  9902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:25.114315  9906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46065991d847dfa3d934961da0830a" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:25.114733  9902 raft_consensus.cc:3060] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:25.118645  9872 leader_election.cc:336] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:25.119320  9872 leader_election.cc:336] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.121739  9906 raft_consensus.cc:3060] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:25.134608  9946 leader_election.cc:336] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:25.147387  9948 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 111943us
I20260709 13:05:25.148236  9948 leader_election.cc:304] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.148996 10566 raft_consensus.cc:2804] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.149369 10566 raft_consensus.cc:493] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.149749 10566 raft_consensus.cc:3060] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.154565  9906 raft_consensus.cc:2468] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:25.140695  9902 raft_consensus.cc:2468] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:25.159286  9904 raft_consensus.cc:2468] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:25.159601  9949 leader_election.cc:304] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 
I20260709 13:05:25.160518  9949 leader_election.cc:304] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.161497 10488 raft_consensus.cc:2804] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.161965 10488 raft_consensus.cc:697] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:25.163095 10328 raft_consensus.cc:2804] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.163618 10328 raft_consensus.cc:697] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:25.164448 10328 consensus_queue.cc:237] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.167996  9949 leader_election.cc:304] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 
I20260709 13:05:25.168025  9948 leader_election.cc:304] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.162777 10488 consensus_queue.cc:237] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.170271  9909 raft_consensus.cc:2468] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:25.173468  9799 leader_election.cc:304] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.174163 10235 raft_consensus.cc:2749] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:25.139192 10565 raft_consensus.cc:2804] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.175781  9949 leader_election.cc:304] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 
I20260709 13:05:25.182979  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b0376b68ba34b8596a5ad96711a0c6e" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:25.183739  9835 raft_consensus.cc:2468] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:25.146795 10567 leader_election.cc:290] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:25.186560  9829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41e6a20c04a4d99ba9b1ea99106480a" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:25.187314  9829 raft_consensus.cc:2468] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:25.187822  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:25.188580  9831 raft_consensus.cc:2393] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 23cbd6234c5e43d89f22c235ed6bb96a in current term 2: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
I20260709 13:05:25.179970 10568 raft_consensus.cc:493] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.192770 10568 raft_consensus.cc:515] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.193612  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe46065991d847dfa3d934961da0830a" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:25.194518  9946 leader_election.cc:336] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.182432 10574 raft_consensus.cc:2804] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.197021 10551 tablet_bootstrap.cc:492] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:05:25.197091  9946 leader_election.cc:336] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.182468 10299 raft_consensus.cc:2804] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.181564  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:25.209340 10299 raft_consensus.cc:493] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.175848 10565 raft_consensus.cc:493] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.211261  9982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2684c424702483b92758b1173702381" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:25.213282  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a981502782314272a8a9bee8da734427" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:25.215099  9979 raft_consensus.cc:2468] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:25.179270 10573 raft_consensus.cc:2804] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.175314 10566 raft_consensus.cc:515] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:25.195741 10568 leader_election.cc:290] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:25.212402  9978 raft_consensus.cc:2468] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:25.222203  9902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a981502782314272a8a9bee8da734427" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:25.214951 10299 raft_consensus.cc:3060] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.224079  9874 leader_election.cc:304] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.218737 10573 raft_consensus.cc:697] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:25.219563 10551 tablet_bootstrap.cc:492] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:25.227448 10574 raft_consensus.cc:697] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:25.234556 10353 raft_consensus.cc:2804] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.246029 10353 raft_consensus.cc:493] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.246413 10353 raft_consensus.cc:3060] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.235980 10551 ts_tablet_manager.cc:1403] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.308s	user 0.021s	sys 0.012s
I20260709 13:05:25.216323 10565 raft_consensus.cc:3060] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.237288  9948 leader_election.cc:304] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
W20260709 13:05:25.238521  9946 leader_election.cc:336] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.252202  9948 leader_election.cc:304] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:25.230446  9976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778ba1e8c7b04438a6633db058c14508" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:25.237047 10574 consensus_queue.cc:237] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.229593  9902 raft_consensus.cc:2468] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 1.
I20260709 13:05:25.235133 10573 consensus_queue.cc:237] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.227502  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "168ffc86a17d47e8aba438779717eab6" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:25.255957  9976 raft_consensus.cc:3060] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.267680 10328 raft_consensus.cc:2804] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.268369 10328 raft_consensus.cc:493] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.268858 10328 raft_consensus.cc:3060] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.239101  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2d12d5566b4c82a5572035a797a82d" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:25.271854  9831 raft_consensus.cc:3060] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.228852 10566 leader_election.cc:290] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:25.238157  9873 leader_election.cc:304] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.277262  9873 leader_election.cc:304] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.256140 10551 raft_consensus.cc:359] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:25.281438 10551 raft_consensus.cc:740] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:25.292290  9873 leader_election.cc:304] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.294381 10551 consensus_queue.cc:260] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:25.301268 10353 raft_consensus.cc:515] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:25.263538  9974 raft_consensus.cc:2468] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 1.
I20260709 13:05:25.308096 10299 raft_consensus.cc:515] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:25.317121  9949 leader_election.cc:304] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 
I20260709 13:05:25.320354 10581 raft_consensus.cc:2804] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.320878 10581 raft_consensus.cc:493] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.321343 10581 raft_consensus.cc:3060] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.302073 10535 raft_consensus.cc:2749] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:25.302615 10568 raft_consensus.cc:2749] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:25.306492 10531 raft_consensus.cc:2804] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.334403 10551 ts_tablet_manager.cc:1434] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.087s	user 0.008s	sys 0.000s
I20260709 13:05:25.335402 10551 tablet_bootstrap.cc:492] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:25.330802 10299 leader_election.cc:290] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:25.346148 10581 raft_consensus.cc:515] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
W20260709 13:05:25.348007  9946 leader_election.cc:336] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:25.348814  9946 leader_election.cc:336] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.353029  9655 catalog_manager.cc:5697] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 1 to 2, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:25.354746  9872 leader_election.cc:336] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.332764 10488 raft_consensus.cc:2804] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.358163 10488 raft_consensus.cc:493] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.356170 10328 raft_consensus.cc:515] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:25.358631 10488 raft_consensus.cc:3060] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.374421  9655 catalog_manager.cc:5697] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 1 to 2, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:25.382258  9946 leader_election.cc:336] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.384336 10353 leader_election.cc:290] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
W20260709 13:05:25.388530  9873 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 67968.7us
I20260709 13:05:25.396634 10328 leader_election.cc:290] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:25.334017 10531 raft_consensus.cc:493] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.404090  9832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d41e6a20c04a4d99ba9b1ea99106480a" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:25.411183  9831 raft_consensus.cc:2468] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
W20260709 13:05:25.416646  9946 leader_election.cc:336] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.417685  9976 raft_consensus.cc:2468] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 2.
I20260709 13:05:25.425876  9873 leader_election.cc:304] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.424835  9832 raft_consensus.cc:3060] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.423070 10531 raft_consensus.cc:3060] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.436051  9829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:25.436750  9829 raft_consensus.cc:3060] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.438488 10570 raft_consensus.cc:2804] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.439002 10570 raft_consensus.cc:493] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:25.439383 10570 raft_consensus.cc:3060] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.444254  9830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b0376b68ba34b8596a5ad96711a0c6e" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:25.444940  9830 raft_consensus.cc:3060] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.453775  9974 raft_consensus.cc:1275] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Refusing update from remote peer a1cbca39e34743d8a40816c3bc5f2030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:25.456297 10575 consensus_queue.cc:1048] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:25.458997  9832 raft_consensus.cc:2468] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 2.
I20260709 13:05:25.460850  9874 leader_election.cc:304] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.462724 10584 raft_consensus.cc:2804] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.463318 10584 raft_consensus.cc:697] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:25.465569 10584 consensus_queue.cc:237] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:25.474501  9902 raft_consensus.cc:1275] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:25.478972 10565 raft_consensus.cc:515] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.480244  9982 raft_consensus.cc:2393] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e10aab26c644389b8161974dcabc5d0 in current term 2: Already voted for candidate 23cbd6234c5e43d89f22c235ed6bb96a in this term.
I20260709 13:05:25.485929 10583 consensus_queue.cc:1048] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:25.490077 10531 raft_consensus.cc:515] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.492174   431 ts_tablet_manager.cc:600] Registering tablets (43/45 complete)
I20260709 13:05:25.505260  9902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2684c424702483b92758b1173702381" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:25.506017  9902 raft_consensus.cc:3060] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.509085 10581 leader_election.cc:290] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
W20260709 13:05:25.512326  9798 consensus_peers.cc:597] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:25.507848 10488 raft_consensus.cc:515] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.513602  9902 raft_consensus.cc:2468] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:25.514631 10488 leader_election.cc:290] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
W20260709 13:05:25.528595  9946 leader_election.cc:336] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.490717 10565 leader_election.cc:290] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:25.530165 10570 raft_consensus.cc:515] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.538957 10570 leader_election.cc:290] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
W20260709 13:05:25.542366 10586 log.cc:927] Time spent T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:05:25.544293 10531 leader_election.cc:290] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:25.556690   431 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:05:25.557137   431 ts_tablet_manager.cc:595] Time spent register tablets: real 1.132s	user 0.320s	sys 0.038s
I20260709 13:05:25.568851 10551 tablet_bootstrap.cc:492] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:25.570251 10551 tablet_bootstrap.cc:492] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:25.570994 10551 ts_tablet_manager.cc:1403] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.236s	user 0.022s	sys 0.003s
I20260709 13:05:25.579003 10551 raft_consensus.cc:359] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:25.579775 10551 raft_consensus.cc:740] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:25.581043 10551 consensus_queue.cc:260] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:25.583434 10551 ts_tablet_manager.cc:1434] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:05:25.584340 10551 tablet_bootstrap.cc:492] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:25.680204 10551 tablet_bootstrap.cc:492] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:25.681272 10551 tablet_bootstrap.cc:492] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:25.682013 10551 ts_tablet_manager.cc:1403] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.098s	user 0.021s	sys 0.000s
I20260709 13:05:25.686138 10551 raft_consensus.cc:359] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:25.686872 10551 raft_consensus.cc:740] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:25.687781 10551 consensus_queue.cc:260] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:25.690121 10551 ts_tablet_manager.cc:1434] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:05:25.691035 10551 tablet_bootstrap.cc:492] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:25.696539 10554 raft_consensus.cc:493] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.697321 10554 raft_consensus.cc:515] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:25.707155 10554 leader_election.cc:290] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:25.737390  9830 raft_consensus.cc:2468] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
W20260709 13:05:25.740492  9872 leader_election.cc:336] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:25.749516  9872 leader_election.cc:336] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:25.750072  9872 leader_election.cc:336] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.759197  9829 raft_consensus.cc:2468] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:25.793934  9655 catalog_manager.cc:5697] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 1 to 2, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:25.847966  9948 leader_election.cc:304] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.857288 10488 raft_consensus.cc:2804] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.863811  9949 leader_election.cc:304] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
W20260709 13:05:25.942646  9949 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:45019, user_credentials={real_user=slave}} blocked reactor thread for 78953.4us
I20260709 13:05:25.964339 10551 tablet_bootstrap.cc:492] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:05:25.980609  9946 leader_election.cc:336] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:25.983836  9948 leader_election.cc:304] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.984974  9948 leader_election.cc:304] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.942097 10488 raft_consensus.cc:697] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:25.990849  9873 leader_election.cc:304] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.994019  9909 raft_consensus.cc:1275] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:25.957976 10574 raft_consensus.cc:2804] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.995728  9873 leader_election.cc:304] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:25.988715 10568 raft_consensus.cc:2804] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.997601 10570 raft_consensus.cc:2804] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:26.008869 10551 tablet_bootstrap.cc:492] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:26.010331 10551 ts_tablet_manager.cc:1403] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.320s	user 0.022s	sys 0.002s
I20260709 13:05:25.996723 10566 raft_consensus.cc:2804] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:26.019565 10328 consensus_queue.cc:1048] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:26.000284  9904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a981502782314272a8a9bee8da734427" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:26.025084  9906 raft_consensus.cc:1275] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:26.025660  9901 raft_consensus.cc:1275] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:26.032421  9798 consensus_peers.cc:597] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.033190  9798 consensus_peers.cc:597] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.004554 10574 raft_consensus.cc:697] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:26.043793 10328 consensus_queue.cc:1048] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:26.042788  9946 consensus_peers.cc:597] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:26.053076  9946 consensus_peers.cc:597] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.058040 10328 consensus_queue.cc:1048] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:26.060464 10551 raft_consensus.cc:359] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:26.063078 10551 raft_consensus.cc:740] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.016237 10568 raft_consensus.cc:697] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:26.016467 10570 raft_consensus.cc:697] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
W20260709 13:05:26.082114  9872 consensus_peers.cc:597] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.082222 10570 consensus_queue.cc:237] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
W20260709 13:05:26.083082  9872 consensus_peers.cc:597] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.083585  9872 consensus_peers.cc:597] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.083988  9872 consensus_peers.cc:597] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.084386  9872 consensus_peers.cc:597] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.048558 10574 consensus_queue.cc:237] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
W20260709 13:05:26.076933  9798 consensus_peers.cc:597] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.081218  9904 raft_consensus.cc:2393] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 23cbd6234c5e43d89f22c235ed6bb96a in current term 2: Already voted for candidate 6e10aab26c644389b8161974dcabc5d0 in this term.
I20260709 13:05:26.034838 10566 raft_consensus.cc:697] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:26.104040  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "168ffc86a17d47e8aba438779717eab6" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:26.020309 10616 raft_consensus.cc:2749] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.109969  9979 raft_consensus.cc:3060] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.064772 10551 consensus_queue.cc:260] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:26.148455 10551 ts_tablet_manager.cc:1434] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.138s	user 0.008s	sys 0.000s
I20260709 13:05:26.147753  9949 leader_election.cc:304] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:26.153192 10328 raft_consensus.cc:2749] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.168192 10551 tablet_bootstrap.cc:492] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:26.169430  9979 raft_consensus.cc:2468] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 2.
I20260709 13:05:26.175545  9873 leader_election.cc:304] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:26.162441 10554 raft_consensus.cc:493] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.182575 10568 consensus_queue.cc:237] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:26.201225  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a981502782314272a8a9bee8da734427" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:26.202358  9980 raft_consensus.cc:2393] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e10aab26c644389b8161974dcabc5d0 in current term 2: Already voted for candidate 23cbd6234c5e43d89f22c235ed6bb96a in this term.
I20260709 13:05:26.204773 10566 consensus_queue.cc:237] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
W20260709 13:05:26.207620  9872 consensus_peers.cc:597] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.208123 10554 raft_consensus.cc:515] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
W20260709 13:05:26.210217  9798 consensus_peers.cc:597] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.113355 10488 consensus_queue.cc:237] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
W20260709 13:05:26.227699  9801 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45019, user_credentials={real_user=slave}} blocked reactor thread for 57018.8us
I20260709 13:05:26.182457 10616 raft_consensus.cc:2804] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:05:26.233186  9946 consensus_peers.cc:597] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.234071  9946 consensus_peers.cc:597] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:26.234616  9946 consensus_peers.cc:597] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.238829  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:26.251368 10616 raft_consensus.cc:697] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:26.382265  9908 raft_consensus.cc:2393] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 2: Already voted for candidate 6e10aab26c644389b8161974dcabc5d0 in this term.
I20260709 13:05:26.387717 10580 raft_consensus.cc:493] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e10aab26c644389b8161974dcabc5d0)
W20260709 13:05:26.443055  9798 consensus_peers.cc:597] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.427276  9830 raft_consensus.cc:1275] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 6e10aab26c644389b8161974dcabc5d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:26.423259 10580 raft_consensus.cc:515] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:26.420341 10616 consensus_queue.cc:237] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:26.519451  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:26.519814 10554 leader_election.cc:290] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:26.520239  9978 raft_consensus.cc:3060] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:26.524052 10636 log.cc:927] Time spent T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:05:26.526211 10630 log.cc:927] Time spent T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:05:26.534405 10580 leader_election.cc:290] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
W20260709 13:05:26.537019  9799 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 59644.6us
W20260709 13:05:26.548559  9946 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 58120.2us
I20260709 13:05:26.612428  9905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912bd6d0efac4042884ce6823f7e8e73" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:26.581936  9978 raft_consensus.cc:2468] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 2.
I20260709 13:05:26.629638  9873 leader_election.cc:304] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:26.638228 10559 raft_consensus.cc:493] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a1cbca39e34743d8a40816c3bc5f2030)
I20260709 13:05:26.638948 10559 raft_consensus.cc:515] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:26.641610 10559 leader_election.cc:290] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:26.642091  9900 raft_consensus.cc:1275] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:26.645103 10531 consensus_queue.cc:1048] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:26.645854 10570 raft_consensus.cc:493] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a1cbca39e34743d8a40816c3bc5f2030)
I20260709 13:05:26.646437 10570 raft_consensus.cc:515] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:26.648670 10570 leader_election.cc:290] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:26.650194  9830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912bd6d0efac4042884ce6823f7e8e73" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:26.663589  9873 leader_election.cc:304] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:26.671831  9829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "196922ea9dc14e078a3da6af1206f79a" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:26.661101 10616 raft_consensus.cc:2749] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.673528  9830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d7bb9a0b174b168df9040b02c70ab9" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
W20260709 13:05:26.702864  9798 consensus_peers.cc:597] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.703689  9798 consensus_peers.cc:597] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.708009 10554 raft_consensus.cc:493] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.708841 10554 raft_consensus.cc:515] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
W20260709 13:05:26.710179  9946 consensus_peers.cc:597] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.710783  9946 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 127208us
I20260709 13:05:26.716981 10554 leader_election.cc:290] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
W20260709 13:05:26.730389  9798 consensus_peers.cc:597] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.732813  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d7bb9a0b174b168df9040b02c70ab9" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:26.751430 10644 raft_consensus.cc:2804] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:26.752948  9977 raft_consensus.cc:1275] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Refusing update from remote peer 6e10aab26c644389b8161974dcabc5d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:26.755343 10551 tablet_bootstrap.cc:492] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:26.756381 10551 tablet_bootstrap.cc:492] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:26.759266  9949 leader_election.cc:304] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:26.758829  9655 catalog_manager.cc:5697] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 1 to 2, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.760673  9655 catalog_manager.cc:5697] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 1 to 2, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } }
I20260709 13:05:26.761626 10574 raft_consensus.cc:2749] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.762887 10644 raft_consensus.cc:697] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:26.763132 10328 consensus_queue.cc:1048] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:26.763345 10004 debug-util.cc:398] Leaking SignalData structure 0x7b0800227ee0 after lost signal to thread 434
I20260709 13:05:26.768999  9902 raft_consensus.cc:1275] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:26.772289 10551 ts_tablet_manager.cc:1403] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.616s	user 0.042s	sys 0.010s
I20260709 13:05:26.763828 10644 consensus_queue.cc:237] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:26.793390 10554 raft_consensus.cc:493] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:26.807377  9946 consensus_peers.cc:597] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.808079  9946 consensus_peers.cc:597] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.775398 10573 consensus_queue.cc:1048] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:05:26.811676 10551 raft_consensus.cc:359] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:26.812426 10551 raft_consensus.cc:740] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:26.813123 10551 consensus_queue.cc:260] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
W20260709 13:05:26.801337 10004 debug-util.cc:398] Leaking SignalData structure 0x7b08003e9000 after lost signal to thread 9692
W20260709 13:05:26.816838  9872 consensus_peers.cc:597] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.817605  9872 consensus_peers.cc:597] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.818277  9872 consensus_peers.cc:597] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.818813  9872 consensus_peers.cc:597] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.806602 10554 raft_consensus.cc:515] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
W20260709 13:05:26.830396  9872 leader_election.cc:336] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:26.830861  9872 leader_election.cc:304] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:26.831012 10554 leader_election.cc:290] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:26.833609  9903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3505eea4224942d7a938f9eaf7427723" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
W20260709 13:05:26.842897 10004 debug-util.cc:398] Leaking SignalData structure 0x7b0800120a80 after lost signal to thread 9859
W20260709 13:05:26.853318 10004 debug-util.cc:398] Leaking SignalData structure 0x7b08001fb500 after lost signal to thread 9933
W20260709 13:05:26.856372 10004 debug-util.cc:398] Leaking SignalData structure 0x7b0800189800 after lost signal to thread 10007
I20260709 13:05:26.862702  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf1d483dd4049a68642f7d94f1cfc70" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
W20260709 13:05:26.868525  9798 consensus_peers.cc:597] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:26.868981  9798 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 137969us
W20260709 13:05:26.869808  9798 consensus_peers.cc:597] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.877786 10551 ts_tablet_manager.cc:1434] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.105s	user 0.004s	sys 0.018s
I20260709 13:05:26.878762 10551 tablet_bootstrap.cc:492] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:26.894659  9873 leader_election.cc:304] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030
W20260709 13:05:26.907229  9946 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 86258us
I20260709 13:05:26.961100  9833 raft_consensus.cc:1275] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:26.962785  9873 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 55940.3us
I20260709 13:05:26.941560  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
W20260709 13:05:26.984588  9946 consensus_peers.cc:597] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:26.985307  9946 consensus_peers.cc:597] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:26.985846  9946 consensus_peers.cc:597] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:26.930068 10559 consensus_queue.cc:1048] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
W20260709 13:05:26.986413  9946 consensus_peers.cc:597] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:26.968787 10566 consensus_queue.cc:1048] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:05:27.002405  9974 raft_consensus.cc:2393] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 2: Already voted for candidate 6e10aab26c644389b8161974dcabc5d0 in this term.
I20260709 13:05:27.003981 10643 raft_consensus.cc:2749] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:27.005220  9946 consensus_peers.cc:597] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:27.005774 10546 leader_election.cc:304] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0
I20260709 13:05:27.007136 10554 raft_consensus.cc:3060] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:27.008438 10571 raft_consensus.cc:2749] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:27.011950 10661 raft_consensus.cc:493] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.012584 10661 raft_consensus.cc:515] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
W20260709 13:05:27.013703  9798 consensus_peers.cc:597] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:27.016815  9827 raft_consensus.cc:1275] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:27.021203 10566 consensus_queue.cc:1048] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:27.026432  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3505eea4224942d7a938f9eaf7427723" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:27.037323  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:27.035640 10661 leader_election.cc:290] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:27.045459  9977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6f94cfb8a734a83872ed245aeea11e6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:27.069453 10554 raft_consensus.cc:2749] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:27.124248  9827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf1d483dd4049a68642f7d94f1cfc70" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
W20260709 13:05:27.048749  9946 consensus_peers.cc:597] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:27.136770  9946 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 87850.9us
W20260709 13:05:27.138024  9946 consensus_peers.cc:597] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:27.139555  9872 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 93545.7us
I20260709 13:05:27.140776  9835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6f94cfb8a734a83872ed245aeea11e6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
W20260709 13:05:27.141151  9872 consensus_peers.cc:597] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:27.145901  9872 consensus_peers.cc:597] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:27.154997  9874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 108553us
I20260709 13:05:27.175500  9975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:27.185478 10615 raft_consensus.cc:493] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.186676 10615 raft_consensus.cc:515] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:27.187975  9826 raft_consensus.cc:1275] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:27.198354 10565 raft_consensus.cc:493] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a1cbca39e34743d8a40816c3bc5f2030)
I20260709 13:05:27.199043 10547 leader_election.cc:304] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:27.199658 10565 raft_consensus.cc:515] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:27.207484 10554 raft_consensus.cc:2749] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:27.231937  9874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 67264.8us
I20260709 13:05:27.232191 10574 consensus_queue.cc:1048] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:05:27.243847 10548 leader_election.cc:304] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0
I20260709 13:05:27.246166 10554 raft_consensus.cc:2749] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:27.248865 10565 leader_election.cc:290] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:27.251304 10547 leader_election.cc:304] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
W20260709 13:05:27.253615 10547 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 50862.2us
I20260709 13:05:27.254865 10554 raft_consensus.cc:2749] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:27.265751  9980 raft_consensus.cc:1275] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Refusing update from remote peer 6e10aab26c644389b8161974dcabc5d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:27.194453 10615 leader_election.cc:290] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:27.274004 10488 raft_consensus.cc:493] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.232661 10551 tablet_bootstrap.cc:492] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:05:27.281721  9872 consensus_peers.cc:597] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.292312 10488 raft_consensus.cc:515] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:27.296089 10551 tablet_bootstrap.cc:492] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:27.297541 10551 ts_tablet_manager.cc:1403] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.419s	user 0.021s	sys 0.004s
W20260709 13:05:27.306133  9798 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 70321.2us
W20260709 13:05:27.308959  9799 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 81503.3us
I20260709 13:05:27.222870  9655 catalog_manager.cc:5697] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 1 to 2, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.268496 10547 leader_election.cc:304] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:27.319499 10554 raft_consensus.cc:2749] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:27.325353  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:27.326432  9974 raft_consensus.cc:2468] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 2.
I20260709 13:05:27.328907 10488 leader_election.cc:290] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
W20260709 13:05:27.334275 10547 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:34259, user_credentials={real_user=slave}} blocked reactor thread for 79392.4us
I20260709 13:05:27.321970  9655 catalog_manager.cc:5697] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 1 to 2, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.355299 10682 consensus_queue.cc:1048] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:27.363955  9872 consensus_peers.cc:597] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:27.401454  9833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:27.402614  9833 raft_consensus.cc:2468] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:27.392756 10554 raft_consensus.cc:493] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.403837 10554 raft_consensus.cc:515] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
W20260709 13:05:27.404917  9946 leader_election.cc:336] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
W20260709 13:05:27.407105  9798 leader_election.cc:336] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:27.412820 10554 leader_election.cc:290] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:27.413618  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23871dd822614a5da07c2289f80141c1" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:27.423166  9948 leader_election.cc:304] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:27.424593 10299 raft_consensus.cc:2804] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:27.425154 10299 raft_consensus.cc:493] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.425644 10299 raft_consensus.cc:3060] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:27.426349  9946 leader_election.cc:336] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:27.340499  9655 catalog_manager.cc:5697] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 1 to 2, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.432494 10551 raft_consensus.cc:359] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:27.438310 10551 raft_consensus.cc:740] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.437601  9826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b189ef5046cd47ebb3d431e2ef24d6fb" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:27.439430 10551 consensus_queue.cc:260] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:27.445982  9948 leader_election.cc:304] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:27.449911 10551 ts_tablet_manager.cc:1434] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.152s	user 0.008s	sys 0.004s
I20260709 13:05:27.451066  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23871dd822614a5da07c2289f80141c1" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:27.453231 10551 tablet_bootstrap.cc:492] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:27.453083 10547 leader_election.cc:304] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:27.454581 10554 raft_consensus.cc:2749] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:27.458606 10328 raft_consensus.cc:2749] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:27.457165  9655 catalog_manager.cc:5697] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 1 to 2, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.466899 10554 raft_consensus.cc:493] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.467840 10554 raft_consensus.cc:515] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:27.476509  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a981502782314272a8a9bee8da734427" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:27.477667  9979 raft_consensus.cc:2393] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 2: Already voted for candidate 23cbd6234c5e43d89f22c235ed6bb96a in this term.
I20260709 13:05:27.470569  9799 leader_election.cc:304] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:27.487905  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a981502782314272a8a9bee8da734427" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:27.489251  9907 raft_consensus.cc:2393] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c8d63af7ee84c38a87435cbaf6deab4 in current term 2: Already voted for candidate 6e10aab26c644389b8161974dcabc5d0 in this term.
I20260709 13:05:27.467980  9655 catalog_manager.cc:5697] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 1 to 2, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.481837 10554 leader_election.cc:290] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:27.500236  9980 raft_consensus.cc:1275] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Refusing update from remote peer 6e10aab26c644389b8161974dcabc5d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:27.506376 10548 leader_election.cc:304] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0
W20260709 13:05:27.517673  9799 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 48564.2us
I20260709 13:05:27.521651 10299 raft_consensus.cc:515] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:27.520382 10682 consensus_queue.cc:1048] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:05:27.539155 10569 raft_consensus.cc:2804] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:27.539765 10569 raft_consensus.cc:493] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.540198 10569 raft_consensus.cc:3060] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:27.528944 10554 raft_consensus.cc:3060] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:27.557371  9946 consensus_peers.cc:597] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:27.569970  9946 consensus_peers.cc:597] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:27.644197 10551 tablet_bootstrap.cc:492] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:27.645329 10551 tablet_bootstrap.cc:492] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:27.646077 10551 ts_tablet_manager.cc:1403] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.193s	user 0.021s	sys 0.005s
I20260709 13:05:27.646453 10299 leader_election.cc:290] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 election: Requested vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:27.647149  9826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:27.649936 10551 raft_consensus.cc:359] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:27.650818 10551 raft_consensus.cc:740] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
W20260709 13:05:27.652047  9946 leader_election.cc:336] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:27.652755 10551 consensus_queue.cc:260] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:27.667680 10551 ts_tablet_manager.cc:1434] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.021s	user 0.013s	sys 0.004s
I20260709 13:05:27.668716 10551 tablet_bootstrap.cc:492] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:27.670377 10569 raft_consensus.cc:515] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:27.688383  9826 raft_consensus.cc:2393] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 23cbd6234c5e43d89f22c235ed6bb96a in current term 3: Already voted for candidate a1cbca39e34743d8a40816c3bc5f2030 in this term.
I20260709 13:05:27.689347 10569 leader_election.cc:290] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 3 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:27.686897 10554 raft_consensus.cc:2749] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:27.694392  9948 leader_election.cc:304] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:27.696235 10488 raft_consensus.cc:2749] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:05:27.712072  9798 leader_election.cc:336] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:27.755931  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:27.759212  9979 raft_consensus.cc:2393] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a1cbca39e34743d8a40816c3bc5f2030 in current term 3: Already voted for candidate 23cbd6234c5e43d89f22c235ed6bb96a in this term.
I20260709 13:05:27.759964 10580 raft_consensus.cc:493] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.760550 10580 raft_consensus.cc:515] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
W20260709 13:05:27.845602 10700 log.cc:927] Time spent T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0: Append to log took a long time: real 0.066s	user 0.005s	sys 0.000s
I20260709 13:05:27.862546  9799 leader_election.cc:304] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1cbca39e34743d8a40816c3bc5f2030; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:27.866024 10575 raft_consensus.cc:2749] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:27.867852 10551 tablet_bootstrap.cc:492] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:27.868176  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a981502782314272a8a9bee8da734427" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:27.869012  9907 raft_consensus.cc:2468] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:27.876605 10580 leader_election.cc:290] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:27.877921  9949 leader_election.cc:304] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 
I20260709 13:05:27.869019 10551 tablet_bootstrap.cc:492] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:27.889770 10580 raft_consensus.cc:2804] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:27.890437 10580 raft_consensus.cc:493] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.890897 10580 raft_consensus.cc:3060] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:27.888298 10551 ts_tablet_manager.cc:1403] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.220s	user 0.031s	sys 0.023s
I20260709 13:05:27.953693  9658 catalog_manager.cc:5697] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 1 to 2, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.926393 10551 raft_consensus.cc:359] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:27.962388 10551 raft_consensus.cc:740] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:27.960987  9658 catalog_manager.cc:5697] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 1 to 2, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:27.972185  9946 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 56914.3us
W20260709 13:05:27.976505  9946 leader_election.cc:336] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:27.989145 10551 consensus_queue.cc:260] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:27.991797 10551 ts_tablet_manager.cc:1434] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 13:05:27.921083 10580 raft_consensus.cc:515] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:28.003502 10580 leader_election.cc:290] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:28.004645 10554 raft_consensus.cc:493] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.010171  9905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a981502782314272a8a9bee8da734427" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:28.055737  9905 raft_consensus.cc:3060] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:28.055523 10554 raft_consensus.cc:515] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:28.059707 10554 leader_election.cc:290] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:28.061412  9826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "196922ea9dc14e078a3da6af1206f79a" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:28.064258  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "196922ea9dc14e078a3da6af1206f79a" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e10aab26c644389b8161974dcabc5d0" is_pre_election: true
I20260709 13:05:28.067284 10548 leader_election.cc:304] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:28.068272 10554 raft_consensus.cc:2749] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:28.052455  9946 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.0.107.254:42979, user_credentials={real_user=slave}} blocked reactor thread for 44230.8us
I20260709 13:05:28.092003 10551 tablet_bootstrap.cc:492] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:28.116766  9905 raft_consensus.cc:2468] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 3.
W20260709 13:05:28.133188  9946 leader_election.cc:336] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:28.136689  9949 leader_election.cc:304] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:28.138391 10580 raft_consensus.cc:2804] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:28.139087 10580 raft_consensus.cc:697] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:28.147352 10580 consensus_queue.cc:237] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:28.192451  9656 catalog_manager.cc:5697] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 1 to 3, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 3 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.225127 10551 tablet_bootstrap.cc:492] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:28.233718 10551 tablet_bootstrap.cc:492] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:28.234396 10551 ts_tablet_manager.cc:1403] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.143s	user 0.022s	sys 0.010s
I20260709 13:05:28.253088 10551 raft_consensus.cc:359] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:28.259564 10551 raft_consensus.cc:740] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.260565 10551 consensus_queue.cc:260] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:28.292099 10551 ts_tablet_manager.cc:1434] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.057s	user 0.004s	sys 0.005s
I20260709 13:05:28.293735 10551 tablet_bootstrap.cc:492] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:28.314167  9798 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111) [suppressed 370 similar messages]
W20260709 13:05:28.406174  9872 consensus_peers.cc:597] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:28.430262  9946 consensus_peers.cc:597] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:28.430964  9946 consensus_peers.cc:597] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:28.435379  9798 consensus_peers.cc:597] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:28.442651 10551 tablet_bootstrap.cc:492] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:28.443960 10551 tablet_bootstrap.cc:492] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:28.444792 10551 ts_tablet_manager.cc:1403] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.151s	user 0.041s	sys 0.062s
I20260709 13:05:28.449892 10551 raft_consensus.cc:359] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:28.450887 10551 raft_consensus.cc:740] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.451766 10551 consensus_queue.cc:260] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:28.454397 10551 ts_tablet_manager.cc:1434] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.003s
I20260709 13:05:28.455776 10551 tablet_bootstrap.cc:492] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:28.467206   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:41383
I20260709 13:05:28.471403 10728 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:41383 every 8 connection(s)
I20260709 13:05:28.546010 10551 tablet_bootstrap.cc:492] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:28.547679 10551 tablet_bootstrap.cc:492] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:28.548418 10551 ts_tablet_manager.cc:1403] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.093s	user 0.037s	sys 0.022s
I20260709 13:05:28.551276 10551 raft_consensus.cc:359] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:28.557098 10551 raft_consensus.cc:740] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.557853 10551 consensus_queue.cc:260] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:28.559506 10551 ts_tablet_manager.cc:1434] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:05:28.560552 10551 tablet_bootstrap.cc:492] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:28.578671  9872 consensus_peers.cc:597] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:28.582829 10684 raft_consensus.cc:3060] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:28.583539 10730 heartbeater.cc:344] Connected to a master server at 127.0.107.254:42979
I20260709 13:05:28.584088 10730 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:28.585276 10730 heartbeater.cc:507] Master 127.0.107.254:42979 requested a full tablet report, sending...
I20260709 13:05:28.621812 10668 raft_consensus.cc:3060] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:28.626183 10551 tablet_bootstrap.cc:492] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:28.627409 10551 tablet_bootstrap.cc:492] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:28.628121 10551 ts_tablet_manager.cc:1403] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.068s	user 0.042s	sys 0.019s
I20260709 13:05:28.631166 10551 raft_consensus.cc:359] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:28.631840 10551 raft_consensus.cc:740] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.632524 10551 consensus_queue.cc:260] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:28.637099 10551 ts_tablet_manager.cc:1434] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:05:28.638041 10551 tablet_bootstrap.cc:492] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:28.659611  9907 raft_consensus.cc:1275] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:28.663224 10488 consensus_queue.cc:1048] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:28.673498 10554 raft_consensus.cc:493] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.674535 10554 raft_consensus.cc:515] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:28.680593 10554 leader_election.cc:290] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:28.682551  9974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0e147704a7413397055b8d3a812a8e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:28.685315  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0e147704a7413397055b8d3a812a8e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030" is_pre_election: true
I20260709 13:05:28.689097 10546 leader_election.cc:304] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:28.703441 10554 raft_consensus.cc:2749] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:28.722231 10551 tablet_bootstrap.cc:492] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:28.723346 10551 tablet_bootstrap.cc:492] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:28.723973 10551 ts_tablet_manager.cc:1403] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.086s	user 0.025s	sys 0.012s
W20260709 13:05:28.745052  9872 consensus_peers.cc:597] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:28.754268 10551 raft_consensus.cc:359] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:28.755182 10551 raft_consensus.cc:740] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.763067 10668 raft_consensus.cc:3060] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:28.770953 10551 consensus_queue.cc:260] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:28.785666 10551 ts_tablet_manager.cc:1434] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.002s
I20260709 13:05:28.786583 10551 tablet_bootstrap.cc:492] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:28.803016 10668 raft_consensus.cc:1275] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:05:28.814798  9798 consensus_peers.cc:597] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:28.815634 10580 consensus_queue.cc:1048] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:28.827119  9656 ts_manager.cc:194] Re-registered known tserver with Master: 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:28.830106 10551 tablet_bootstrap.cc:492] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:05:28.830471  9946 consensus_peers.cc:597] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:28.833218 10551 tablet_bootstrap.cc:492] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:28.833982 10551 ts_tablet_manager.cc:1403] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.048s	user 0.019s	sys 0.000s
I20260709 13:05:28.837667 10551 raft_consensus.cc:359] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:28.839032 10551 raft_consensus.cc:740] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.839885 10551 consensus_queue.cc:260] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:28.842083 10551 ts_tablet_manager.cc:1434] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:05:28.843109 10551 tablet_bootstrap.cc:492] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:28.897018  9798 consensus_peers.cc:597] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:28.906816 10551 tablet_bootstrap.cc:492] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:28.908464 10551 tablet_bootstrap.cc:492] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:28.909433 10551 ts_tablet_manager.cc:1403] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.067s	user 0.019s	sys 0.028s
I20260709 13:05:28.912415 10551 raft_consensus.cc:359] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:28.913149 10551 raft_consensus.cc:740] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.914215 10551 consensus_queue.cc:260] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:28.916664 10551 ts_tablet_manager.cc:1434] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:28.917634 10551 tablet_bootstrap.cc:492] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:28.959152  9872 consensus_peers.cc:597] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:28.959589 10551 tablet_bootstrap.cc:492] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:28.960644 10551 tablet_bootstrap.cc:492] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:28.961304 10551 ts_tablet_manager.cc:1403] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.011s
I20260709 13:05:28.964164 10551 raft_consensus.cc:359] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:28.964848 10551 raft_consensus.cc:740] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:28.965704 10668 raft_consensus.cc:3060] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:28.966006  9656 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33880
I20260709 13:05:28.965494 10551 consensus_queue.cc:260] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:28.972311 10730 heartbeater.cc:499] Master 127.0.107.254:42979 was elected leader, sending a full tablet report...
I20260709 13:05:28.993564 10551 ts_tablet_manager.cc:1434] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.001s
I20260709 13:05:28.995071 10551 tablet_bootstrap.cc:492] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:29.033831 10551 tablet_bootstrap.cc:492] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:29.034824 10551 tablet_bootstrap.cc:492] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:29.035674 10551 ts_tablet_manager.cc:1403] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.041s	user 0.018s	sys 0.008s
W20260709 13:05:29.089490  9801 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45019, user_credentials={real_user=slave}} blocked reactor thread for 47175.8us
I20260709 13:05:29.089754 10668 raft_consensus.cc:3060] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:29.096446  9799 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 49573.8us
W20260709 13:05:29.105072  9872 consensus_peers.cc:597] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:29.105413 10551 raft_consensus.cc:359] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:29.106267 10551 raft_consensus.cc:740] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.107049 10551 consensus_queue.cc:260] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:29.111680 10551 ts_tablet_manager.cc:1434] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.000s
I20260709 13:05:29.118574 10551 tablet_bootstrap.cc:492] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:29.119118  9798 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 43506.3us
W20260709 13:05:29.122743  9872 consensus_peers.cc:597] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:29.127658  9798 consensus_peers.cc:597] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:29.187073 10551 tablet_bootstrap.cc:492] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:29.188032 10551 tablet_bootstrap.cc:492] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:29.188678 10551 ts_tablet_manager.cc:1403] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.070s	user 0.018s	sys 0.000s
I20260709 13:05:29.198421 10551 raft_consensus.cc:359] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:29.199241 10551 raft_consensus.cc:740] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.200111 10551 consensus_queue.cc:260] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:29.202780 10551 ts_tablet_manager.cc:1434] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:05:29.203691 10551 tablet_bootstrap.cc:492] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:29.210106  9798 consensus_peers.cc:597] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:29.214149  9946 consensus_peers.cc:597] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:29.225668  9946 consensus_peers.cc:597] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:29.258095  9946 consensus_peers.cc:597] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:29.260056  9798 consensus_peers.cc:597] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:29.278714 10551 tablet_bootstrap.cc:492] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:29.279793 10551 tablet_bootstrap.cc:492] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:29.286067 10668 raft_consensus.cc:3060] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:29.292438  9872 consensus_peers.cc:597] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:29.280495 10551 ts_tablet_manager.cc:1403] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.077s	user 0.013s	sys 0.010s
I20260709 13:05:29.306963 10551 raft_consensus.cc:359] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:29.307806 10551 raft_consensus.cc:740] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.314667 10551 consensus_queue.cc:260] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:29.317054 10551 ts_tablet_manager.cc:1434] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:05:29.318298 10551 tablet_bootstrap.cc:492] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:29.324224  9798 consensus_peers.cc:597] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:29.355749  9946 consensus_peers.cc:597] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:29.368270 10551 tablet_bootstrap.cc:492] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:29.372000 10551 tablet_bootstrap.cc:492] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:29.373039 10551 ts_tablet_manager.cc:1403] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.055s	user 0.016s	sys 0.005s
I20260709 13:05:29.376622 10551 raft_consensus.cc:359] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:29.377406 10551 raft_consensus.cc:740] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.378161 10551 consensus_queue.cc:260] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:29.380402 10551 ts_tablet_manager.cc:1434] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:05:29.381497 10551 tablet_bootstrap.cc:492] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:29.414980 10679 raft_consensus.cc:3060] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.415405 10551 tablet_bootstrap.cc:492] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:29.417770 10551 tablet_bootstrap.cc:492] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:29.418502 10551 ts_tablet_manager.cc:1403] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.016s
I20260709 13:05:29.421696 10551 raft_consensus.cc:359] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:29.422505 10551 raft_consensus.cc:740] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.423446 10551 consensus_queue.cc:260] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:29.427556 10551 ts_tablet_manager.cc:1434] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:05:29.428577 10551 tablet_bootstrap.cc:492] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:29.487473 10551 tablet_bootstrap.cc:492] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:29.488560 10551 tablet_bootstrap.cc:492] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:29.489269 10551 ts_tablet_manager.cc:1403] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.061s	user 0.012s	sys 0.012s
I20260709 13:05:29.495853 10551 raft_consensus.cc:359] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:29.501123 10551 raft_consensus.cc:740] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.501942 10551 consensus_queue.cc:260] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:29.504145 10551 ts_tablet_manager.cc:1434] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:05:29.505036 10551 tablet_bootstrap.cc:492] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:29.522239 10667 raft_consensus.cc:3060] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:29.526482  9946 consensus_peers.cc:597] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:29.551576  9946 consensus_peers.cc:597] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:29.564191 10760 raft_consensus.cc:493] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.564803 10760 raft_consensus.cc:515] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:29.566893 10551 tablet_bootstrap.cc:492] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:29.567082 10760 leader_election.cc:290] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:29.567922 10551 tablet_bootstrap.cc:492] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:29.570937  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" is_pre_election: true
I20260709 13:05:29.571641  9980 raft_consensus.cc:2468] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 3.
I20260709 13:05:29.575377 10667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" is_pre_election: true
I20260709 13:05:29.575775  9799 leader_election.cc:304] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:29.577099 10760 raft_consensus.cc:2804] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:29.577387 10551 ts_tablet_manager.cc:1403] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.073s	user 0.021s	sys 0.010s
I20260709 13:05:29.577653 10760 raft_consensus.cc:493] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.578194 10760 raft_consensus.cc:3060] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:29.580947 10551 raft_consensus.cc:359] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:29.581657 10551 raft_consensus.cc:740] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
W20260709 13:05:29.582769  9798 leader_election.cc:343] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 4 pre-election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.582350 10551 consensus_queue.cc:260] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:29.585000 10551 ts_tablet_manager.cc:1434] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:05:29.585770 10551 tablet_bootstrap.cc:492] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:29.605188 10760 raft_consensus.cc:515] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:29.609056  9980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:29.609696  9980 raft_consensus.cc:3060] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:29.623800  9946 consensus_peers.cc:597] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:29.640185  9980 raft_consensus.cc:2468] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 4.
I20260709 13:05:29.655475 10760 leader_election.cc:290] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 4 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:29.658680 10679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27c539d1a0b54f8ca31dba142cb38599" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
W20260709 13:05:29.660347  9798 leader_election.cc:343] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 4 election: Tablet error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:29.661489  9799 leader_election.cc:304] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:29.662597 10760 raft_consensus.cc:2804] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:29.663170 10760 raft_consensus.cc:697] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 4 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:29.664114 10760 consensus_queue.cc:237] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:29.673975 10679 raft_consensus.cc:3060] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.680604 10551 tablet_bootstrap.cc:492] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:29.681919 10551 tablet_bootstrap.cc:492] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:29.682690 10551 ts_tablet_manager.cc:1403] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.097s	user 0.030s	sys 0.035s
I20260709 13:05:29.687904 10551 raft_consensus.cc:359] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:29.688618 10551 raft_consensus.cc:740] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.690321 10551 consensus_queue.cc:260] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:29.692893 10551 ts_tablet_manager.cc:1434] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:05:29.693760 10551 tablet_bootstrap.cc:492] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:29.716552  9658 catalog_manager.cc:5697] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 1 to 4, leader changed from 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193) to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:29.732672 10551 tablet_bootstrap.cc:492] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:29.733807 10551 tablet_bootstrap.cc:492] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:29.734791 10551 ts_tablet_manager.cc:1403] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.041s	user 0.019s	sys 0.005s
I20260709 13:05:29.737740 10551 raft_consensus.cc:359] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:29.738428 10551 raft_consensus.cc:740] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.762645 10551 consensus_queue.cc:260] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:29.768518 10551 ts_tablet_manager.cc:1434] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.000s
I20260709 13:05:29.769419 10551 tablet_bootstrap.cc:492] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:29.834555 10551 tablet_bootstrap.cc:492] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:29.835776 10551 tablet_bootstrap.cc:492] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:29.836421 10551 ts_tablet_manager.cc:1403] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.067s	user 0.027s	sys 0.001s
I20260709 13:05:29.839365 10551 raft_consensus.cc:359] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:29.840121 10551 raft_consensus.cc:740] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.864904 10551 consensus_queue.cc:260] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:29.867426 10551 ts_tablet_manager.cc:1434] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 13:05:29.868562 10551 tablet_bootstrap.cc:492] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:29.872278 10684 raft_consensus.cc:3060] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.902379 10551 tablet_bootstrap.cc:492] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:29.903883 10551 tablet_bootstrap.cc:492] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:29.904788 10551 ts_tablet_manager.cc:1403] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.001s
I20260709 13:05:29.908821 10551 raft_consensus.cc:359] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:29.918758 10551 raft_consensus.cc:740] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.919629 10551 consensus_queue.cc:260] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:29.922544 10551 ts_tablet_manager.cc:1434] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:05:29.923504 10551 tablet_bootstrap.cc:492] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:29.953820 10551 tablet_bootstrap.cc:492] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:29.954810 10551 tablet_bootstrap.cc:492] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:29.955526 10551 ts_tablet_manager.cc:1403] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.032s	user 0.018s	sys 0.000s
I20260709 13:05:29.958122 10551 raft_consensus.cc:359] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:29.958841 10551 raft_consensus.cc:740] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:29.959609 10551 consensus_queue.cc:260] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:29.962183 10551 ts_tablet_manager.cc:1434] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:05:29.963168 10551 tablet_bootstrap.cc:492] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:29.995434 10551 tablet_bootstrap.cc:492] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:29.996469 10551 tablet_bootstrap.cc:492] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:29.997449 10551 ts_tablet_manager.cc:1403] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.014s
I20260709 13:05:30.000502 10551 raft_consensus.cc:359] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:30.001430 10551 raft_consensus.cc:740] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.002415 10551 consensus_queue.cc:260] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:30.009104 10551 ts_tablet_manager.cc:1434] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:05:30.010097 10551 tablet_bootstrap.cc:492] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:30.043610 10551 tablet_bootstrap.cc:492] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:30.044589 10551 tablet_bootstrap.cc:492] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:30.045228 10551 ts_tablet_manager.cc:1403] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.005s
I20260709 13:05:30.047776 10551 raft_consensus.cc:359] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:30.048436 10551 raft_consensus.cc:740] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.051128 10551 consensus_queue.cc:260] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:30.053371 10551 ts_tablet_manager.cc:1434] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:05:30.057454 10551 tablet_bootstrap.cc:492] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:30.070641  9798 consensus_peers.cc:597] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:30.093209  9980 raft_consensus.cc:1275] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 4 FOLLOWER]: Refusing update from remote peer a1cbca39e34743d8a40816c3bc5f2030: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:05:30.095422 10759 consensus_queue.cc:1048] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:30.109797 10551 tablet_bootstrap.cc:492] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:30.111295 10551 tablet_bootstrap.cc:492] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:30.112133 10551 ts_tablet_manager.cc:1403] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.055s	user 0.017s	sys 0.004s
I20260709 13:05:30.115165 10551 raft_consensus.cc:359] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:30.115998 10551 raft_consensus.cc:740] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.116858 10551 consensus_queue.cc:260] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:30.126151 10551 ts_tablet_manager.cc:1434] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:05:30.137229 10551 tablet_bootstrap.cc:492] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
W20260709 13:05:30.253580  9798 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 68833us
I20260709 13:05:30.259099 10551 tablet_bootstrap.cc:492] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:30.261264 10551 tablet_bootstrap.cc:492] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:30.261930 10551 ts_tablet_manager.cc:1403] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.134s	user 0.027s	sys 0.007s
I20260709 13:05:30.264578 10551 raft_consensus.cc:359] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:30.265259 10551 raft_consensus.cc:740] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.265949 10551 consensus_queue.cc:260] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:30.281790 10551 ts_tablet_manager.cc:1434] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:05:30.292891 10551 tablet_bootstrap.cc:492] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:30.326725 10684 raft_consensus.cc:3060] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:30.354822 10551 tablet_bootstrap.cc:492] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:30.355885 10551 tablet_bootstrap.cc:492] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:30.356679 10551 ts_tablet_manager.cc:1403] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.064s	user 0.013s	sys 0.005s
I20260709 13:05:30.359237 10551 raft_consensus.cc:359] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:30.359959 10551 raft_consensus.cc:740] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.360678 10551 consensus_queue.cc:260] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:30.362469 10551 ts_tablet_manager.cc:1434] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:30.364683 10551 tablet_bootstrap.cc:492] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:30.392351 10551 tablet_bootstrap.cc:492] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:30.393627 10551 tablet_bootstrap.cc:492] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:30.398845 10551 ts_tablet_manager.cc:1403] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.007s
I20260709 13:05:30.401984 10551 raft_consensus.cc:359] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:30.402853 10551 raft_consensus.cc:740] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.403852 10551 consensus_queue.cc:260] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:30.405922 10551 ts_tablet_manager.cc:1434] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:05:30.406999 10551 tablet_bootstrap.cc:492] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:30.430653 10551 tablet_bootstrap.cc:492] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:30.432032 10551 tablet_bootstrap.cc:492] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:30.433050 10551 ts_tablet_manager.cc:1403] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.005s
I20260709 13:05:30.467489 10551 raft_consensus.cc:359] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:30.468253 10551 raft_consensus.cc:740] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.469324 10551 consensus_queue.cc:260] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:30.471676 10551 ts_tablet_manager.cc:1434] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 13:05:30.472582 10551 tablet_bootstrap.cc:492] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:30.508463 10667 raft_consensus.cc:3060] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 4
I20260709 13:05:30.557225 10551 tablet_bootstrap.cc:492] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:30.558292 10551 tablet_bootstrap.cc:492] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:30.559074 10551 ts_tablet_manager.cc:1403] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.087s	user 0.015s	sys 0.009s
I20260709 13:05:30.562878 10551 raft_consensus.cc:359] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:30.563900 10551 raft_consensus.cc:740] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.564733 10551 consensus_queue.cc:260] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:30.568207 10551 ts_tablet_manager.cc:1434] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 13:05:30.571540 10551 tablet_bootstrap.cc:492] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:30.571491 10684 raft_consensus.cc:3060] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:30.604956 10551 tablet_bootstrap.cc:492] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:30.606026 10551 tablet_bootstrap.cc:492] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:30.606739 10551 ts_tablet_manager.cc:1403] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.011s
I20260709 13:05:30.609570 10551 raft_consensus.cc:359] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:30.610298 10551 raft_consensus.cc:740] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.611013 10551 consensus_queue.cc:260] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:30.622176 10551 ts_tablet_manager.cc:1434] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:05:30.623391 10551 tablet_bootstrap.cc:492] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:30.664537 10551 tablet_bootstrap.cc:492] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:30.665565 10551 tablet_bootstrap.cc:492] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:30.666247 10551 ts_tablet_manager.cc:1403] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.043s	user 0.032s	sys 0.005s
I20260709 13:05:30.669126 10551 raft_consensus.cc:359] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:30.669866 10551 raft_consensus.cc:740] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.670550 10551 consensus_queue.cc:260] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:30.672962 10551 ts_tablet_manager.cc:1434] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:05:30.673784 10551 tablet_bootstrap.cc:492] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:30.718148 10551 tablet_bootstrap.cc:492] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:30.719290 10551 tablet_bootstrap.cc:492] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:30.720031 10551 ts_tablet_manager.cc:1403] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.046s	user 0.021s	sys 0.007s
I20260709 13:05:30.726488 10551 raft_consensus.cc:359] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:30.727277 10551 raft_consensus.cc:740] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.728120 10551 consensus_queue.cc:260] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:30.730428 10551 ts_tablet_manager.cc:1434] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:05:30.731339 10551 tablet_bootstrap.cc:492] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:30.818171 10551 tablet_bootstrap.cc:492] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:05:30.823769  9872 consensus_peers.cc:597] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:05:30.825654 10551 tablet_bootstrap.cc:492] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:30.826613 10551 ts_tablet_manager.cc:1403] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.096s	user 0.021s	sys 0.008s
I20260709 13:05:30.871253 10551 raft_consensus.cc:359] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:30.871973 10551 raft_consensus.cc:740] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:30.872695 10551 consensus_queue.cc:260] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:30.890079 10551 ts_tablet_manager.cc:1434] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.038s	user 0.005s	sys 0.000s
I20260709 13:05:30.890980 10551 tablet_bootstrap.cc:492] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:30.900102 10668 raft_consensus.cc:3060] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:31.238004 10551 tablet_bootstrap.cc:492] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:31.239430 10551 tablet_bootstrap.cc:492] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:31.240332 10551 ts_tablet_manager.cc:1403] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.350s	user 0.079s	sys 0.099s
I20260709 13:05:31.253535 10551 raft_consensus.cc:359] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:31.254335 10551 raft_consensus.cc:740] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:31.255128 10551 consensus_queue.cc:260] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:31.274423 10551 ts_tablet_manager.cc:1434] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:05:31.275364 10551 tablet_bootstrap.cc:492] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap starting.
I20260709 13:05:31.308532 10551 tablet_bootstrap.cc:492] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:31.309578 10551 tablet_bootstrap.cc:492] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4: Bootstrap complete.
I20260709 13:05:31.310469 10551 ts_tablet_manager.cc:1403] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.000s
I20260709 13:05:31.314038 10551 raft_consensus.cc:359] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:31.316499 10551 raft_consensus.cc:740] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Initialized, Role: FOLLOWER
I20260709 13:05:31.317209 10551 consensus_queue.cc:260] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:31.319746 10551 ts_tablet_manager.cc:1434] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
W20260709 13:05:31.754470  9798 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 79924.4us
W20260709 13:05:31.846288  9872 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 155360us
I20260709 13:05:31.899569 10679 raft_consensus.cc:3060] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:35.655071  9801 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:45019, user_credentials={real_user=slave}} blocked reactor thread for 51860.9us
W20260709 13:05:35.658890  9798 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 56837us
W20260709 13:05:35.676658  9799 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 83528.2us
I20260709 13:05:38.543097   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:38.592068  9900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e"
dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
mode: GRACEFUL
new_leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:48434
I20260709 13:05:38.592903  9900 raft_consensus.cc:606] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Received request to transfer leadership to TS 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:38.606079  9979 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "755ff96760b04fd49a520c059f90b1fe"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
mode: GRACEFUL
new_leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:51936
I20260709 13:05:38.606724  9979 raft_consensus.cc:606] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Received request to transfer leadership to TS 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:38.619861  9826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0c62d82ebe5420bb298d787cda221aa"
dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
mode: GRACEFUL
new_leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
 from {username='slave'} at 127.0.0.1:33390
I20260709 13:05:38.620450  9826 raft_consensus.cc:606] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Received request to transfer leadership to TS 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:38.623903  9979 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8f99aa4491c4458bb5ebc03bc0cd813"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
mode: GRACEFUL
new_leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:51936
I20260709 13:05:38.624610  9979 raft_consensus.cc:606] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Received request to transfer leadership to TS 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:38.627081  9979 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
mode: GRACEFUL
new_leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:51936
I20260709 13:05:38.627583  9979 raft_consensus.cc:606] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Received request to transfer leadership to TS 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:38.630261  9900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eeefc94068a14a0b9dd656a55fd68ccc"
dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
mode: GRACEFUL
new_leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:48434
I20260709 13:05:38.630738  9900 raft_consensus.cc:606] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Received request to transfer leadership to TS 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:38.631814   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:38.633198   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:38.741312 10839 raft_consensus.cc:993] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a: : Instructing follower 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:38.741842 10839 raft_consensus.cc:1081] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Signalling peer 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:38.744758 10667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "755ff96760b04fd49a520c059f90b1fe"
dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:49900
I20260709 13:05:38.745536 10667 raft_consensus.cc:493] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:38.745992 10667 raft_consensus.cc:3060] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.759233 10667 raft_consensus.cc:515] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:38.762061 10667 leader_election.cc:290] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 3 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:38.762571  9900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "755ff96760b04fd49a520c059f90b1fe" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:38.763574  9900 raft_consensus.cc:3060] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.764492  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "755ff96760b04fd49a520c059f90b1fe" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:38.765098  9978 raft_consensus.cc:3055] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:38.765421  9978 raft_consensus.cc:740] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:38.766258  9978 consensus_queue.cc:260] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:38.769629  9978 raft_consensus.cc:3060] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.773891  9900 raft_consensus.cc:2468] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 3.
I20260709 13:05:38.775825 10548 leader_election.cc:304] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:38.785792 10829 raft_consensus.cc:993] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030: : Instructing follower 23cbd6234c5e43d89f22c235ed6bb96a to start an election
I20260709 13:05:38.787825 10819 raft_consensus.cc:1081] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Signalling peer 23cbd6234c5e43d89f22c235ed6bb96a to start an election
I20260709 13:05:38.788863 10847 raft_consensus.cc:2804] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:38.789300  9978 raft_consensus.cc:2468] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 3.
I20260709 13:05:38.793948 10840 raft_consensus.cc:993] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0: : Instructing follower 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:38.794766  9979 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0c62d82ebe5420bb298d787cda221aa"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
 from {username='slave'} at 127.0.0.1:54620
I20260709 13:05:38.794862 10840 raft_consensus.cc:1081] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Signalling peer 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:38.795426  9979 raft_consensus.cc:493] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:38.795867  9979 raft_consensus.cc:3060] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.800619 10667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e"
dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:49910
I20260709 13:05:38.801303 10667 raft_consensus.cc:493] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:38.802665 10667 raft_consensus.cc:3060] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.807513 10847 raft_consensus.cc:697] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:38.807936  9979 raft_consensus.cc:515] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:38.808862 10847 consensus_queue.cc:237] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:38.810492  9979 leader_election.cc:290] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:38.811072  9826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c62d82ebe5420bb298d787cda221aa" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:38.811681  9826 raft_consensus.cc:3055] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:38.811986  9826 raft_consensus.cc:740] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:38.812785  9826 consensus_queue.cc:260] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:38.814213  9826 raft_consensus.cc:3060] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.814311  9900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c62d82ebe5420bb298d787cda221aa" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:38.815161  9900 raft_consensus.cc:3060] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.822660  9826 raft_consensus.cc:2468] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:38.824529 10846 raft_consensus.cc:993] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a: : Instructing follower 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:38.825093 10839 raft_consensus.cc:1081] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Signalling peer 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:38.825592  9948 leader_election.cc:304] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:38.826581 10839 raft_consensus.cc:2804] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:38.828088 10668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8f99aa4491c4458bb5ebc03bc0cd813"
dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:49900
I20260709 13:05:38.828685 10668 raft_consensus.cc:493] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:38.829041 10668 raft_consensus.cc:3060] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.830262 10839 raft_consensus.cc:697] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:38.831466 10839 consensus_queue.cc:237] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:38.845266 10668 raft_consensus.cc:515] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:38.848240  9900 raft_consensus.cc:2468] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:38.840217 10667 raft_consensus.cc:515] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:38.850399  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f99aa4491c4458bb5ebc03bc0cd813" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:38.850909 10840 raft_consensus.cc:993] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0: : Instructing follower 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:38.851126  9908 raft_consensus.cc:3060] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.851465 10840 raft_consensus.cc:1081] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Signalling peer 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:38.852015  9900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:38.852602  9900 raft_consensus.cc:3055] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:38.853618 10665 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eeefc94068a14a0b9dd656a55fd68ccc"
dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:49910
I20260709 13:05:38.854399 10665 raft_consensus.cc:493] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:38.853434 10667 leader_election.cc:290] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 3 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:38.855033 10665 raft_consensus.cc:3060] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.853725  9900 raft_consensus.cc:740] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:38.856086  9900 consensus_queue.cc:260] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:38.854346  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b0a6c62c14440c9e613e73781c542e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:38.857734  9978 raft_consensus.cc:3060] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.858045 10668 leader_election.cc:290] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:38.857276  9900 raft_consensus.cc:3060] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.862835  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f99aa4491c4458bb5ebc03bc0cd813" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:38.865499  9979 raft_consensus.cc:3055] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:38.865787  9979 raft_consensus.cc:740] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:38.866447 10665 raft_consensus.cc:515] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:38.866560  9979 consensus_queue.cc:260] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:38.867492  9900 raft_consensus.cc:2468] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 3.
I20260709 13:05:38.868733 10548 leader_election.cc:304] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:38.869578 10847 raft_consensus.cc:2804] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:38.870090 10847 raft_consensus.cc:697] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:38.871626  9978 raft_consensus.cc:2468] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 3.
I20260709 13:05:38.871950  9831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeefc94068a14a0b9dd656a55fd68ccc" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:38.872740 10847 consensus_queue.cc:237] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:38.873656  9831 raft_consensus.cc:3060] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.873690 10665 leader_election.cc:290] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:38.872882  9908 raft_consensus.cc:2468] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:38.874161  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeefc94068a14a0b9dd656a55fd68ccc" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:38.874435  9979 raft_consensus.cc:3060] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.880832  9907 raft_consensus.cc:3055] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:38.881239  9907 raft_consensus.cc:740] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:38.882184  9907 consensus_queue.cc:260] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:38.883692  9907 raft_consensus.cc:3060] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:38.885855  9656 catalog_manager.cc:5697] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 2 to 3, leader changed from 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196) to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:38.887890 10548 leader_election.cc:304] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:38.888159  9979 raft_consensus.cc:2468] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:38.886056  9654 catalog_manager.cc:5697] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 1 to 2, leader changed from a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:38.888962 10850 raft_consensus.cc:2804] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:38.892655  9907 raft_consensus.cc:2468] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:38.894477 10548 leader_election.cc:304] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:38.894960  9831 raft_consensus.cc:2468] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:38.897462 10850 raft_consensus.cc:697] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:38.898103 10854 raft_consensus.cc:2804] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:38.898715 10854 raft_consensus.cc:697] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:38.898726 10850 consensus_queue.cc:237] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:38.900025 10854 consensus_queue.cc:237] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:38.959173 10833 raft_consensus.cc:993] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a: : Instructing follower 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:38.959700 10833 raft_consensus.cc:1081] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Signalling peer 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:38.961899 10667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3"
dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:49900
I20260709 13:05:38.962525 10667 raft_consensus.cc:493] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:38.963501 10667 raft_consensus.cc:3060] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.972440 10667 raft_consensus.cc:515] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:38.975215  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:38.975943  9979 raft_consensus.cc:3055] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:38.976305  9979 raft_consensus.cc:740] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:38.977169  9979 consensus_queue.cc:260] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:38.978344  9979 raft_consensus.cc:3060] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.985939  9979 raft_consensus.cc:2468] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 3.
I20260709 13:05:38.987283 10546 leader_election.cc:304] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:38.988282 10854 raft_consensus.cc:2804] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:38.988762 10854 raft_consensus.cc:697] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:38.989656 10854 consensus_queue.cc:237] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:38.991338  9826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e80fbd6291544644996b96fec0e8f2c3" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:38.992331  9826 raft_consensus.cc:3060] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.993556 10667 leader_election.cc:290] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 3 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:39.009313  9826 raft_consensus.cc:2468] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 3.
I20260709 13:05:39.014941  9656 catalog_manager.cc:5697] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 2 to 3, leader changed from 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195) to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.016623  9656 catalog_manager.cc:5697] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 1 to 2, leader changed from 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196) to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.018062  9656 catalog_manager.cc:5697] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 1 to 2, leader changed from 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195) to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.049422  9656 catalog_manager.cc:5697] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 2 to 3, leader changed from 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196) to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.242604  9907 raft_consensus.cc:1275] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:39.244270 10846 consensus_queue.cc:1048] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:39.258679  9826 raft_consensus.cc:1275] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:39.262581 10846 consensus_queue.cc:1048] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:39.277887  9979 raft_consensus.cc:1275] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:39.280272 10854 consensus_queue.cc:1048] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:39.330592  9979 raft_consensus.cc:1275] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:39.332314 10854 consensus_queue.cc:1048] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:39.375118  9907 raft_consensus.cc:1275] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:39.377964 10854 consensus_queue.cc:1048] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:05:39.401714  9826 raft_consensus.cc:1275] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:39.408123  9979 raft_consensus.cc:1275] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:05:39.402062  9799 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:34321, user_credentials={real_user=slave}} blocked reactor thread for 52330.6us
I20260709 13:05:39.407501 10847 consensus_queue.cc:1048] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:05:39.413712  9908 raft_consensus.cc:1275] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:39.418634 10847 consensus_queue.cc:1048] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
W20260709 13:05:39.435562  9798 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 54687.7us
I20260709 13:05:39.434295 10854 consensus_queue.cc:1048] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:05:39.475543  9979 raft_consensus.cc:1275] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:39.479803  9905 raft_consensus.cc:1275] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:39.487669 10854 consensus_queue.cc:1048] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:39.489440 10856 consensus_queue.cc:1048] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:05:39.555323  9834 raft_consensus.cc:1275] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:39.557134  9907 raft_consensus.cc:1275] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:39.568667 10856 consensus_queue.cc:1048] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:39.581000 10881 consensus_queue.cc:1048] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:05:39.634152   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:39.679927   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:39.722023  9907 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1064d1c0e3ba4ab68fb20fd5f093abef"
dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
mode: GRACEFUL
new_leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:48434
I20260709 13:05:39.722548  9907 raft_consensus.cc:606] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Received request to transfer leadership to TS 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:39.725121  9907 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "168ffc86a17d47e8aba438779717eab6"
dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
mode: GRACEFUL
new_leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:48434
I20260709 13:05:39.725620  9907 raft_consensus.cc:606] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Received request to transfer leadership to TS 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:39.728614  9907 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "251be49566d64412bd3d6e3d72c5795e"
dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
mode: GRACEFUL
new_leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:48434
I20260709 13:05:39.729198  9907 raft_consensus.cc:606] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Received request to transfer leadership to TS 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:39.731778  9826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b228f0642a242db93a7d11c1afab0ea"
dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
mode: GRACEFUL
new_leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
 from {username='slave'} at 127.0.0.1:33390
I20260709 13:05:39.732318  9826 raft_consensus.cc:606] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Received request to transfer leadership to TS 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:39.735049  9979 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98a516bf1d774804b94074f9765006ad"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
mode: GRACEFUL
new_leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:51936
I20260709 13:05:39.735617  9979 raft_consensus.cc:606] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Received request to transfer leadership to TS 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:39.738319  9907 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b799cecf522841009ed9a2b1b421be50"
dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
mode: GRACEFUL
new_leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
 from {username='slave'} at 127.0.0.1:48434
I20260709 13:05:39.739063  9907 raft_consensus.cc:606] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Received request to transfer leadership to TS 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:39.740296   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:39.741709   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:39.817502 10859 raft_consensus.cc:993] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a: : Instructing follower 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:39.818040 10833 raft_consensus.cc:1081] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Signalling peer 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:39.819509 10667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98a516bf1d774804b94074f9765006ad"
dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:49900
I20260709 13:05:39.820048 10667 raft_consensus.cc:493] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:39.820425 10667 raft_consensus.cc:3060] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.827040 10667 raft_consensus.cc:515] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:39.829425  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a516bf1d774804b94074f9765006ad" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:39.829583  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a516bf1d774804b94074f9765006ad" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:39.830040  9907 raft_consensus.cc:3060] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.830189  9979 raft_consensus.cc:3055] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:39.830613  9979 raft_consensus.cc:740] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:39.831413  9979 consensus_queue.cc:260] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:39.832195  9979 raft_consensus.cc:3060] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.829139 10667 leader_election.cc:290] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:39.838649  9907 raft_consensus.cc:2468] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:39.838640  9979 raft_consensus.cc:2468] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:39.839912 10548 leader_election.cc:304] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:39.840868 10856 raft_consensus.cc:2804] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:39.841425 10856 raft_consensus.cc:697] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:39.842201 10856 consensus_queue.cc:237] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:39.853477  9658 catalog_manager.cc:5697] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 1 to 2, leader changed from 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196) to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.933614 10876 raft_consensus.cc:993] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0: : Instructing follower 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:39.934144 10873 raft_consensus.cc:1081] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Signalling peer 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:39.936118 10667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "168ffc86a17d47e8aba438779717eab6"
dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:49910
I20260709 13:05:39.936677 10667 raft_consensus.cc:493] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:39.936950 10667 raft_consensus.cc:3060] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.942957 10667 raft_consensus.cc:515] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:39.945006 10667 leader_election.cc:290] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 3 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:39.945972  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "168ffc86a17d47e8aba438779717eab6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:39.946169  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "168ffc86a17d47e8aba438779717eab6" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:39.946547  9907 raft_consensus.cc:3055] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:39.946786  9979 raft_consensus.cc:3060] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.946906  9907 raft_consensus.cc:740] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:39.947788  9907 consensus_queue.cc:260] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:39.948521  9907 raft_consensus.cc:3060] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.953768  9979 raft_consensus.cc:2468] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 3.
I20260709 13:05:39.954139  9907 raft_consensus.cc:2468] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 3.
I20260709 13:05:39.955113 10546 leader_election.cc:304] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:39.956035 10856 raft_consensus.cc:2804] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:39.956519 10856 raft_consensus.cc:697] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:39.957329 10856 consensus_queue.cc:237] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:39.973183 10837 raft_consensus.cc:993] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0: : Instructing follower 23cbd6234c5e43d89f22c235ed6bb96a to start an election
I20260709 13:05:39.973757 10837 raft_consensus.cc:1081] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Signalling peer 23cbd6234c5e43d89f22c235ed6bb96a to start an election
I20260709 13:05:39.974711  9658 catalog_manager.cc:5697] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 2 to 3, leader changed from 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195) to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.975335  9979 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b799cecf522841009ed9a2b1b421be50"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
 from {username='slave'} at 127.0.0.1:54606
I20260709 13:05:39.975982  9979 raft_consensus.cc:493] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:39.976320  9979 raft_consensus.cc:3060] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.982247  9979 raft_consensus.cc:515] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:39.984185  9979 leader_election.cc:290] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:39.985942  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b799cecf522841009ed9a2b1b421be50" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:39.986285  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b799cecf522841009ed9a2b1b421be50" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:39.986487  9907 raft_consensus.cc:3055] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:39.986871  9907 raft_consensus.cc:740] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:39.986936  9834 raft_consensus.cc:3060] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.987777  9907 consensus_queue.cc:260] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:39.988694  9907 raft_consensus.cc:3060] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.993330  9834 raft_consensus.cc:2468] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:39.995563  9907 raft_consensus.cc:2468] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:39.997134 10873 raft_consensus.cc:993] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0: : Instructing follower 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:39.998404  9949 leader_election.cc:304] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 
I20260709 13:05:39.999042 10873 raft_consensus.cc:1081] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Signalling peer 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:39.999598 10859 raft_consensus.cc:2804] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.000166 10859 raft_consensus.cc:697] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:40.001179 10667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1064d1c0e3ba4ab68fb20fd5f093abef"
dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:49910
I20260709 13:05:40.001097 10859 consensus_queue.cc:237] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:40.001761 10667 raft_consensus.cc:493] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:40.002619 10667 raft_consensus.cc:3060] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.009231 10667 raft_consensus.cc:515] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:40.011561 10667 leader_election.cc:290] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:40.011806  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1064d1c0e3ba4ab68fb20fd5f093abef" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:40.012144  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1064d1c0e3ba4ab68fb20fd5f093abef" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:40.012732  9907 raft_consensus.cc:3055] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:40.012755  9834 raft_consensus.cc:3060] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.013135  9907 raft_consensus.cc:740] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:40.014642  9658 catalog_manager.cc:5697] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 1 to 2, leader changed from 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.014758  9907 consensus_queue.cc:260] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:40.021371  9907 raft_consensus.cc:3060] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.026760  9834 raft_consensus.cc:2468] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:40.028852  9907 raft_consensus.cc:2468] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:40.030742 10548 leader_election.cc:304] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:40.031729 10856 raft_consensus.cc:2804] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.032234 10856 raft_consensus.cc:697] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:40.036707 10856 consensus_queue.cc:237] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:40.047224  9658 catalog_manager.cc:5697] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 1 to 2, leader changed from 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195) to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.174638 10876 raft_consensus.cc:993] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0: : Instructing follower 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:40.175207 10837 raft_consensus.cc:1081] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Signalling peer 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:40.176942 10667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "251be49566d64412bd3d6e3d72c5795e"
dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:49910
I20260709 13:05:40.177515 10667 raft_consensus.cc:493] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:40.177898 10667 raft_consensus.cc:3060] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.178956 10848 raft_consensus.cc:993] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030: : Instructing follower 23cbd6234c5e43d89f22c235ed6bb96a to start an election
I20260709 13:05:40.179553 10849 raft_consensus.cc:1081] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Signalling peer 23cbd6234c5e43d89f22c235ed6bb96a to start an election
I20260709 13:05:40.181247  9978 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b228f0642a242db93a7d11c1afab0ea"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
 from {username='slave'} at 127.0.0.1:54620
I20260709 13:05:40.181875  9978 raft_consensus.cc:493] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:40.182291  9978 raft_consensus.cc:3060] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.186076 10667 raft_consensus.cc:515] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:40.189221 10667 leader_election.cc:290] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:40.191439  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251be49566d64412bd3d6e3d72c5795e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:40.192098  9834 raft_consensus.cc:3060] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.195390  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251be49566d64412bd3d6e3d72c5795e" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:40.196148  9907 raft_consensus.cc:3055] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:40.196522  9907 raft_consensus.cc:740] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:40.199879  9978 raft_consensus.cc:515] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:40.200109  9907 consensus_queue.cc:260] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:40.201679  9907 raft_consensus.cc:3060] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.204857  9834 raft_consensus.cc:2468] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:40.206604 10547 leader_election.cc:304] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:40.208029 10881 raft_consensus.cc:2804] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.208604 10881 raft_consensus.cc:697] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:40.209599 10881 consensus_queue.cc:237] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } }
I20260709 13:05:40.210399  9907 raft_consensus.cc:2468] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:40.211721  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b228f0642a242db93a7d11c1afab0ea" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:40.212353  9834 raft_consensus.cc:3055] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:40.212649  9834 raft_consensus.cc:740] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:40.213317  9978 leader_election.cc:290] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:40.213433  9834 consensus_queue.cc:260] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:40.214669  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b228f0642a242db93a7d11c1afab0ea" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:40.215389  9907 raft_consensus.cc:3060] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.215651  9834 raft_consensus.cc:3060] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:40.225168  9907 raft_consensus.cc:2468] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:40.226558  9949 leader_election.cc:304] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 
I20260709 13:05:40.227934 10833 raft_consensus.cc:2804] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.228353  9834 raft_consensus.cc:2468] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:40.228586 10833 raft_consensus.cc:697] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:40.229773 10833 consensus_queue.cc:237] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:40.248366  9658 catalog_manager.cc:5697] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 1 to 2, leader changed from 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195) to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:40.255718  9900 raft_consensus.cc:1275] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.259059 10881 consensus_queue.cc:1048] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:40.258471  9656 catalog_manager.cc:5697] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 1 to 2, leader changed from a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.273375  9975 raft_consensus.cc:1275] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.275336 10856 consensus_queue.cc:1048] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:40.483373  9900 raft_consensus.cc:1275] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.484912 10859 consensus_queue.cc:1048] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:40.488349  9978 raft_consensus.cc:1275] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:40.490814 10856 consensus_queue.cc:1048] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:40.497267  9900 raft_consensus.cc:1275] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:40.497922  9826 raft_consensus.cc:1275] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.500018 10859 consensus_queue.cc:1048] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:40.500202 10856 consensus_queue.cc:1048] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:40.550237  9826 raft_consensus.cc:1275] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.551719 10854 consensus_queue.cc:1048] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:40.562594  9907 raft_consensus.cc:1275] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.563963 10856 consensus_queue.cc:1048] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:40.703435  9900 raft_consensus.cc:1275] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.704973 10859 consensus_queue.cc:1048] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:40.715583  9826 raft_consensus.cc:1275] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.716976 10859 consensus_queue.cc:1048] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:40.740097  9826 raft_consensus.cc:1275] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.741631 10881 consensus_queue.cc:1048] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:40.742630   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:40.764441  9900 raft_consensus.cc:1275] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:40.767234 10854 consensus_queue.cc:1048] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:40.804486   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:40.846524  9907 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4cd54dca51434fcf8e5a473955555bfa"
dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
mode: GRACEFUL
new_leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
 from {username='slave'} at 127.0.0.1:48434
I20260709 13:05:40.847224  9907 raft_consensus.cc:606] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Received request to transfer leadership to TS 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:40.850241  9978 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "96d8c823d37d4e9a864b46403d267028"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
mode: GRACEFUL
new_leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:51936
I20260709 13:05:40.850865  9978 raft_consensus.cc:606] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Received request to transfer leadership to TS 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:40.854049  9834 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b189ef5046cd47ebb3d431e2ef24d6fb"
dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
mode: GRACEFUL
new_leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:33390
I20260709 13:05:40.854774  9834 raft_consensus.cc:606] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Received request to transfer leadership to TS 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:40.858036  9978 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0c62d82ebe5420bb298d787cda221aa"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
mode: GRACEFUL
new_leader_uuid: "6e10aab26c644389b8161974dcabc5d0"
 from {username='slave'} at 127.0.0.1:51936
I20260709 13:05:40.858673  9978 raft_consensus.cc:606] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Received request to transfer leadership to TS 6e10aab26c644389b8161974dcabc5d0
I20260709 13:05:40.862332  9834 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2d7bb9a0b174b168df9040b02c70ab9"
dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
mode: GRACEFUL
new_leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
 from {username='slave'} at 127.0.0.1:33390
I20260709 13:05:40.863013  9834 raft_consensus.cc:606] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Received request to transfer leadership to TS 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:40.864099   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:05:40.865648   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:41.046164 10916 raft_consensus.cc:993] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a: : Instructing follower 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:41.046618 10916 raft_consensus.cc:1081] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Signalling peer 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:41.048787 10667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "96d8c823d37d4e9a864b46403d267028"
dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:49900
I20260709 13:05:41.049384 10667 raft_consensus.cc:493] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:41.049816 10667 raft_consensus.cc:3060] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.055807 10667 raft_consensus.cc:515] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:41.057466 10667 leader_election.cc:290] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:41.058410  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d8c823d37d4e9a864b46403d267028" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:41.058676  9826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d8c823d37d4e9a864b46403d267028" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:41.059199  9826 raft_consensus.cc:3060] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.059199  9978 raft_consensus.cc:3055] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:41.059679  9978 raft_consensus.cc:740] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:41.060818  9978 consensus_queue.cc:260] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:41.062096  9978 raft_consensus.cc:3060] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.066983  9826 raft_consensus.cc:2468] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:41.068358 10547 leader_election.cc:304] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:41.069482 10881 raft_consensus.cc:2804] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:41.070242  9978 raft_consensus.cc:2468] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:41.070737 10881 raft_consensus.cc:697] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:41.072116 10881 consensus_queue.cc:237] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:41.084154  9654 catalog_manager.cc:5697] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 1 to 2, leader changed from 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196) to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.101611 10911 raft_consensus.cc:993] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0: : Instructing follower 23cbd6234c5e43d89f22c235ed6bb96a to start an election
I20260709 13:05:41.102072 10837 raft_consensus.cc:1081] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Signalling peer 23cbd6234c5e43d89f22c235ed6bb96a to start an election
I20260709 13:05:41.103804  9978 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4cd54dca51434fcf8e5a473955555bfa"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
 from {username='slave'} at 127.0.0.1:54606
I20260709 13:05:41.104348  9978 raft_consensus.cc:493] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:41.104652  9978 raft_consensus.cc:3060] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.110740  9978 raft_consensus.cc:515] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:41.113391  9978 leader_election.cc:290] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:41.113569  9826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd54dca51434fcf8e5a473955555bfa" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:41.114344  9826 raft_consensus.cc:3060] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.114645  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd54dca51434fcf8e5a473955555bfa" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:41.115276  9908 raft_consensus.cc:3055] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:41.115541  9908 raft_consensus.cc:740] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:41.116210  9908 consensus_queue.cc:260] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:41.117049  9908 raft_consensus.cc:3060] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.120625  9826 raft_consensus.cc:2468] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:41.123214  9948 leader_election.cc:304] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:41.124643  9908 raft_consensus.cc:2468] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:41.124773 10895 raft_consensus.cc:2804] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:41.125643 10895 raft_consensus.cc:697] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:41.126618 10895 consensus_queue.cc:237] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:41.137575  9654 catalog_manager.cc:5697] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 1 to 2, leader changed from 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.139910 10916 raft_consensus.cc:993] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a: : Instructing follower 6e10aab26c644389b8161974dcabc5d0 to start an election
I20260709 13:05:41.140839 10916 raft_consensus.cc:1081] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Signalling peer 6e10aab26c644389b8161974dcabc5d0 to start an election
I20260709 13:05:41.142818  9908 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0c62d82ebe5420bb298d787cda221aa"
dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
 from {username='slave'} at 127.0.0.1:58978
I20260709 13:05:41.143741  9908 raft_consensus.cc:493] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:41.144129  9908 raft_consensus.cc:3060] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.153184  9908 raft_consensus.cc:515] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:41.154973  9908 leader_election.cc:290] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 3 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:41.156059  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c62d82ebe5420bb298d787cda221aa" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:41.156059  9826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c62d82ebe5420bb298d787cda221aa" candidate_uuid: "6e10aab26c644389b8161974dcabc5d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:41.157055  9978 raft_consensus.cc:3055] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:41.157100  9826 raft_consensus.cc:3060] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.157558  9978 raft_consensus.cc:740] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:41.158408  9978 consensus_queue.cc:260] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:41.159757  9978 raft_consensus.cc:3060] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.165774  9826 raft_consensus.cc:2468] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 3.
I20260709 13:05:41.167289  9978 raft_consensus.cc:2468] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e10aab26c644389b8161974dcabc5d0 in term 3.
I20260709 13:05:41.167320  9874 leader_election.cc:304] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:41.168401 10911 raft_consensus.cc:2804] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:41.168915 10911 raft_consensus.cc:697] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 3 LEADER]: Becoming Leader. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:41.170007 10911 consensus_queue.cc:237] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:41.182698 10898 raft_consensus.cc:993] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030: : Instructing follower 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:41.182380  9654 catalog_manager.cc:5697] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 reported cstate change: term changed from 2 to 3, leader changed from 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196) to 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "6e10aab26c644389b8161974dcabc5d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.184288 10898 raft_consensus.cc:1081] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Signalling peer 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:41.186432 10848 raft_consensus.cc:993] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030: : Instructing follower 23cbd6234c5e43d89f22c235ed6bb96a to start an election
I20260709 13:05:41.187057 10848 raft_consensus.cc:1081] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Signalling peer 23cbd6234c5e43d89f22c235ed6bb96a to start an election
I20260709 13:05:41.187667 10668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b189ef5046cd47ebb3d431e2ef24d6fb"
dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:49898
I20260709 13:05:41.188331 10668 raft_consensus.cc:493] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:41.188740 10668 raft_consensus.cc:3060] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.189931  9978 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2d7bb9a0b174b168df9040b02c70ab9"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
 from {username='slave'} at 127.0.0.1:54620
I20260709 13:05:41.190557  9978 raft_consensus.cc:493] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:41.190898  9978 raft_consensus.cc:3060] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.198886 10668 raft_consensus.cc:515] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:41.202608  9975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b189ef5046cd47ebb3d431e2ef24d6fb" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:41.202889  9826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b189ef5046cd47ebb3d431e2ef24d6fb" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:41.203837  9975 raft_consensus.cc:3060] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.203982  9826 raft_consensus.cc:3055] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:41.204413  9826 raft_consensus.cc:740] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:41.205257  9826 consensus_queue.cc:260] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:41.206574  9826 raft_consensus.cc:3060] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.215412 10668 leader_election.cc:290] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:41.218092  9826 raft_consensus.cc:2468] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:41.219435 10547 leader_election.cc:304] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:41.220093 10881 raft_consensus.cc:2804] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:41.220563 10881 raft_consensus.cc:697] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:41.221498 10881 consensus_queue.cc:237] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:41.223950  9978 raft_consensus.cc:515] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:41.224953  9975 raft_consensus.cc:2468] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:41.233021  9826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d7bb9a0b174b168df9040b02c70ab9" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:41.233304  9978 leader_election.cc:290] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:41.233677  9826 raft_consensus.cc:3055] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:41.234046  9826 raft_consensus.cc:740] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:41.236464  9907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d7bb9a0b174b168df9040b02c70ab9" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:41.237115  9907 raft_consensus.cc:3060] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.234858  9826 consensus_queue.cc:260] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:41.239562  9826 raft_consensus.cc:3060] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.241226  9654 catalog_manager.cc:5697] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 1 to 2, leader changed from a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194) to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:41.254588  9826 raft_consensus.cc:2468] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:41.258227  9948 leader_election.cc:304] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:41.259254 10895 raft_consensus.cc:2804] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:41.259835 10895 raft_consensus.cc:697] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:41.260811 10895 consensus_queue.cc:237] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:41.267367  9907 raft_consensus.cc:2468] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 2.
I20260709 13:05:41.279596  9654 catalog_manager.cc:5697] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 1 to 2, leader changed from a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 2 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:41.585490  9975 raft_consensus.cc:1275] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Refusing update from remote peer 6e10aab26c644389b8161974dcabc5d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:41.586748 10837 consensus_queue.cc:1048] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:41.596153  9826 raft_consensus.cc:1275] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Refusing update from remote peer 6e10aab26c644389b8161974dcabc5d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:41.599995 10911 consensus_queue.cc:1048] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:41.651019  9907 raft_consensus.cc:1275] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.652217 10916 consensus_queue.cc:1048] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:41.664398  9834 raft_consensus.cc:1275] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.665987 10856 consensus_queue.cc:1048] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:41.666958  9826 raft_consensus.cc:1275] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.668628 10916 consensus_queue.cc:1048] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:41.685839  9975 raft_consensus.cc:1275] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.690776 10854 consensus_queue.cc:1048] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:41.717655  9978 raft_consensus.cc:1275] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.727389 10937 consensus_queue.cc:1048] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:41.748428  9826 raft_consensus.cc:1275] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.749392  9908 raft_consensus.cc:1275] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.750891 10916 consensus_queue.cc:1048] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:41.753005 10937 consensus_queue.cc:1048] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:41.763743  9834 raft_consensus.cc:1275] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:41.765719 10916 consensus_queue.cc:1048] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:41.866753   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:41.912117   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:41.950639  9975 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b228f0642a242db93a7d11c1afab0ea"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
mode: GRACEFUL
new_leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
 from {username='slave'} at 127.0.0.1:51936
I20260709 13:05:41.951305  9975 raft_consensus.cc:606] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Received request to transfer leadership to TS a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:41.953765  9908 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "778ba1e8c7b04438a6633db058c14508"
dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
mode: GRACEFUL
new_leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:48434
I20260709 13:05:41.954257  9908 raft_consensus.cc:606] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Received request to transfer leadership to TS 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:41.956875  9975 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
mode: GRACEFUL
new_leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:51936
I20260709 13:05:41.957404  9975 raft_consensus.cc:606] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Received request to transfer leadership to TS 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:41.958563   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:41.960359   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:41.968209 10922 raft_consensus.cc:993] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0: : Instructing follower 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:41.968881 10911 raft_consensus.cc:1081] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Signalling peer 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:41.970466 10668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "778ba1e8c7b04438a6633db058c14508"
dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:49910
I20260709 13:05:41.970960 10668 raft_consensus.cc:493] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:41.971252 10668 raft_consensus.cc:3060] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.975849 10668 raft_consensus.cc:515] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:41.979094 10668 leader_election.cc:290] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 3 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:41.980556  9975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778ba1e8c7b04438a6633db058c14508" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:41.981721  9900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778ba1e8c7b04438a6633db058c14508" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:41.981896  9975 raft_consensus.cc:3060] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.982465  9900 raft_consensus.cc:3055] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:41.982849  9900 raft_consensus.cc:740] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e10aab26c644389b8161974dcabc5d0, State: Running, Role: LEADER
I20260709 13:05:41.983641  9900 consensus_queue.cc:260] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:41.984855  9900 raft_consensus.cc:3060] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:41.987529  9975 raft_consensus.cc:2468] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 3.
I20260709 13:05:41.988978 10546 leader_election.cc:304] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:41.989897 10937 raft_consensus.cc:2804] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:41.990403 10937 raft_consensus.cc:697] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:41.991902 10937 consensus_queue.cc:237] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:41.993855  9900 raft_consensus.cc:2468] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 3.
I20260709 13:05:42.005369  9656 catalog_manager.cc:5697] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 2 to 3, leader changed from 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195) to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:42.275921 10895 raft_consensus.cc:993] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a: : Instructing follower 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:42.276609 10895 raft_consensus.cc:1081] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Signalling peer 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:42.278443 10668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9"
dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:49900
I20260709 13:05:42.279081 10668 raft_consensus.cc:493] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:42.279428 10668 raft_consensus.cc:3060] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.284708 10668 raft_consensus.cc:515] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:42.286317 10668 leader_election.cc:290] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 3 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:42.287200  9900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:42.287360  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab473e8b9be44bdaae62679a55757fe9" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:42.287825  9900 raft_consensus.cc:3060] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.288045  9978 raft_consensus.cc:3055] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:42.288455  9978 raft_consensus.cc:740] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:42.289273  9978 consensus_queue.cc:260] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:42.290292  9978 raft_consensus.cc:3060] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.295279  9900 raft_consensus.cc:2468] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 3.
I20260709 13:05:42.297012 10548 leader_election.cc:304] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:42.297977 10937 raft_consensus.cc:2804] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:42.298488 10937 raft_consensus.cc:697] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:42.299400 10937 consensus_queue.cc:237] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:42.312564  9656 catalog_manager.cc:5697] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 2 to 3, leader changed from 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196) to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:42.335492 10916 raft_consensus.cc:993] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a: : Instructing follower a1cbca39e34743d8a40816c3bc5f2030 to start an election
I20260709 13:05:42.336194 10916 raft_consensus.cc:1081] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Signalling peer a1cbca39e34743d8a40816c3bc5f2030 to start an election
I20260709 13:05:42.338583  9834 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b228f0642a242db93a7d11c1afab0ea"
dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
 from {username='slave'} at 127.0.0.1:37840
I20260709 13:05:42.339231  9834 raft_consensus.cc:493] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:42.339519  9834 raft_consensus.cc:3060] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.349350  9978 raft_consensus.cc:2468] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 3.
I20260709 13:05:42.345571  9834 raft_consensus.cc:515] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:42.356740  9834 leader_election.cc:290] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 3 election: Requested vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321)
I20260709 13:05:42.360284  9908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b228f0642a242db93a7d11c1afab0ea" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:42.360976  9908 raft_consensus.cc:3060] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.364113  9978 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b228f0642a242db93a7d11c1afab0ea" candidate_uuid: "a1cbca39e34743d8a40816c3bc5f2030" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:42.367852  9908 raft_consensus.cc:2468] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 3.
I20260709 13:05:42.367048  9978 raft_consensus.cc:3055] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:42.375651  9978 raft_consensus.cc:740] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:42.375875  9801 leader_election.cc:304] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e10aab26c644389b8161974dcabc5d0, a1cbca39e34743d8a40816c3bc5f2030; no voters: 
I20260709 13:05:42.376580  9978 consensus_queue.cc:260] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:42.377003 10942 raft_consensus.cc:2804] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:42.383955  9978 raft_consensus.cc:3060] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.392174  9978 raft_consensus.cc:2468] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cbca39e34743d8a40816c3bc5f2030 in term 3.
I20260709 13:05:42.394435 10942 raft_consensus.cc:697] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 3 LEADER]: Becoming Leader. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:42.395403 10942 consensus_queue.cc:237] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:42.404497  9656 catalog_manager.cc:5697] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 reported cstate change: term changed from 2 to 3, leader changed from 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196) to a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "a1cbca39e34743d8a40816c3bc5f2030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.450623  9979 raft_consensus.cc:1275] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:42.452031 10937 consensus_queue.cc:1048] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:42.464201  9900 raft_consensus.cc:1275] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:42.465703 10937 consensus_queue.cc:1048] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:42.752153  9900 raft_consensus.cc:1275] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:42.753254 10854 consensus_queue.cc:1048] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:42.771072  9978 raft_consensus.cc:1275] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:42.772686 10854 consensus_queue.cc:1048] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:42.790024  9979 raft_consensus.cc:1275] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Refusing update from remote peer a1cbca39e34743d8a40816c3bc5f2030: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:42.791165 10949 consensus_queue.cc:1048] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:42.810869  9900 raft_consensus.cc:1275] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Refusing update from remote peer a1cbca39e34743d8a40816c3bc5f2030: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:42.812224 10949 consensus_queue.cc:1048] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:42.961377   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:43.004124   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:43.041821  9826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352"
dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
mode: GRACEFUL
new_leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
 from {username='slave'} at 127.0.0.1:33390
I20260709 13:05:43.042464  9826 raft_consensus.cc:606] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 LEADER]: Received request to transfer leadership to TS 23cbd6234c5e43d89f22c235ed6bb96a
I20260709 13:05:43.043951   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:05:43.045027   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:43.265129 10848 raft_consensus.cc:993] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030: : Instructing follower 23cbd6234c5e43d89f22c235ed6bb96a to start an election
I20260709 13:05:43.265758 10848 raft_consensus.cc:1081] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 LEADER]: Signalling peer 23cbd6234c5e43d89f22c235ed6bb96a to start an election
I20260709 13:05:43.267544  9978 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
 from {username='slave'} at 127.0.0.1:54620
I20260709 13:05:43.268425  9978 raft_consensus.cc:493] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:43.268827  9978 raft_consensus.cc:3060] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.276003  9978 raft_consensus.cc:515] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:43.278275  9978 leader_election.cc:290] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 election: Requested vote from peers a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259), 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019)
I20260709 13:05:43.278720  9826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:43.278980  9900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4f626ff2a94a99bb6505423eba3352" candidate_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "6e10aab26c644389b8161974dcabc5d0"
I20260709 13:05:43.279412  9826 raft_consensus.cc:3055] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:43.279488  9900 raft_consensus.cc:3060] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.279810  9826 raft_consensus.cc:740] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a1cbca39e34743d8a40816c3bc5f2030, State: Running, Role: LEADER
I20260709 13:05:43.280601  9826 consensus_queue.cc:260] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:43.281963  9826 raft_consensus.cc:3060] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.289492  9900 raft_consensus.cc:2468] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 3.
I20260709 13:05:43.290993  9949 leader_election.cc:304] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 6e10aab26c644389b8161974dcabc5d0; no voters: 
I20260709 13:05:43.291379  9826 raft_consensus.cc:2468] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23cbd6234c5e43d89f22c235ed6bb96a in term 3.
I20260709 13:05:43.291996 10962 raft_consensus.cc:2804] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:43.292579 10962 raft_consensus.cc:697] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 LEADER]: Becoming Leader. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:43.293509 10962 consensus_queue.cc:237] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } }
I20260709 13:05:43.307035  9654 catalog_manager.cc:5697] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a reported cstate change: term changed from 2 to 3, leader changed from a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194) to 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196). New cstate: current_term: 3 leader_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:43.708945  9900 raft_consensus.cc:1275] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:43.710371 10962 consensus_queue.cc:1048] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:43.726379  9834 raft_consensus.cc:1275] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Refusing update from remote peer 23cbd6234c5e43d89f22c235ed6bb96a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:43.728508 10916 consensus_queue.cc:1048] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:44.046484   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:44.114899   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:44.166137  9979 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10a930896e984848a0516f17a1213ab5"
dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
mode: GRACEFUL
new_leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:51936
I20260709 13:05:44.167220  9979 raft_consensus.cc:606] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Received request to transfer leadership to TS 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:44.169034   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:05:44.170586   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:44.763414 10974 raft_consensus.cc:993] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a: : Instructing follower 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:44.765307 10974 raft_consensus.cc:1081] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Signalling peer 9c8d63af7ee84c38a87435cbaf6deab4 to start an election
I20260709 13:05:44.767066 10668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "10a930896e984848a0516f17a1213ab5"
dest_uuid: "9c8d63af7ee84c38a87435cbaf6deab4"
 from {username='slave'} at 127.0.0.1:49900
I20260709 13:05:44.767671 10668 raft_consensus.cc:493] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:44.768062 10668 raft_consensus.cc:3060] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.774961 10668 raft_consensus.cc:515] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:44.778093  9979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a930896e984848a0516f17a1213ab5" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "23cbd6234c5e43d89f22c235ed6bb96a"
I20260709 13:05:44.778275  9834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10a930896e984848a0516f17a1213ab5" candidate_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a1cbca39e34743d8a40816c3bc5f2030"
I20260709 13:05:44.778684  9979 raft_consensus.cc:3055] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:44.778975  9834 raft_consensus.cc:3060] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.779085  9979 raft_consensus.cc:740] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 23cbd6234c5e43d89f22c235ed6bb96a, State: Running, Role: LEADER
I20260709 13:05:44.779929  9979 consensus_queue.cc:260] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:44.780988  9979 raft_consensus.cc:3060] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.788275  9834 raft_consensus.cc:2468] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:44.788975  9979 raft_consensus.cc:2468] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c8d63af7ee84c38a87435cbaf6deab4 in term 2.
I20260709 13:05:44.789289 10668 leader_election.cc:290] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Requested vote from peers 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196:34321), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:44.793105 10546 leader_election.cc:304] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a, 9c8d63af7ee84c38a87435cbaf6deab4; no voters: 
I20260709 13:05:44.796911 10975 raft_consensus.cc:2804] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:44.797432 10975 raft_consensus.cc:697] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Becoming Leader. State: Replica: 9c8d63af7ee84c38a87435cbaf6deab4, State: Running, Role: LEADER
I20260709 13:05:44.800593 10975 consensus_queue.cc:237] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:44.809702  9656 catalog_manager.cc:5697] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 reported cstate change: term changed from 1 to 2, leader changed from 23cbd6234c5e43d89f22c235ed6bb96a (127.0.107.196) to 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:45.171657   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:45.190807  9826 raft_consensus.cc:1275] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:45.192600 10977 consensus_queue.cc:1048] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:45.224277   431 tablet_server.cc:179] TabletServer@127.0.107.193:41383 shutting down...
I20260709 13:05:45.244990  9978 raft_consensus.cc:1275] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Refusing update from remote peer 9c8d63af7ee84c38a87435cbaf6deab4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:45.248152 10854 consensus_queue.cc:1048] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:45.255441  9798 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 13 similar messages]
W20260709 13:05:45.324579  9798 consensus_peers.cc:597] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.338908  9798 consensus_peers.cc:597] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.339485  9872 consensus_peers.cc:597] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.344442  9872 consensus_peers.cc:597] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.348731  9872 consensus_peers.cc:597] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.348594  9946 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:41383, user_credentials={real_user=slave}} blocked reactor thread for 51377.7us
W20260709 13:05:45.349977  9798 consensus_peers.cc:597] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.362035  9946 consensus_peers.cc:597] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.369202  9946 consensus_peers.cc:597] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.369979  9946 consensus_peers.cc:597] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.370961  9946 consensus_peers.cc:597] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.371651  9946 consensus_peers.cc:597] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.383069  9798 consensus_peers.cc:597] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.390388  9872 consensus_peers.cc:597] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.401659   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:45.402699   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.403759   431 raft_consensus.cc:2243] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.404392   431 raft_consensus.cc:2272] T d41e6a20c04a4d99ba9b1ea99106480a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.407130   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.408157   431 raft_consensus.cc:2243] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.408761   431 raft_consensus.cc:2272] T fe46065991d847dfa3d934961da0830a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.411947   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.412827   431 raft_consensus.cc:2243] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.413326   431 raft_consensus.cc:2272] T a8a61e076bba425385e5327d369522de P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.416198   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.417193   431 raft_consensus.cc:2243] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.418080   431 raft_consensus.cc:2272] T 96d8c823d37d4e9a864b46403d267028 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.421160   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.422118   431 raft_consensus.cc:2243] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.422705   431 raft_consensus.cc:2272] T d69b492d298d42e18ab6b82be92774ca P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.425721   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.426409   431 raft_consensus.cc:2243] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.426970   431 raft_consensus.cc:2272] T f9623952612f4929825a90c043f1c6ba P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.429451   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.430188   431 raft_consensus.cc:2243] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.430708   431 raft_consensus.cc:2272] T 27c539d1a0b54f8ca31dba142cb38599 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.433246   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.433872   431 raft_consensus.cc:2243] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.434320   431 raft_consensus.cc:2272] T a6bd3b28a03c44fda0c31e4198a86726 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.436508   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.437234   431 raft_consensus.cc:2243] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.437775   431 raft_consensus.cc:2272] T 748a545c945244afa32bec8019157d0c P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.441596   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.442670   431 raft_consensus.cc:2243] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.443271   431 raft_consensus.cc:2272] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.446275   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:05:45.446472  9798 consensus_peers.cc:597] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.447412   431 raft_consensus.cc:2243] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.448501   431 raft_consensus.cc:2272] T b189ef5046cd47ebb3d431e2ef24d6fb P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.451200   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.452065   431 raft_consensus.cc:2243] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:45.452229  9946 consensus_peers.cc:597] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.452622   431 raft_consensus.cc:2272] T 716c25a1e46547c999c1c7c860e23f3b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.456070   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.457046   431 raft_consensus.cc:2243] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:05:45.457296  9872 consensus_peers.cc:597] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.458093   431 raft_consensus.cc:2272] T eeefc94068a14a0b9dd656a55fd68ccc P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.461212   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.462080   431 raft_consensus.cc:2243] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.463029   431 raft_consensus.cc:2272] T 778ba1e8c7b04438a6633db058c14508 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.465207   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.465934   431 raft_consensus.cc:2243] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.466523   431 raft_consensus.cc:2272] T 6a2d12d5566b4c82a5572035a797a82d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.469352   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.470297   431 raft_consensus.cc:2243] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:05:45.471081  9946 consensus_peers.cc:597] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.471300   431 raft_consensus.cc:2272] T d8f99aa4491c4458bb5ebc03bc0cd813 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.473780   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.474447   431 raft_consensus.cc:2243] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.475023   431 raft_consensus.cc:2272] T b6ff13f827e7411d9a5782fb06e93c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.477183   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.477926   431 raft_consensus.cc:2243] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.478358   431 raft_consensus.cc:2272] T 1dddadb853be408b8e6a1103a45ce0a1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.480458   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.481304   431 raft_consensus.cc:2243] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.481771   431 raft_consensus.cc:2272] T 622d51c8b2124715a814796353745c9b P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.485009   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.485855   431 raft_consensus.cc:2243] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.486414   431 raft_consensus.cc:2272] T 9579648f7487465aa44ecf73e23731d8 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.489653   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.490322   431 raft_consensus.cc:2243] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 LEADER]: Raft consensus shutting down.
W20260709 13:05:45.490811  9872 consensus_peers.cc:597] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.491376   431 raft_consensus.cc:2272] T e80fbd6291544644996b96fec0e8f2c3 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.493588   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.494189   431 raft_consensus.cc:2243] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.495024   431 raft_consensus.cc:2272] T 30b0a6c62c14440c9e613e73781c542e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.497885   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.498562   431 raft_consensus.cc:2243] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.499138   431 raft_consensus.cc:2272] T fbbee4fe35264c5aa3ea32adc034b9c7 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.501677   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.502625   431 raft_consensus.cc:2243] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.503762   431 raft_consensus.cc:2272] T 1064d1c0e3ba4ab68fb20fd5f093abef P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.507344   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:05:45.507462  9872 consensus_peers.cc:597] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.508266   431 raft_consensus.cc:2243] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.509131   431 raft_consensus.cc:2272] T 755ff96760b04fd49a520c059f90b1fe P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.511397   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.511987   431 raft_consensus.cc:2243] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.512399   431 raft_consensus.cc:2272] T 148fd9eeaa2647bca8ad75116e461ac6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.514482   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.515173   431 raft_consensus.cc:2243] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.515662   431 raft_consensus.cc:2272] T af25fe6172fa4032a6570d9a3c7757c6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.518311   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.519243   431 raft_consensus.cc:2243] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.519820   431 raft_consensus.cc:2272] T dc0e147704a7413397055b8d3a812a8e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.521914   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.522426   431 raft_consensus.cc:2243] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.522799   431 raft_consensus.cc:2272] T c2684c424702483b92758b1173702381 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.524853   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.525385   431 raft_consensus.cc:2243] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.526124   431 raft_consensus.cc:2272] T 10a930896e984848a0516f17a1213ab5 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.529248   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.529974   431 raft_consensus.cc:2243] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.530565   431 raft_consensus.cc:2272] T e82b4b9bc0c546adac703578be161959 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.533413   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.534224   431 raft_consensus.cc:2243] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.535082   431 raft_consensus.cc:2272] T ab473e8b9be44bdaae62679a55757fe9 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.537483   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.538106   431 raft_consensus.cc:2243] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.538525   431 raft_consensus.cc:2272] T aaf1d483dd4049a68642f7d94f1cfc70 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.540810   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.541476   431 raft_consensus.cc:2243] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.541874   431 raft_consensus.cc:2272] T ecae0626a5c744f0ad9571147ccc239d P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.544603   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.545503   431 raft_consensus.cc:2243] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.546017   431 raft_consensus.cc:2272] T 4b0376b68ba34b8596a5ad96711a0c6e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.548777   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.549517   431 raft_consensus.cc:2243] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.550150   431 raft_consensus.cc:2272] T a981502782314272a8a9bee8da734427 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:45.553231  9946 consensus_peers.cc:597] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.553449   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.554265   431 raft_consensus.cc:2243] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.554719   431 raft_consensus.cc:2272] T 196922ea9dc14e078a3da6af1206f79a P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.557238   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.558017   431 raft_consensus.cc:2243] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.559070   431 raft_consensus.cc:2272] T 98a516bf1d774804b94074f9765006ad P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.561904   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:05:45.562579  9798 consensus_peers.cc:597] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.562708   431 raft_consensus.cc:2243] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.563551   431 raft_consensus.cc:2272] T e6f94cfb8a734a83872ed245aeea11e6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:45.565960  9872 consensus_peers.cc:597] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.566550   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.567345   431 raft_consensus.cc:2243] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.567842   431 raft_consensus.cc:2272] T db11dbab2c874e55b70a46aba53095e0 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.570753   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.571578   431 raft_consensus.cc:2243] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.572109   431 raft_consensus.cc:2272] T 23871dd822614a5da07c2289f80141c1 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.575321   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.576126   431 raft_consensus.cc:2243] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.576680   431 raft_consensus.cc:2272] T 3505eea4224942d7a938f9eaf7427723 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.579708   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.580309   431 raft_consensus.cc:2243] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.580807   431 raft_consensus.cc:2272] T c23fd23d082a4c58ab8404e8b9895ad4 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.583137   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.583932   431 raft_consensus.cc:2243] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.584857   431 raft_consensus.cc:2272] T 251be49566d64412bd3d6e3d72c5795e P 9c8d63af7ee84c38a87435cbaf6deab4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.587731   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:05:45.588387   431 raft_consensus.cc:2243] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.589531   431 raft_consensus.cc:2272] T 168ffc86a17d47e8aba438779717eab6 P 9c8d63af7ee84c38a87435cbaf6deab4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:45.603549  9946 consensus_peers.cc:597] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.610337  9798 consensus_peers.cc:597] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.619550  9798 consensus_peers.cc:597] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.621215  9946 consensus_peers.cc:597] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.627745  9798 consensus_peers.cc:597] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.651678  9798 consensus_peers.cc:597] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.672588  9946 consensus_peers.cc:597] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.680014   431 tablet_server.cc:196] TabletServer@127.0.107.193:41383 shutdown complete.
W20260709 13:05:45.717068  9798 consensus_peers.cc:597] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.740199  9798 consensus_peers.cc:597] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 -> Peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Couldn't send request to peer 9c8d63af7ee84c38a87435cbaf6deab4. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.796784   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:05:45.818547  9874 consensus_peers.cc:597] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.830132  9874 consensus_peers.cc:597] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:45.832355  9874 consensus_peers.cc:597] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.890707   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:45.891896   431 tablet_replica.cc:333] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.893047   431 raft_consensus.cc:2243] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.893603   431 raft_consensus.cc:2272] T 140e604980a14cfe9b9ee5918dc45bb2 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.896766   431 tablet_replica.cc:333] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.897745   431 raft_consensus.cc:2243] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.898365   431 raft_consensus.cc:2272] T 96d8c823d37d4e9a864b46403d267028 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.901019   431 tablet_replica.cc:333] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.901830   431 raft_consensus.cc:2243] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.902413   431 raft_consensus.cc:2272] T 912bd6d0efac4042884ce6823f7e8e73 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.905349   431 tablet_replica.cc:333] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.906183   431 raft_consensus.cc:2243] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.906805   431 raft_consensus.cc:2272] T 4cd54dca51434fcf8e5a473955555bfa P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.909715   431 tablet_replica.cc:333] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.910606   431 raft_consensus.cc:2243] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.911187   431 raft_consensus.cc:2272] T 10a930896e984848a0516f17a1213ab5 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.915182   431 tablet_replica.cc:333] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.915889   431 raft_consensus.cc:2243] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.916435   431 raft_consensus.cc:2272] T e6f94cfb8a734a83872ed245aeea11e6 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.919269   431 tablet_replica.cc:333] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.919919   431 raft_consensus.cc:2243] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.920543   431 raft_consensus.cc:2272] T 1064d1c0e3ba4ab68fb20fd5f093abef P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.923234   431 tablet_replica.cc:333] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.923888   431 raft_consensus.cc:2243] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.925246   431 raft_consensus.cc:2272] T dc0e147704a7413397055b8d3a812a8e P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.928131   431 tablet_replica.cc:333] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
W20260709 13:05:45.929122  9874 consensus_peers.cc:597] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:45.929170   431 raft_consensus.cc:2243] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.930442   431 raft_consensus.cc:2272] T e01c76363b154905982dc75dbdead3be P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.933101   431 tablet_replica.cc:333] T c23fd23d082a4c58ab8404e8b9895ad4 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.933851   431 raft_consensus.cc:2243] T c23fd23d082a4c58ab8404e8b9895ad4 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.934432   431 raft_consensus.cc:2272] T c23fd23d082a4c58ab8404e8b9895ad4 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.937373   431 tablet_replica.cc:333] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.938280   431 raft_consensus.cc:2243] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.939343   431 raft_consensus.cc:2272] T 196922ea9dc14e078a3da6af1206f79a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.941778   431 tablet_replica.cc:333] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.942382   431 raft_consensus.cc:2243] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.943395   431 raft_consensus.cc:2272] T 622d51c8b2124715a814796353745c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.946178   431 tablet_replica.cc:333] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.946869   431 raft_consensus.cc:2243] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.947430   431 raft_consensus.cc:2272] T c0c62d82ebe5420bb298d787cda221aa P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.949577   431 tablet_replica.cc:333] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.950052   431 raft_consensus.cc:2243] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.950969   431 raft_consensus.cc:2272] T 1dddadb853be408b8e6a1103a45ce0a1 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.952981   431 tablet_replica.cc:333] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.953563   431 raft_consensus.cc:2243] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.954005   431 raft_consensus.cc:2272] T 9579648f7487465aa44ecf73e23731d8 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.956094   431 tablet_replica.cc:333] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.956705   431 raft_consensus.cc:2243] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.957474   431 raft_consensus.cc:2272] T f9623952612f4929825a90c043f1c6ba P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.959525   431 tablet_replica.cc:333] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.960048   431 raft_consensus.cc:2243] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.960425   431 raft_consensus.cc:2272] T adca2c30b75149c988cf71f9effb9d6c P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.962387   431 tablet_replica.cc:333] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.962896   431 raft_consensus.cc:2243] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.963342   431 raft_consensus.cc:2272] T c2d7bb9a0b174b168df9040b02c70ab9 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.965937   431 tablet_replica.cc:333] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.966990   431 raft_consensus.cc:2243] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.967582   431 raft_consensus.cc:2272] T a6bd3b28a03c44fda0c31e4198a86726 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.970212   431 tablet_replica.cc:333] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.971079   431 raft_consensus.cc:2243] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.971937   431 raft_consensus.cc:2272] T 748a545c945244afa32bec8019157d0c P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.974429   431 tablet_replica.cc:333] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.975343   431 raft_consensus.cc:2243] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.976485   431 raft_consensus.cc:2272] T 27c539d1a0b54f8ca31dba142cb38599 P a1cbca39e34743d8a40816c3bc5f2030 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.979384   431 tablet_replica.cc:333] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.980070   431 raft_consensus.cc:2243] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.981046   431 raft_consensus.cc:2272] T aa9f688643024d419097e12e80ced0d2 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.983224   431 tablet_replica.cc:333] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.983866   431 raft_consensus.cc:2243] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.984383   431 raft_consensus.cc:2272] T ea4f626ff2a94a99bb6505423eba3352 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.986405   431 tablet_replica.cc:333] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.986989   431 raft_consensus.cc:2243] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.987486   431 raft_consensus.cc:2272] T 0e044b992b684d52b54e3e8a4aa9720a P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.989428   431 tablet_replica.cc:333] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.989933   431 raft_consensus.cc:2243] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.990334   431 raft_consensus.cc:2272] T db11dbab2c874e55b70a46aba53095e0 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.992429   431 tablet_replica.cc:333] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.993088   431 raft_consensus.cc:2243] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:45.993877   431 raft_consensus.cc:2272] T 23871dd822614a5da07c2289f80141c1 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.996034   431 tablet_replica.cc:333] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.996682   431 raft_consensus.cc:2243] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:45.997109   431 raft_consensus.cc:2272] T b6ff13f827e7411d9a5782fb06e93c9b P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:45.999321   431 tablet_replica.cc:333] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:45.999997   431 raft_consensus.cc:2243] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.000468   431 raft_consensus.cc:2272] T 6a2d12d5566b4c82a5572035a797a82d P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.002508   431 tablet_replica.cc:333] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.003144   431 raft_consensus.cc:2243] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.003934   431 raft_consensus.cc:2272] T 2b228f0642a242db93a7d11c1afab0ea P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.007087   431 tablet_replica.cc:333] T ba93a39b54a742c6b391a2eb491f3d79 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.007916   431 raft_consensus.cc:2243] T ba93a39b54a742c6b391a2eb491f3d79 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.008505   431 raft_consensus.cc:2272] T ba93a39b54a742c6b391a2eb491f3d79 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.011387   431 tablet_replica.cc:333] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.012161   431 raft_consensus.cc:2243] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.012737   431 raft_consensus.cc:2272] T e80fbd6291544644996b96fec0e8f2c3 P a1cbca39e34743d8a40816c3bc5f2030 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.015496   431 tablet_replica.cc:333] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.016355   431 raft_consensus.cc:2243] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:46.016426  9948 consensus_peers.cc:597] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.016945   431 raft_consensus.cc:2272] T b189ef5046cd47ebb3d431e2ef24d6fb P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.018988   431 tablet_replica.cc:333] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.019546   431 raft_consensus.cc:2243] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.020414   431 raft_consensus.cc:2272] T aec2f732f8024a00ae0b4d7bc00bb8e3 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.022419   431 tablet_replica.cc:333] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.023059   431 raft_consensus.cc:2243] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.023849   431 raft_consensus.cc:2272] T 3505eea4224942d7a938f9eaf7427723 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.026979   431 tablet_replica.cc:333] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.027796   431 raft_consensus.cc:2243] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.028391   431 raft_consensus.cc:2272] T ecae0626a5c744f0ad9571147ccc239d P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:46.031237  9948 consensus_peers.cc:597] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.031241   431 tablet_replica.cc:333] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.032475   431 raft_consensus.cc:2243] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.033023   431 raft_consensus.cc:2272] T 4b0376b68ba34b8596a5ad96711a0c6e P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:46.033655  9874 consensus_peers.cc:597] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.035565   431 tablet_replica.cc:333] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.036260   431 raft_consensus.cc:2243] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.036746   431 raft_consensus.cc:2272] T b011ab28e64b483db8daaf9ed27f45ce P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.038694   431 tablet_replica.cc:333] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.039289   431 raft_consensus.cc:2243] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.039701   431 raft_consensus.cc:2272] T e82b4b9bc0c546adac703578be161959 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.041653   431 tablet_replica.cc:333] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.042243   431 raft_consensus.cc:2243] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.043090   431 raft_consensus.cc:2272] T aaf1d483dd4049a68642f7d94f1cfc70 P a1cbca39e34743d8a40816c3bc5f2030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.045028   431 tablet_replica.cc:333] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.045570   431 raft_consensus.cc:2243] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.046027   431 raft_consensus.cc:2272] T eeefc94068a14a0b9dd656a55fd68ccc P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.048132   431 tablet_replica.cc:333] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.048678   431 raft_consensus.cc:2243] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.049180   431 raft_consensus.cc:2272] T b799cecf522841009ed9a2b1b421be50 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.051105   431 tablet_replica.cc:333] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.051666   431 raft_consensus.cc:2243] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.052179   431 raft_consensus.cc:2272] T d41e6a20c04a4d99ba9b1ea99106480a P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.054946   431 tablet_replica.cc:333] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
I20260709 13:05:46.055688   431 raft_consensus.cc:2243] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.056269   431 raft_consensus.cc:2272] T 251be49566d64412bd3d6e3d72c5795e P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.059010   431 tablet_replica.cc:333] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030: stopping tablet replica
W20260709 13:05:46.059471  9874 consensus_peers.cc:597] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.059558   431 raft_consensus.cc:2243] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.060540   431 raft_consensus.cc:2272] T 148fd9eeaa2647bca8ad75116e461ac6 P a1cbca39e34743d8a40816c3bc5f2030 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:46.065521  9948 consensus_peers.cc:597] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.068708  9948 consensus_peers.cc:597] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.088546  9874 consensus_peers.cc:597] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.105799  9874 consensus_peers.cc:597] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.115711  9874 consensus_peers.cc:597] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.139329  9874 consensus_peers.cc:597] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:34259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.172775  9948 consensus_peers.cc:597] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:34259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.179253   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:05:46.186676  9874 consensus_peers.cc:597] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:34259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.187371  9948 consensus_peers.cc:597] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:34259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.238452  9874 consensus_peers.cc:597] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:34259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.238524  9948 consensus_peers.cc:597] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:34259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.245679  9948 consensus_peers.cc:597] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:34259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.278180   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:05:46.306468  9874 consensus_peers.cc:597] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:34259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.320156  9949 consensus_peers.cc:597] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Couldn't send request to peer 6e10aab26c644389b8161974dcabc5d0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.328580  9948 consensus_peers.cc:597] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Couldn't send request to peer a1cbca39e34743d8a40816c3bc5f2030. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:34259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.332654   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:46.333774   431 tablet_replica.cc:333] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.334582   431 raft_consensus.cc:2243] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.335131   431 raft_consensus.cc:2272] T 98a516bf1d774804b94074f9765006ad P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.337567   431 tablet_replica.cc:333] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.338177   431 raft_consensus.cc:2243] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.339020   431 raft_consensus.cc:2272] T 140e604980a14cfe9b9ee5918dc45bb2 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.340945   431 tablet_replica.cc:333] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.341540   431 raft_consensus.cc:2243] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.342334   431 raft_consensus.cc:2272] T ecae0626a5c744f0ad9571147ccc239d P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.344493   431 tablet_replica.cc:333] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.345178   431 raft_consensus.cc:2243] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.345788   431 raft_consensus.cc:2272] T a981502782314272a8a9bee8da734427 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.348704   431 tablet_replica.cc:333] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.349560   431 raft_consensus.cc:2243] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.350531   431 raft_consensus.cc:2272] T 912bd6d0efac4042884ce6823f7e8e73 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:46.351729  9949 consensus_peers.cc:597] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Couldn't send request to peer 6e10aab26c644389b8161974dcabc5d0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.353348   431 tablet_replica.cc:333] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.354036   431 raft_consensus.cc:2243] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.354480   431 raft_consensus.cc:2272] T d69b492d298d42e18ab6b82be92774ca P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.356535   431 tablet_replica.cc:333] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.357147   431 raft_consensus.cc:2243] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.357611   431 raft_consensus.cc:2272] T fe46065991d847dfa3d934961da0830a P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.359699   431 tablet_replica.cc:333] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.360306   431 raft_consensus.cc:2243] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.360888   431 raft_consensus.cc:2272] T e01c76363b154905982dc75dbdead3be P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.363497   431 tablet_replica.cc:333] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.364290   431 raft_consensus.cc:2243] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.364826   431 raft_consensus.cc:2272] T d8f99aa4491c4458bb5ebc03bc0cd813 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.367129   431 tablet_replica.cc:333] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.367782   431 raft_consensus.cc:2243] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.368292   431 raft_consensus.cc:2272] T 196922ea9dc14e078a3da6af1206f79a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.370296   431 tablet_replica.cc:333] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.370891   431 raft_consensus.cc:2243] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.371789   431 raft_consensus.cc:2272] T c23fd23d082a4c58ab8404e8b9895ad4 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.373885   431 tablet_replica.cc:333] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.374495   431 raft_consensus.cc:2243] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.375006   431 raft_consensus.cc:2272] T f9623952612f4929825a90c043f1c6ba P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.376914   431 tablet_replica.cc:333] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.377528   431 raft_consensus.cc:2243] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.378314   431 raft_consensus.cc:2272] T adca2c30b75149c988cf71f9effb9d6c P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.380396   431 tablet_replica.cc:333] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.380992   431 raft_consensus.cc:2243] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.381436   431 raft_consensus.cc:2272] T c2d7bb9a0b174b168df9040b02c70ab9 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.383816   431 tablet_replica.cc:333] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.384487   431 raft_consensus.cc:2243] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.385006   431 raft_consensus.cc:2272] T 30b0a6c62c14440c9e613e73781c542e P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.387414   431 tablet_replica.cc:333] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.388031   431 raft_consensus.cc:2243] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.388489   431 raft_consensus.cc:2272] T 2b228f0642a242db93a7d11c1afab0ea P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.390600   431 tablet_replica.cc:333] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.391294   431 raft_consensus.cc:2243] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.392112   431 raft_consensus.cc:2272] T ba93a39b54a742c6b391a2eb491f3d79 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.394539   431 tablet_replica.cc:333] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.395205   431 raft_consensus.cc:2243] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.395992   431 raft_consensus.cc:2272] T fbbee4fe35264c5aa3ea32adc034b9c7 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.398130   431 tablet_replica.cc:333] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.398751   431 raft_consensus.cc:2243] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.399569   431 raft_consensus.cc:2272] T c0c62d82ebe5420bb298d787cda221aa P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.401433   431 tablet_replica.cc:333] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.402120   431 raft_consensus.cc:2243] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.402535   431 raft_consensus.cc:2272] T 1dddadb853be408b8e6a1103a45ce0a1 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.404431   431 tablet_replica.cc:333] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.405004   431 raft_consensus.cc:2243] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.405426   431 raft_consensus.cc:2272] T 716c25a1e46547c999c1c7c860e23f3b P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.407739   431 tablet_replica.cc:333] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.408324   431 raft_consensus.cc:2243] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.408823   431 raft_consensus.cc:2272] T 1064d1c0e3ba4ab68fb20fd5f093abef P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.411459   431 tablet_replica.cc:333] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.412155   431 raft_consensus.cc:2243] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.412669   431 raft_consensus.cc:2272] T 755ff96760b04fd49a520c059f90b1fe P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.414813   431 tablet_replica.cc:333] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.415475   431 raft_consensus.cc:2243] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.416016   431 raft_consensus.cc:2272] T 3505eea4224942d7a938f9eaf7427723 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.418113   431 tablet_replica.cc:333] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.418823   431 raft_consensus.cc:2243] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.419891   431 raft_consensus.cc:2272] T af25fe6172fa4032a6570d9a3c7757c6 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.422130   431 tablet_replica.cc:333] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.422772   431 raft_consensus.cc:2243] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.423290   431 raft_consensus.cc:2272] T ea4f626ff2a94a99bb6505423eba3352 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.425374   431 tablet_replica.cc:333] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.426002   431 raft_consensus.cc:2243] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.426872   431 raft_consensus.cc:2272] T 0e044b992b684d52b54e3e8a4aa9720a P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.429764   431 tablet_replica.cc:333] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.430577   431 raft_consensus.cc:2243] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.431694   431 raft_consensus.cc:2272] T db11dbab2c874e55b70a46aba53095e0 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.435001   431 tablet_replica.cc:333] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
W20260709 13:05:46.435461  9949 consensus_peers.cc:597] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Couldn't send request to peer 6e10aab26c644389b8161974dcabc5d0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.435658   431 raft_consensus.cc:2243] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.436302   431 raft_consensus.cc:2272] T 778ba1e8c7b04438a6633db058c14508 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.438983   431 tablet_replica.cc:333] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.439567   431 raft_consensus.cc:2243] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.440006   431 raft_consensus.cc:2272] T eeefc94068a14a0b9dd656a55fd68ccc P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.442224   431 tablet_replica.cc:333] T a8a61e076bba425385e5327d369522de P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.442860   431 raft_consensus.cc:2243] T a8a61e076bba425385e5327d369522de P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.443611   431 raft_consensus.cc:2272] T a8a61e076bba425385e5327d369522de P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.445739   431 tablet_replica.cc:333] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.446362   431 raft_consensus.cc:2243] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.446803   431 raft_consensus.cc:2272] T b799cecf522841009ed9a2b1b421be50 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.449199   431 tablet_replica.cc:333] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.449937   431 raft_consensus.cc:2243] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.451020   431 raft_consensus.cc:2272] T d41e6a20c04a4d99ba9b1ea99106480a P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.453267   431 tablet_replica.cc:333] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.453809   431 raft_consensus.cc:2243] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.454212   431 raft_consensus.cc:2272] T 168ffc86a17d47e8aba438779717eab6 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.456278   431 tablet_replica.cc:333] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.456804   431 raft_consensus.cc:2243] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.457201   431 raft_consensus.cc:2272] T 251be49566d64412bd3d6e3d72c5795e P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.460008   431 tablet_replica.cc:333] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.460557   431 raft_consensus.cc:2243] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.461457   431 raft_consensus.cc:2272] T a6bd3b28a03c44fda0c31e4198a86726 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.463328   431 tablet_replica.cc:333] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.463843   431 raft_consensus.cc:2243] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.464246   431 raft_consensus.cc:2272] T 748a545c945244afa32bec8019157d0c P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.466243   431 tablet_replica.cc:333] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.466748   431 raft_consensus.cc:2243] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.467216   431 raft_consensus.cc:2272] T aa9f688643024d419097e12e80ced0d2 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.469368   431 tablet_replica.cc:333] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.469993   431 raft_consensus.cc:2243] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.470425   431 raft_consensus.cc:2272] T 4cd54dca51434fcf8e5a473955555bfa P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.472747   431 tablet_replica.cc:333] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.473265   431 raft_consensus.cc:2243] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.473654   431 raft_consensus.cc:2272] T c2684c424702483b92758b1173702381 P 6e10aab26c644389b8161974dcabc5d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.475638   431 tablet_replica.cc:333] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.476132   431 raft_consensus.cc:2243] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.477190   431 raft_consensus.cc:2272] T b011ab28e64b483db8daaf9ed27f45ce P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.479250   431 tablet_replica.cc:333] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.479931   431 raft_consensus.cc:2243] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.481017   431 raft_consensus.cc:2272] T e82b4b9bc0c546adac703578be161959 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.483839   431 tablet_replica.cc:333] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
W20260709 13:05:46.484066  9949 consensus_peers.cc:597] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Couldn't send request to peer 6e10aab26c644389b8161974dcabc5d0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.484513   431 raft_consensus.cc:2243] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.485126   431 raft_consensus.cc:2272] T ab473e8b9be44bdaae62679a55757fe9 P 6e10aab26c644389b8161974dcabc5d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.487422   431 tablet_replica.cc:333] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0: stopping tablet replica
I20260709 13:05:46.487851   431 raft_consensus.cc:2243] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.488235   431 raft_consensus.cc:2272] T aaf1d483dd4049a68642f7d94f1cfc70 P 6e10aab26c644389b8161974dcabc5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.557981   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:05:46.577422 10974 raft_consensus.cc:493] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:46.578070 10974 raft_consensus.cc:515] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:46.580109 10974 leader_election.cc:290] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
W20260709 13:05:46.586252  9948 leader_election.cc:336] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Network error: Client connection negotiation failed: client connection to 127.0.107.194:34259: connect: Connection refused (error 111)
W20260709 13:05:46.587122  9946 leader_election.cc:336] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:46.587630  9946 leader_election.cc:304] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:46.588482 10962 raft_consensus.cc:2749] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:46.607667 10974 raft_consensus.cc:493] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:46.608245 10974 raft_consensus.cc:515] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:46.610342 10974 leader_election.cc:290] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
W20260709 13:05:46.617422  9946 leader_election.cc:336] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:46.619341 10974 raft_consensus.cc:493] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:46.619892 10974 raft_consensus.cc:515] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
W20260709 13:05:46.620981  9948 leader_election.cc:336] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Network error: Client connection negotiation failed: client connection to 127.0.107.194:34259: connect: Connection refused (error 111)
I20260709 13:05:46.622419  9948 leader_election.cc:304] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:46.623209 10976 raft_consensus.cc:2749] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:46.623399 10974 leader_election.cc:290] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
W20260709 13:05:46.631106  9949 leader_election.cc:336] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45019: connect: Connection refused (error 111)
W20260709 13:05:46.631987  9946 leader_election.cc:336] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:46.632447  9946 leader_election.cc:304] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:46.633203 10974 raft_consensus.cc:2749] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:05:46.643976  9949 consensus_peers.cc:597] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Couldn't send request to peer 6e10aab26c644389b8161974dcabc5d0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45019: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.648835   431 tablet_server.cc:179] TabletServer@127.0.107.196:0 shutting down...
W20260709 13:05:46.688644  9949 consensus_peers.cc:597] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Couldn't send request to peer 6e10aab26c644389b8161974dcabc5d0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45019: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.692987 10976 raft_consensus.cc:493] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:46.693603 10976 raft_consensus.cc:515] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:46.696548 10976 leader_election.cc:290] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
W20260709 13:05:46.708441  9949 consensus_peers.cc:597] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Couldn't send request to peer 6e10aab26c644389b8161974dcabc5d0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45019: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:46.708997  9949 leader_election.cc:336] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45019: connect: Connection refused (error 111)
W20260709 13:05:46.709848  9946 leader_election.cc:336] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:46.710356  9946 leader_election.cc:304] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:46.711192 10976 raft_consensus.cc:2749] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.721448 10976 raft_consensus.cc:493] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:46.721951 10976 raft_consensus.cc:515] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:46.722586   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:46.723614   431 tablet_replica.cc:333] T 140e604980a14cfe9b9ee5918dc45bb2 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.724047 10976 leader_election.cc:290] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:46.724591   431 raft_consensus.cc:2243] T 140e604980a14cfe9b9ee5918dc45bb2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.725299   431 raft_consensus.cc:2272] T 140e604980a14cfe9b9ee5918dc45bb2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.728086   431 tablet_replica.cc:333] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
W20260709 13:05:46.729228  9948 leader_election.cc:336] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Network error: Client connection negotiation failed: client connection to 127.0.107.194:34259: connect: Connection refused (error 111)
I20260709 13:05:46.729326   431 raft_consensus.cc:2243] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:46.729768  9946 leader_election.cc:336] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:46.730089   431 raft_consensus.cc:2272] T 912bd6d0efac4042884ce6823f7e8e73 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.730209  9946 leader_election.cc:304] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:46.731088 10974 raft_consensus.cc:2749] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.731659 10976 raft_consensus.cc:493] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:46.732314 10976 raft_consensus.cc:515] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:46.733119   431 tablet_replica.cc:333] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.734056   431 raft_consensus.cc:2243] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.734719 10976 leader_election.cc:290] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:46.735435   431 raft_consensus.cc:2272] T d69b492d298d42e18ab6b82be92774ca P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:46.736647  9949 consensus_peers.cc:597] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Couldn't send request to peer 6e10aab26c644389b8161974dcabc5d0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45019: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.738462   431 tablet_replica.cc:333] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
W20260709 13:05:46.739105  9949 leader_election.cc:336] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45019: connect: Connection refused (error 111)
I20260709 13:05:46.739579   431 raft_consensus.cc:2243] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.740231   431 raft_consensus.cc:2272] T 96d8c823d37d4e9a864b46403d267028 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:46.740604  9946 leader_election.cc:336] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:46.741034  9946 leader_election.cc:304] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:46.741732 10976 raft_consensus.cc:2749] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.742748   431 tablet_replica.cc:333] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.743543   431 raft_consensus.cc:2243] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.744385   431 raft_consensus.cc:2272] T fe46065991d847dfa3d934961da0830a P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.746587   431 tablet_replica.cc:333] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.747313   431 raft_consensus.cc:2243] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.748178   431 raft_consensus.cc:2272] T b799cecf522841009ed9a2b1b421be50 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.750360   431 tablet_replica.cc:333] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.751076   431 raft_consensus.cc:2243] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.751551   431 raft_consensus.cc:2272] T 98a516bf1d774804b94074f9765006ad P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.753676   431 tablet_replica.cc:333] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.754479   431 raft_consensus.cc:2243] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.755378   431 raft_consensus.cc:2272] T e6f94cfb8a734a83872ed245aeea11e6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.757607   431 tablet_replica.cc:333] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.758225   431 raft_consensus.cc:2243] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.758726   431 raft_consensus.cc:2272] T dc0e147704a7413397055b8d3a812a8e P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.760730   431 tablet_replica.cc:333] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.761315   431 raft_consensus.cc:2243] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.761766   431 raft_consensus.cc:2272] T 168ffc86a17d47e8aba438779717eab6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.763710   431 tablet_replica.cc:333] T e01c76363b154905982dc75dbdead3be P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.764272   431 raft_consensus.cc:2243] T e01c76363b154905982dc75dbdead3be P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.764707   431 raft_consensus.cc:2272] T e01c76363b154905982dc75dbdead3be P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.766649   431 tablet_replica.cc:333] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.767439   431 raft_consensus.cc:2243] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.768075   431 raft_consensus.cc:2272] T ab473e8b9be44bdaae62679a55757fe9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.769743 10976 raft_consensus.cc:493] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:46.770249 10976 raft_consensus.cc:515] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } } peers { permanent_uuid: "a1cbca39e34743d8a40816c3bc5f2030" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34259 } }
I20260709 13:05:46.770774   431 tablet_replica.cc:333] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
W20260709 13:05:46.770946  9949 consensus_peers.cc:597] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Couldn't send request to peer 6e10aab26c644389b8161974dcabc5d0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45019: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.771646   431 raft_consensus.cc:2243] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.772226   431 raft_consensus.cc:2272] T 622d51c8b2124715a814796353745c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.772378 10976 leader_election.cc:290] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383), a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259)
I20260709 13:05:46.775061   431 tablet_replica.cc:333] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.776024   431 raft_consensus.cc:2243] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:46.776414  9946 leader_election.cc:336] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:46.776582   431 raft_consensus.cc:2272] T c0c62d82ebe5420bb298d787cda221aa P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:46.778237  9948 leader_election.cc:336] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a1cbca39e34743d8a40816c3bc5f2030 (127.0.107.194:34259): Network error: Client connection negotiation failed: client connection to 127.0.107.194:34259: connect: Connection refused (error 111)
I20260709 13:05:46.778707  9948 leader_election.cc:304] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 9c8d63af7ee84c38a87435cbaf6deab4, a1cbca39e34743d8a40816c3bc5f2030
I20260709 13:05:46.779222   431 tablet_replica.cc:333] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.779522 10974 raft_consensus.cc:2749] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:46.780247   431 raft_consensus.cc:2243] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.781219   431 raft_consensus.cc:2272] T 9579648f7487465aa44ecf73e23731d8 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.783627   431 tablet_replica.cc:333] T adca2c30b75149c988cf71f9effb9d6c P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.784309   431 raft_consensus.cc:2243] T adca2c30b75149c988cf71f9effb9d6c P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.784710   431 raft_consensus.cc:2272] T adca2c30b75149c988cf71f9effb9d6c P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.786605   431 tablet_replica.cc:333] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.787209   431 raft_consensus.cc:2243] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.787967   431 raft_consensus.cc:2272] T c2d7bb9a0b174b168df9040b02c70ab9 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.789973   431 tablet_replica.cc:333] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.790717   431 raft_consensus.cc:2243] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.791278   431 raft_consensus.cc:2272] T 30b0a6c62c14440c9e613e73781c542e P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.793565   431 tablet_replica.cc:333] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.793836 10974 raft_consensus.cc:493] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9c8d63af7ee84c38a87435cbaf6deab4)
I20260709 13:05:46.794514   431 raft_consensus.cc:2243] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.794428 10974 raft_consensus.cc:515] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e10aab26c644389b8161974dcabc5d0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 45019 } } peers { permanent_uuid: "23cbd6234c5e43d89f22c235ed6bb96a" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 34321 } } peers { permanent_uuid: "9c8d63af7ee84c38a87435cbaf6deab4" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 41383 } }
I20260709 13:05:46.795194   431 raft_consensus.cc:2272] T 2b228f0642a242db93a7d11c1afab0ea P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.797065 10974 leader_election.cc:290] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019), 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383)
I20260709 13:05:46.798225   431 tablet_replica.cc:333] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.799223   431 raft_consensus.cc:2243] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.799800   431 raft_consensus.cc:2272] T ba93a39b54a742c6b391a2eb491f3d79 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:46.801936  9949 leader_election.cc:336] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Network error: Client connection negotiation failed: client connection to 127.0.107.195:45019: connect: Connection refused (error 111)
W20260709 13:05:46.802497  9946 leader_election.cc:336] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9c8d63af7ee84c38a87435cbaf6deab4 (127.0.107.193:41383): Network error: Client connection negotiation failed: client connection to 127.0.107.193:41383: connect: Connection refused (error 111)
I20260709 13:05:46.802891   431 tablet_replica.cc:333] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.803004  9946 leader_election.cc:304] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23cbd6234c5e43d89f22c235ed6bb96a; no voters: 6e10aab26c644389b8161974dcabc5d0, 9c8d63af7ee84c38a87435cbaf6deab4
I20260709 13:05:46.804003 10974 raft_consensus.cc:2749] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.804024   431 raft_consensus.cc:2243] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:46.804682  9949 consensus_peers.cc:597] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a -> Peer 6e10aab26c644389b8161974dcabc5d0 (127.0.107.195:45019): Couldn't send request to peer 6e10aab26c644389b8161974dcabc5d0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:45019: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:46.804783   431 raft_consensus.cc:2272] T e80fbd6291544644996b96fec0e8f2c3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.807099   431 tablet_replica.cc:333] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.807624   431 raft_consensus.cc:2243] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.807993   431 raft_consensus.cc:2272] T fbbee4fe35264c5aa3ea32adc034b9c7 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.810043   431 tablet_replica.cc:333] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.810760   431 raft_consensus.cc:2243] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.811542   431 raft_consensus.cc:2272] T 716c25a1e46547c999c1c7c860e23f3b P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.813911   431 tablet_replica.cc:333] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.814502   431 raft_consensus.cc:2243] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.814985   431 raft_consensus.cc:2272] T aec2f732f8024a00ae0b4d7bc00bb8e3 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.816812   431 tablet_replica.cc:333] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.817299   431 raft_consensus.cc:2243] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.817643   431 raft_consensus.cc:2272] T b189ef5046cd47ebb3d431e2ef24d6fb P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.819378   431 tablet_replica.cc:333] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.819859   431 raft_consensus.cc:2243] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.820236   431 raft_consensus.cc:2272] T b011ab28e64b483db8daaf9ed27f45ce P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.821995   431 tablet_replica.cc:333] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.822490   431 raft_consensus.cc:2243] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.823278   431 raft_consensus.cc:2272] T ea4f626ff2a94a99bb6505423eba3352 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.825124   431 tablet_replica.cc:333] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.825644   431 raft_consensus.cc:2243] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.826037   431 raft_consensus.cc:2272] T 0e044b992b684d52b54e3e8a4aa9720a P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.828028   431 tablet_replica.cc:333] T 23871dd822614a5da07c2289f80141c1 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.828567   431 raft_consensus.cc:2243] T 23871dd822614a5da07c2289f80141c1 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.828992   431 raft_consensus.cc:2272] T 23871dd822614a5da07c2289f80141c1 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.830770   431 tablet_replica.cc:333] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.831353   431 raft_consensus.cc:2243] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.832038   431 raft_consensus.cc:2272] T b6ff13f827e7411d9a5782fb06e93c9b P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.833937   431 tablet_replica.cc:333] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.834460   431 raft_consensus.cc:2243] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.834813   431 raft_consensus.cc:2272] T d8f99aa4491c4458bb5ebc03bc0cd813 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.836790   431 tablet_replica.cc:333] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.837296   431 raft_consensus.cc:2243] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.838843   431 raft_consensus.cc:2272] T 6a2d12d5566b4c82a5572035a797a82d P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.840794   431 tablet_replica.cc:333] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.841279   431 raft_consensus.cc:2243] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.841686   431 raft_consensus.cc:2272] T 148fd9eeaa2647bca8ad75116e461ac6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.843703   431 tablet_replica.cc:333] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.844238   431 raft_consensus.cc:2243] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.844624   431 raft_consensus.cc:2272] T af25fe6172fa4032a6570d9a3c7757c6 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.846462   431 tablet_replica.cc:333] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.846966   431 raft_consensus.cc:2243] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.847325   431 raft_consensus.cc:2272] T 755ff96760b04fd49a520c059f90b1fe P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.849205   431 tablet_replica.cc:333] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.849692   431 raft_consensus.cc:2243] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.850381   431 raft_consensus.cc:2272] T a8a61e076bba425385e5327d369522de P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.852242   431 tablet_replica.cc:333] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.852849   431 raft_consensus.cc:2243] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.853245   431 raft_consensus.cc:2272] T 27c539d1a0b54f8ca31dba142cb38599 P 23cbd6234c5e43d89f22c235ed6bb96a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.855082   431 tablet_replica.cc:333] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.855562   431 raft_consensus.cc:2243] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.855912   431 raft_consensus.cc:2272] T aa9f688643024d419097e12e80ced0d2 P 23cbd6234c5e43d89f22c235ed6bb96a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.857609   431 tablet_replica.cc:333] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.858064   431 raft_consensus.cc:2243] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.858888   431 raft_consensus.cc:2272] T 4cd54dca51434fcf8e5a473955555bfa P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.860652   431 tablet_replica.cc:333] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.861164   431 raft_consensus.cc:2243] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.861519   431 raft_consensus.cc:2272] T 10a930896e984848a0516f17a1213ab5 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.863251   431 tablet_replica.cc:333] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.863693   431 raft_consensus.cc:2243] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.864531   431 raft_consensus.cc:2272] T c2684c424702483b92758b1173702381 P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.866700   431 tablet_replica.cc:333] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.867209   431 raft_consensus.cc:2243] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.867915   431 raft_consensus.cc:2272] T a981502782314272a8a9bee8da734427 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.869735   431 tablet_replica.cc:333] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.870559   431 raft_consensus.cc:2243] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:46.872756   431 raft_consensus.cc:2272] T 4b0376b68ba34b8596a5ad96711a0c6e P 23cbd6234c5e43d89f22c235ed6bb96a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.874678   431 tablet_replica.cc:333] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a: stopping tablet replica
I20260709 13:05:46.875406   431 raft_consensus.cc:2243] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:46.875892   431 raft_consensus.cc:2272] T 778ba1e8c7b04438a6633db058c14508 P 23cbd6234c5e43d89f22c235ed6bb96a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:46.979656   431 tablet_server.cc:196] TabletServer@127.0.107.196:0 shutdown complete.
I20260709 13:05:47.047652   431 master.cc:562] Master@127.0.107.254:42979 shutting down...
I20260709 13:05:47.092559   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:47.093309   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:47.093735   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 242b0818bc8c46a6b7943f2e8320d4fd: stopping tablet replica
I20260709 13:05:47.119323   431 master.cc:584] Master@127.0.107.254:42979 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (37642 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:05:47.210444   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:39169
I20260709 13:05:47.211602   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:47.217945 11002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:47.217978 11003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:47.218887 11005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:47.219645   431 server_base.cc:1061] running on GCE node
I20260709 13:05:47.220866   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:47.221045   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:47.221266   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602347221228 us; error 0 us; skew 500 ppm
I20260709 13:05:47.221833   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:47.224712   431 webserver.cc:533] Webserver started at http://127.0.107.254:41905/ using document root <none> and password file <none>
I20260709 13:05:47.225289   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:47.225453   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:47.225790   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:47.227291   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/master-0-root/instance:
uuid: "d8dfab7811414a58a2bda1e4ab50887a"
format_stamp: "Formatted at 2026-07-09 13:05:47 on dist-test-slave-cq4t"
I20260709 13:05:47.231968   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:05:47.236827 11010 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:47.237653   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:05:47.237967   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/master-0-root
uuid: "d8dfab7811414a58a2bda1e4ab50887a"
format_stamp: "Formatted at 2026-07-09 13:05:47 on dist-test-slave-cq4t"
I20260709 13:05:47.238278   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:47.262308   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:47.264075   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:47.334854   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:39169
I20260709 13:05:47.335085 11062 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:39169 every 8 connection(s)
I20260709 13:05:47.340247 11063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.354775 11063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a: Bootstrap starting.
I20260709 13:05:47.360249 11063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:47.365244 11063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a: No bootstrap required, opened a new log
I20260709 13:05:47.368014 11063 raft_consensus.cc:359] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8dfab7811414a58a2bda1e4ab50887a" member_type: VOTER }
I20260709 13:05:47.368574 11063 raft_consensus.cc:385] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:47.368861 11063 raft_consensus.cc:740] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8dfab7811414a58a2bda1e4ab50887a, State: Initialized, Role: FOLLOWER
I20260709 13:05:47.369558 11063 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8dfab7811414a58a2bda1e4ab50887a" member_type: VOTER }
I20260709 13:05:47.370082 11063 raft_consensus.cc:399] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:05:47.370299 11063 raft_consensus.cc:493] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:05:47.370626 11063 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:47.375815 11063 raft_consensus.cc:515] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8dfab7811414a58a2bda1e4ab50887a" member_type: VOTER }
I20260709 13:05:47.376379 11063 leader_election.cc:304] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8dfab7811414a58a2bda1e4ab50887a; no voters: 
I20260709 13:05:47.377664 11063 leader_election.cc:290] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:05:47.378193 11066 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:47.379726 11066 raft_consensus.cc:697] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [term 1 LEADER]: Becoming Leader. State: Replica: d8dfab7811414a58a2bda1e4ab50887a, State: Running, Role: LEADER
I20260709 13:05:47.380538 11066 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8dfab7811414a58a2bda1e4ab50887a" member_type: VOTER }
I20260709 13:05:47.380847 11063 sys_catalog.cc:565] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:05:47.388511 11068 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8dfab7811414a58a2bda1e4ab50887a. Latest consensus state: current_term: 1 leader_uuid: "d8dfab7811414a58a2bda1e4ab50887a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8dfab7811414a58a2bda1e4ab50887a" member_type: VOTER } }
I20260709 13:05:47.388711 11067 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8dfab7811414a58a2bda1e4ab50887a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8dfab7811414a58a2bda1e4ab50887a" member_type: VOTER } }
I20260709 13:05:47.389389 11068 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:47.389755 11067 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [sys.catalog]: This master's current role is: LEADER
I20260709 13:05:47.391611 11076 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:05:47.397260 11076 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:05:47.398941   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:05:47.407073 11076 catalog_manager.cc:1383] Generated new cluster ID: 44f28eb8bd2d41fbadcd90489b1bf185
I20260709 13:05:47.407519 11076 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:05:47.423728 11076 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:05:47.425274 11076 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:05:47.436528 11076 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a: Generated new TSK 0
I20260709 13:05:47.437234 11076 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:05:47.466648   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:47.474967 11084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:47.476001 11085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:47.478184 11087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:47.478683   431 server_base.cc:1061] running on GCE node
I20260709 13:05:47.479851   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:47.480091   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:47.480285   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602347480264 us; error 0 us; skew 500 ppm
I20260709 13:05:47.480891   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:47.483824   431 webserver.cc:533] Webserver started at http://127.0.107.193:36023/ using document root <none> and password file <none>
I20260709 13:05:47.484376   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:47.484599   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:47.484925   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:47.486244   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-0-root/instance:
uuid: "353726f02e9c439792e790fb0958a80d"
format_stamp: "Formatted at 2026-07-09 13:05:47 on dist-test-slave-cq4t"
I20260709 13:05:47.491459   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:05:47.495555 11092 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:47.496465   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:05:47.496805   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-0-root
uuid: "353726f02e9c439792e790fb0958a80d"
format_stamp: "Formatted at 2026-07-09 13:05:47 on dist-test-slave-cq4t"
I20260709 13:05:47.497076   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:47.517691   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:47.519215   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:47.520982   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:47.524021   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:47.524255   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:47.524592   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:47.524786   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:47.575062   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:33527
I20260709 13:05:47.575176 11154 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:33527 every 8 connection(s)
I20260709 13:05:47.581358   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:47.589913 11159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:47.590667 11160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:47.594447 11162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:47.595453 11155 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39169
I20260709 13:05:47.595983 11155 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:47.596196   431 server_base.cc:1061] running on GCE node
I20260709 13:05:47.597070 11155 heartbeater.cc:507] Master 127.0.107.254:39169 requested a full tablet report, sending...
I20260709 13:05:47.597611   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:47.597894   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:47.598119   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602347598098 us; error 0 us; skew 500 ppm
I20260709 13:05:47.598860   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:47.599715 11028 ts_manager.cc:194] Registered new tserver with Master: 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527)
I20260709 13:05:47.601939   431 webserver.cc:533] Webserver started at http://127.0.107.194:41935/ using document root <none> and password file <none>
I20260709 13:05:47.602285 11028 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50544
I20260709 13:05:47.602648   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:47.603022   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:47.603346   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:47.604882   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-1-root/instance:
uuid: "bc100c804e924ea6b35a60fc507e1d6f"
format_stamp: "Formatted at 2026-07-09 13:05:47 on dist-test-slave-cq4t"
I20260709 13:05:47.610033   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:05:47.614166 11167 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:47.615214   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:05:47.615629   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-1-root
uuid: "bc100c804e924ea6b35a60fc507e1d6f"
format_stamp: "Formatted at 2026-07-09 13:05:47 on dist-test-slave-cq4t"
I20260709 13:05:47.616055   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:47.633872   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:47.635753   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:47.637444   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:47.640012   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:47.640283   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:47.640563   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:47.640753   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:47.688681   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:37675
I20260709 13:05:47.688803 11229 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:37675 every 8 connection(s)
I20260709 13:05:47.694545   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:47.704218 11234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:47.706313 11233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:47.708555 11236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:47.709293 11230 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39169
I20260709 13:05:47.709913 11230 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:47.709846   431 server_base.cc:1061] running on GCE node
I20260709 13:05:47.711117 11230 heartbeater.cc:507] Master 127.0.107.254:39169 requested a full tablet report, sending...
I20260709 13:05:47.711160   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:47.711536   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:47.711731   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602347711703 us; error 0 us; skew 500 ppm
I20260709 13:05:47.712553   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:47.713349 11028 ts_manager.cc:194] Registered new tserver with Master: bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:47.714820 11028 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50554
I20260709 13:05:47.715565   431 webserver.cc:533] Webserver started at http://127.0.107.195:35007/ using document root <none> and password file <none>
I20260709 13:05:47.716240   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:47.716440   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:47.716722   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:47.718400   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-2-root/instance:
uuid: "93009b2e2d7441f4a9d48b4d094b3711"
format_stamp: "Formatted at 2026-07-09 13:05:47 on dist-test-slave-cq4t"
I20260709 13:05:47.723595   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:05:47.727929 11241 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:47.729013   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:05:47.729540   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-2-root
uuid: "93009b2e2d7441f4a9d48b4d094b3711"
format_stamp: "Formatted at 2026-07-09 13:05:47 on dist-test-slave-cq4t"
I20260709 13:05:47.729866   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:47.767897   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:47.769275   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:47.770733   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:47.773306   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:47.773583   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:47.773872   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:47.774080   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:47.821472   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:40727
I20260709 13:05:47.821579 11303 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:40727 every 8 connection(s)
I20260709 13:05:47.837486 11304 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39169
I20260709 13:05:47.837852 11304 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:47.838642 11304 heartbeater.cc:507] Master 127.0.107.254:39169 requested a full tablet report, sending...
I20260709 13:05:47.840826 11028 ts_manager.cc:194] Registered new tserver with Master: 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:47.841912   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016530211s
I20260709 13:05:47.842509 11028 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50570
I20260709 13:05:47.872308 11028 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50576:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:05:47.875886 11028 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:05:47.987671 11195 tablet_service.cc:1511] Processing CreateTablet for tablet 970302250f4e4b4cb92307cc423adceb (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:05:47.989390 11195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 970302250f4e4b4cb92307cc423adceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.992225 11194 tablet_service.cc:1511] Processing CreateTablet for tablet ebae99d48338401f8a5c420bd5695aa5 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:05:47.993970 11194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebae99d48338401f8a5c420bd5695aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.994951 11190 tablet_service.cc:1511] Processing CreateTablet for tablet e28402092b5d42a6a0dde934ed6d0154 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:05:47.996731 11190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e28402092b5d42a6a0dde934ed6d0154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.995060 11193 tablet_service.cc:1511] Processing CreateTablet for tablet 35015db7e03c4af6a10a7a81b575dea9 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:05:47.997125 11191 tablet_service.cc:1511] Processing CreateTablet for tablet 5a6eb9aaa1494f99a0b3cb2a61935c2f (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:05:47.998142 11193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35015db7e03c4af6a10a7a81b575dea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.998539 11120 tablet_service.cc:1511] Processing CreateTablet for tablet 970302250f4e4b4cb92307cc423adceb (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:05:48.000124 11120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 970302250f4e4b4cb92307cc423adceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:47.999879 11192 tablet_service.cc:1511] Processing CreateTablet for tablet d8ea549dae19466b9e30bde1c7b4721d (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:05:48.001353 11192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8ea549dae19466b9e30bde1c7b4721d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.002108 11116 tablet_service.cc:1511] Processing CreateTablet for tablet 5a6eb9aaa1494f99a0b3cb2a61935c2f (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:05:48.003715 11116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a6eb9aaa1494f99a0b3cb2a61935c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.003501 11119 tablet_service.cc:1511] Processing CreateTablet for tablet ebae99d48338401f8a5c420bd5695aa5 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:05:48.005144 11119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebae99d48338401f8a5c420bd5695aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.007333 11191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a6eb9aaa1494f99a0b3cb2a61935c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.006259 11115 tablet_service.cc:1511] Processing CreateTablet for tablet e28402092b5d42a6a0dde934ed6d0154 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:05:48.008255 11118 tablet_service.cc:1511] Processing CreateTablet for tablet 35015db7e03c4af6a10a7a81b575dea9 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:05:48.009524 11115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e28402092b5d42a6a0dde934ed6d0154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.010833 11117 tablet_service.cc:1511] Processing CreateTablet for tablet d8ea549dae19466b9e30bde1c7b4721d (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:05:48.012401 11117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8ea549dae19466b9e30bde1c7b4721d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.008265 11114 tablet_service.cc:1511] Processing CreateTablet for tablet 5323137cf0c94b33bbde48d8b7ce1e01 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:05:48.014393 11114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5323137cf0c94b33bbde48d8b7ce1e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.014775 11188 tablet_service.cc:1511] Processing CreateTablet for tablet bbf50a2fc30148e3b1b5b7e732cb4c1e (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:05:48.016376 11189 tablet_service.cc:1511] Processing CreateTablet for tablet 5323137cf0c94b33bbde48d8b7ce1e01 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:05:48.018426 11187 tablet_service.cc:1511] Processing CreateTablet for tablet d56f50814ef548078489152c9af6ee90 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:05:48.019778 11118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35015db7e03c4af6a10a7a81b575dea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.031891 11265 tablet_service.cc:1511] Processing CreateTablet for tablet 5a6eb9aaa1494f99a0b3cb2a61935c2f (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:05:48.034022 11265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a6eb9aaa1494f99a0b3cb2a61935c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.019429 11269 tablet_service.cc:1511] Processing CreateTablet for tablet 970302250f4e4b4cb92307cc423adceb (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:05:48.036115 11269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 970302250f4e4b4cb92307cc423adceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.016438 11188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf50a2fc30148e3b1b5b7e732cb4c1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.021368 11186 tablet_service.cc:1511] Processing CreateTablet for tablet a81596d1743546d89f983c09be2c83ea (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:05:48.039414 11186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a81596d1743546d89f983c09be2c83ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.040503 11187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d56f50814ef548078489152c9af6ee90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.024453 11268 tablet_service.cc:1511] Processing CreateTablet for tablet ebae99d48338401f8a5c420bd5695aa5 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:05:48.042114 11268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebae99d48338401f8a5c420bd5695aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.025349 11267 tablet_service.cc:1511] Processing CreateTablet for tablet 35015db7e03c4af6a10a7a81b575dea9 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:05:48.043431 11189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5323137cf0c94b33bbde48d8b7ce1e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.044097 11267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35015db7e03c4af6a10a7a81b575dea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.018543 11113 tablet_service.cc:1511] Processing CreateTablet for tablet bbf50a2fc30148e3b1b5b7e732cb4c1e (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:05:48.046252 11113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf50a2fc30148e3b1b5b7e732cb4c1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.029707 11266 tablet_service.cc:1511] Processing CreateTablet for tablet d8ea549dae19466b9e30bde1c7b4721d (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:05:48.048138 11266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8ea549dae19466b9e30bde1c7b4721d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.048388 11111 tablet_service.cc:1511] Processing CreateTablet for tablet a81596d1743546d89f983c09be2c83ea (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:05:48.030434 11264 tablet_service.cc:1511] Processing CreateTablet for tablet e28402092b5d42a6a0dde934ed6d0154 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:05:48.050235 11111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a81596d1743546d89f983c09be2c83ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.050474 11264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e28402092b5d42a6a0dde934ed6d0154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.022886 11112 tablet_service.cc:1511] Processing CreateTablet for tablet d56f50814ef548078489152c9af6ee90 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:05:48.052850 11112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d56f50814ef548078489152c9af6ee90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.084461 11319 tablet_bootstrap.cc:492] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:48.094741 11319 tablet_bootstrap.cc:654] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.096722 11263 tablet_service.cc:1511] Processing CreateTablet for tablet 5323137cf0c94b33bbde48d8b7ce1e01 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:05:48.097105 11262 tablet_service.cc:1511] Processing CreateTablet for tablet bbf50a2fc30148e3b1b5b7e732cb4c1e (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:05:48.100956 11263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5323137cf0c94b33bbde48d8b7ce1e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.101457 11261 tablet_service.cc:1511] Processing CreateTablet for tablet d56f50814ef548078489152c9af6ee90 (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:05:48.102475 11262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf50a2fc30148e3b1b5b7e732cb4c1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.100521 11260 tablet_service.cc:1511] Processing CreateTablet for tablet a81596d1743546d89f983c09be2c83ea (DEFAULT_TABLE table=test-workload [id=9bac311ea2c84af8945926e7d4529be0]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:05:48.115442 11261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d56f50814ef548078489152c9af6ee90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.119577 11260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a81596d1743546d89f983c09be2c83ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:48.198000 11319 tablet_bootstrap.cc:492] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d: No bootstrap required, opened a new log
I20260709 13:05:48.198590 11319 ts_tablet_manager.cc:1403] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.115s	user 0.062s	sys 0.044s
I20260709 13:05:48.201998 11319 raft_consensus.cc:359] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:48.202770 11319 raft_consensus.cc:385] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.203118 11319 raft_consensus.cc:740] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.204063 11319 consensus_queue.cc:260] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:48.274274 11321 tablet_bootstrap.cc:492] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f: Bootstrap starting.
I20260709 13:05:48.279421 11319 ts_tablet_manager.cc:1434] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.021s
I20260709 13:05:48.280332 11155 heartbeater.cc:499] Master 127.0.107.254:39169 was elected leader, sending a full tablet report...
I20260709 13:05:48.280646 11319 tablet_bootstrap.cc:492] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:48.290239 11319 tablet_bootstrap.cc:654] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.274282 11322 tablet_bootstrap.cc:492] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711: Bootstrap starting.
I20260709 13:05:48.305018 11322 tablet_bootstrap.cc:654] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.308635 11321 tablet_bootstrap.cc:654] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.320851 11321 tablet_bootstrap.cc:492] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f: No bootstrap required, opened a new log
I20260709 13:05:48.321449 11321 ts_tablet_manager.cc:1403] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f: Time spent bootstrapping tablet: real 0.048s	user 0.019s	sys 0.000s
I20260709 13:05:48.324522 11321 raft_consensus.cc:359] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.325477 11321 raft_consensus.cc:385] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.325830 11321 raft_consensus.cc:740] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.326788 11321 consensus_queue.cc:260] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
W20260709 13:05:48.330469 11156 tablet.cc:2406] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:05:48.346426 11230 heartbeater.cc:499] Master 127.0.107.254:39169 was elected leader, sending a full tablet report...
I20260709 13:05:48.346730 11322 tablet_bootstrap.cc:492] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711: No bootstrap required, opened a new log
I20260709 13:05:48.346995 11321 ts_tablet_manager.cc:1434] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f: Time spent starting tablet: real 0.025s	user 0.002s	sys 0.020s
I20260709 13:05:48.347313 11319 tablet_bootstrap.cc:492] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d: No bootstrap required, opened a new log
I20260709 13:05:48.347496 11322 ts_tablet_manager.cc:1403] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711: Time spent bootstrapping tablet: real 0.074s	user 0.024s	sys 0.023s
I20260709 13:05:48.347922 11319 ts_tablet_manager.cc:1403] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.068s	user 0.020s	sys 0.041s
I20260709 13:05:48.348085 11321 tablet_bootstrap.cc:492] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f: Bootstrap starting.
I20260709 13:05:48.351634 11319 raft_consensus.cc:359] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.351884 11322 raft_consensus.cc:359] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.352445 11319 raft_consensus.cc:385] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.352628 11322 raft_consensus.cc:385] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.352816 11319 raft_consensus.cc:740] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.352985 11322 raft_consensus.cc:740] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.353690 11319 consensus_queue.cc:260] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.353781 11322 consensus_queue.cc:260] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.356117 11319 ts_tablet_manager.cc:1434] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:48.357177 11319 tablet_bootstrap.cc:492] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:48.357723 11321 tablet_bootstrap.cc:654] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.366532 11319 tablet_bootstrap.cc:654] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.378795 11323 raft_consensus.cc:493] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.379549 11323 raft_consensus.cc:515] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.380270 11319 tablet_bootstrap.cc:492] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d: No bootstrap required, opened a new log
I20260709 13:05:48.381108 11319 ts_tablet_manager.cc:1403] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.007s
I20260709 13:05:48.385644 11319 raft_consensus.cc:359] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.386710 11319 raft_consensus.cc:385] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.387230 11319 raft_consensus.cc:740] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.388393 11323 leader_election.cc:290] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:48.388875 11319 consensus_queue.cc:260] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
W20260709 13:05:48.393471 11075 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:48.395265 11304 heartbeater.cc:499] Master 127.0.107.254:39169 was elected leader, sending a full tablet report...
I20260709 13:05:48.399149 11322 ts_tablet_manager.cc:1434] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711: Time spent starting tablet: real 0.050s	user 0.019s	sys 0.020s
I20260709 13:05:48.400269 11322 tablet_bootstrap.cc:492] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711: Bootstrap starting.
I20260709 13:05:48.400671 11319 ts_tablet_manager.cc:1434] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.002s
I20260709 13:05:48.401800 11319 tablet_bootstrap.cc:492] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:48.407248 11322 tablet_bootstrap.cc:654] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.410518 11321 tablet_bootstrap.cc:492] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f: No bootstrap required, opened a new log
I20260709 13:05:48.411413 11321 ts_tablet_manager.cc:1403] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f: Time spent bootstrapping tablet: real 0.064s	user 0.033s	sys 0.026s
I20260709 13:05:48.416832 11321 raft_consensus.cc:359] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.417650 11321 raft_consensus.cc:385] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.417963 11321 raft_consensus.cc:740] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.420845 11319 tablet_bootstrap.cc:654] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.421345 11322 tablet_bootstrap.cc:492] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711: No bootstrap required, opened a new log
I20260709 13:05:48.421947 11322 ts_tablet_manager.cc:1403] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.007s
I20260709 13:05:48.418751 11321 consensus_queue.cc:260] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.425186 11322 raft_consensus.cc:359] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.427909 11322 raft_consensus.cc:385] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.432296 11321 ts_tablet_manager.cc:1434] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:05:48.428427 11322 raft_consensus.cc:740] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.436033 11322 consensus_queue.cc:260] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.438493 11319 tablet_bootstrap.cc:492] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d: No bootstrap required, opened a new log
I20260709 13:05:48.439170 11319 ts_tablet_manager.cc:1403] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.003s
I20260709 13:05:48.440361 11322 ts_tablet_manager.cc:1434] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 13:05:48.441560 11322 tablet_bootstrap.cc:492] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711: Bootstrap starting.
I20260709 13:05:48.442203 11205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f" is_pre_election: true
I20260709 13:05:48.442488 11319 raft_consensus.cc:359] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.443470 11319 raft_consensus.cc:385] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.443780 11319 raft_consensus.cc:740] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.444662 11319 consensus_queue.cc:260] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
W20260709 13:05:48.444476 11094 leader_election.cc:343] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.449357 11322 tablet_bootstrap.cc:654] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.449667 11321 tablet_bootstrap.cc:492] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f: Bootstrap starting.
I20260709 13:05:48.450435 11319 ts_tablet_manager.cc:1434] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.005s
I20260709 13:05:48.451591 11319 tablet_bootstrap.cc:492] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:48.457840 11321 tablet_bootstrap.cc:654] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.468467 11319 tablet_bootstrap.cc:654] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.468516 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711" is_pre_election: true
I20260709 13:05:48.469918 11279 raft_consensus.cc:2468] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 353726f02e9c439792e790fb0958a80d in term 0.
I20260709 13:05:48.471698 11094 leader_election.cc:304] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:48.473174 11323 raft_consensus.cc:2804] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.473596 11323 raft_consensus.cc:493] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.473958 11323 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.475411 11322 tablet_bootstrap.cc:492] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711: No bootstrap required, opened a new log
I20260709 13:05:48.475546 11319 tablet_bootstrap.cc:492] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d: No bootstrap required, opened a new log
I20260709 13:05:48.476096 11322 ts_tablet_manager.cc:1403] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.018s
I20260709 13:05:48.476292 11319 ts_tablet_manager.cc:1403] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.000s
I20260709 13:05:48.479064 11319 raft_consensus.cc:359] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.479460 11322 raft_consensus.cc:359] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.480451 11319 raft_consensus.cc:385] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.480584 11322 raft_consensus.cc:385] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.480886 11319 raft_consensus.cc:740] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.481014 11322 raft_consensus.cc:740] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.481611 11321 tablet_bootstrap.cc:492] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f: No bootstrap required, opened a new log
I20260709 13:05:48.482025 11319 consensus_queue.cc:260] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.482259 11321 ts_tablet_manager.cc:1403] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.012s
I20260709 13:05:48.482241 11322 consensus_queue.cc:260] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.483377 11323 raft_consensus.cc:515] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.485100 11322 ts_tablet_manager.cc:1434] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.004s
I20260709 13:05:48.486224 11322 tablet_bootstrap.cc:492] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711: Bootstrap starting.
I20260709 13:05:48.486495 11205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:48.486557 11321 raft_consensus.cc:359] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.487643 11321 raft_consensus.cc:385] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.488022 11321 raft_consensus.cc:740] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Initialized, Role: FOLLOWER
W20260709 13:05:48.488181 11094 leader_election.cc:343] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.488958 11321 consensus_queue.cc:260] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.490839 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:48.491609 11279 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.492231 11321 ts_tablet_manager.cc:1434] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:05:48.493589 11321 tablet_bootstrap.cc:492] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f: Bootstrap starting.
I20260709 13:05:48.493983 11322 tablet_bootstrap.cc:654] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.500619 11321 tablet_bootstrap.cc:654] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.503093 11323 leader_election.cc:290] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 election: Requested vote from peers bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:48.513054 11279 raft_consensus.cc:2468] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 353726f02e9c439792e790fb0958a80d in term 1.
I20260709 13:05:48.515071 11094 leader_election.cc:304] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:48.516816 11323 raft_consensus.cc:2804] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.517509 11323 raft_consensus.cc:697] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 1 LEADER]: Becoming Leader. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Running, Role: LEADER
I20260709 13:05:48.520464 11323 consensus_queue.cc:237] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.526881 11319 ts_tablet_manager.cc:1434] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.050s	user 0.009s	sys 0.006s
I20260709 13:05:48.528631 11319 tablet_bootstrap.cc:492] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:48.530448 11322 tablet_bootstrap.cc:492] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711: No bootstrap required, opened a new log
I20260709 13:05:48.531139 11322 ts_tablet_manager.cc:1403] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711: Time spent bootstrapping tablet: real 0.045s	user 0.025s	sys 0.019s
I20260709 13:05:48.531499 11321 tablet_bootstrap.cc:492] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f: No bootstrap required, opened a new log
I20260709 13:05:48.532244 11321 ts_tablet_manager.cc:1403] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f: Time spent bootstrapping tablet: real 0.039s	user 0.026s	sys 0.007s
I20260709 13:05:48.534301 11322 raft_consensus.cc:359] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:48.535136 11322 raft_consensus.cc:385] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.535461 11322 raft_consensus.cc:740] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.535627 11319 tablet_bootstrap.cc:654] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.535693 11321 raft_consensus.cc:359] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.536770 11321 raft_consensus.cc:385] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.536383 11322 consensus_queue.cc:260] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:48.537243 11321 raft_consensus.cc:740] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.537672 11026 catalog_manager.cc:5697] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d reported cstate change: term changed from 0 to 1, leader changed from <none> to 353726f02e9c439792e790fb0958a80d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "353726f02e9c439792e790fb0958a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:48.538303 11321 consensus_queue.cc:260] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.542057 11322 ts_tablet_manager.cc:1434] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:05:48.543403 11322 tablet_bootstrap.cc:492] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711: Bootstrap starting.
I20260709 13:05:48.548420 11321 ts_tablet_manager.cc:1434] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:05:48.554505 11322 tablet_bootstrap.cc:654] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.558399 11321 tablet_bootstrap.cc:492] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f: Bootstrap starting.
I20260709 13:05:48.567153 11322 tablet_bootstrap.cc:492] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711: No bootstrap required, opened a new log
I20260709 13:05:48.568078 11322 ts_tablet_manager.cc:1403] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.007s
I20260709 13:05:48.571190 11319 tablet_bootstrap.cc:492] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d: No bootstrap required, opened a new log
I20260709 13:05:48.571003 11322 raft_consensus.cc:359] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.571429 11321 tablet_bootstrap.cc:654] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.571738 11322 raft_consensus.cc:385] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.572070 11322 raft_consensus.cc:740] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.572820 11322 consensus_queue.cc:260] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.573621 11323 raft_consensus.cc:493] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.571887 11319 ts_tablet_manager.cc:1403] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.011s
I20260709 13:05:48.574136 11323 raft_consensus.cc:515] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.575342 11322 ts_tablet_manager.cc:1434] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:05:48.576419 11322 tablet_bootstrap.cc:492] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711: Bootstrap starting.
I20260709 13:05:48.576407 11323 leader_election.cc:290] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:48.579420 11205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebae99d48338401f8a5c420bd5695aa5" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f" is_pre_election: true
I20260709 13:05:48.580183 11205 raft_consensus.cc:2468] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 353726f02e9c439792e790fb0958a80d in term 0.
I20260709 13:05:48.581336 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebae99d48338401f8a5c420bd5695aa5" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711" is_pre_election: true
I20260709 13:05:48.581442 11319 raft_consensus.cc:359] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.582098 11279 raft_consensus.cc:2468] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 353726f02e9c439792e790fb0958a80d in term 0.
I20260709 13:05:48.582412 11319 raft_consensus.cc:385] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.582852 11319 raft_consensus.cc:740] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.584375 11094 leader_election.cc:304] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: 
I20260709 13:05:48.584976 11322 tablet_bootstrap.cc:654] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.586169 11323 raft_consensus.cc:2804] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.586707 11323 raft_consensus.cc:493] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.587159 11323 raft_consensus.cc:3060] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.595748 11319 consensus_queue.cc:260] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.599643 11323 raft_consensus.cc:515] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.602415 11321 tablet_bootstrap.cc:492] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f: No bootstrap required, opened a new log
I20260709 13:05:48.603077 11321 ts_tablet_manager.cc:1403] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f: Time spent bootstrapping tablet: real 0.045s	user 0.018s	sys 0.006s
I20260709 13:05:48.604395 11205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebae99d48338401f8a5c420bd5695aa5" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:48.605298 11205 raft_consensus.cc:3060] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.605851 11321 raft_consensus.cc:359] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:48.606410 11321 raft_consensus.cc:385] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.606602 11321 raft_consensus.cc:740] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.607488 11321 consensus_queue.cc:260] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:48.608474 11319 ts_tablet_manager.cc:1434] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:05:48.612160 11319 tablet_bootstrap.cc:492] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:48.617607 11321 ts_tablet_manager.cc:1434] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:05:48.619835 11319 tablet_bootstrap.cc:654] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.622007 11323 leader_election.cc:290] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 election: Requested vote from peers bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:48.623538 11321 tablet_bootstrap.cc:492] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f: Bootstrap starting.
I20260709 13:05:48.623629 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebae99d48338401f8a5c420bd5695aa5" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:48.624447 11279 raft_consensus.cc:3060] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.628108 11323 raft_consensus.cc:493] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.629123 11205 raft_consensus.cc:2468] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 353726f02e9c439792e790fb0958a80d in term 1.
I20260709 13:05:48.628862 11323 raft_consensus.cc:515] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:48.630652 11094 leader_election.cc:304] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:48.631743 11338 raft_consensus.cc:2804] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.632511 11279 raft_consensus.cc:2468] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 353726f02e9c439792e790fb0958a80d in term 1.
I20260709 13:05:48.626107 11322 tablet_bootstrap.cc:492] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711: No bootstrap required, opened a new log
I20260709 13:05:48.634101 11322 ts_tablet_manager.cc:1403] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711: Time spent bootstrapping tablet: real 0.058s	user 0.024s	sys 0.021s
I20260709 13:05:48.638718 11321 tablet_bootstrap.cc:654] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.638273 11322 raft_consensus.cc:359] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.639339 11322 raft_consensus.cc:385] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.639727 11322 raft_consensus.cc:740] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.640900 11322 consensus_queue.cc:260] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.643417 11322 ts_tablet_manager.cc:1434] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:05:48.644202 11327 raft_consensus.cc:493] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.644538 11322 tablet_bootstrap.cc:492] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711: Bootstrap starting.
I20260709 13:05:48.644923 11327 raft_consensus.cc:515] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:48.645514 11328 raft_consensus.cc:493] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.646036 11328 raft_consensus.cc:515] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.649528 11327 leader_election.cc:290] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727), 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527)
I20260709 13:05:48.651495 11205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970302250f4e4b4cb92307cc423adceb" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f" is_pre_election: true
I20260709 13:05:48.651747 11328 leader_election.cc:290] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:48.652246 11205 raft_consensus.cc:2468] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 353726f02e9c439792e790fb0958a80d in term 0.
I20260709 13:05:48.652534 11322 tablet_bootstrap.cc:654] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.653473 11094 leader_election.cc:304] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:48.657613 11323 leader_election.cc:290] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:48.658449 11323 raft_consensus.cc:2804] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.658895 11323 raft_consensus.cc:493] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.659339 11323 raft_consensus.cc:3060] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.641763 11338 raft_consensus.cc:697] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 1 LEADER]: Becoming Leader. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Running, Role: LEADER
I20260709 13:05:48.672672 11338 consensus_queue.cc:237] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.675787 11323 raft_consensus.cc:515] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:48.678690 11323 leader_election.cc:290] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 election: Requested vote from peers bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:48.685413 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970302250f4e4b4cb92307cc423adceb" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711" is_pre_election: true
I20260709 13:05:48.686509 11279 raft_consensus.cc:2468] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 353726f02e9c439792e790fb0958a80d in term 0.
W20260709 13:05:48.701560 11094 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:37675, user_credentials={real_user=slave}} blocked reactor thread for 48161.2us
I20260709 13:05:48.703286 11205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970302250f4e4b4cb92307cc423adceb" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:48.703346 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970302250f4e4b4cb92307cc423adceb" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711" is_pre_election: true
I20260709 13:05:48.703886 11205 raft_consensus.cc:3060] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.704396 11279 raft_consensus.cc:2468] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 0.
I20260709 13:05:48.706053 11169 leader_election.cc:304] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:48.710246 11205 raft_consensus.cc:2468] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 353726f02e9c439792e790fb0958a80d in term 1.
I20260709 13:05:48.710947 11327 raft_consensus.cc:2764] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:05:48.715927 11321 tablet_bootstrap.cc:492] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f: No bootstrap required, opened a new log
I20260709 13:05:48.716316 11094 leader_election.cc:304] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:48.716632 11321 ts_tablet_manager.cc:1403] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f: Time spent bootstrapping tablet: real 0.094s	user 0.013s	sys 0.003s
I20260709 13:05:48.718591 11338 raft_consensus.cc:2804] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.719156 11338 raft_consensus.cc:697] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 1 LEADER]: Becoming Leader. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Running, Role: LEADER
I20260709 13:05:48.720034 11338 consensus_queue.cc:237] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:48.720520 11321 raft_consensus.cc:359] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.721438 11321 raft_consensus.cc:385] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.721184 11026 catalog_manager.cc:5697] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d reported cstate change: term changed from 0 to 1, leader changed from <none> to 353726f02e9c439792e790fb0958a80d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "353726f02e9c439792e790fb0958a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:48.722158 11321 raft_consensus.cc:740] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.726958 11321 consensus_queue.cc:260] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.728827 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970302250f4e4b4cb92307cc423adceb" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:48.729727 11279 raft_consensus.cc:3060] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.731184 11321 ts_tablet_manager.cc:1434] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.004s
I20260709 13:05:48.732502 11321 tablet_bootstrap.cc:492] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f: Bootstrap starting.
I20260709 13:05:48.738646 11319 tablet_bootstrap.cc:492] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d: No bootstrap required, opened a new log
I20260709 13:05:48.739323 11319 ts_tablet_manager.cc:1403] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.127s	user 0.018s	sys 0.020s
I20260709 13:05:48.739842 11321 tablet_bootstrap.cc:654] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.739830 11279 raft_consensus.cc:2468] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 353726f02e9c439792e790fb0958a80d in term 1.
I20260709 13:05:48.742523 11319 raft_consensus.cc:359] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.744240 11205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5323137cf0c94b33bbde48d8b7ce1e01" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f" is_pre_election: true
W20260709 13:05:48.746876 11243 leader_election.cc:343] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.746781 11322 tablet_bootstrap.cc:492] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711: No bootstrap required, opened a new log
I20260709 13:05:48.748185 11322 ts_tablet_manager.cc:1403] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711: Time spent bootstrapping tablet: real 0.104s	user 0.053s	sys 0.032s
I20260709 13:05:48.749155 11319 raft_consensus.cc:385] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.749820 11319 raft_consensus.cc:740] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.750756 11319 consensus_queue.cc:260] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.752298 11322 raft_consensus.cc:359] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.753192 11322 raft_consensus.cc:385] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.753520 11322 raft_consensus.cc:740] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.754293 11322 consensus_queue.cc:260] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.754676 11350 raft_consensus.cc:493] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.755352 11350 raft_consensus.cc:515] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.756114 11026 catalog_manager.cc:5697] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d reported cstate change: term changed from 0 to 1, leader changed from <none> to 353726f02e9c439792e790fb0958a80d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "353726f02e9c439792e790fb0958a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:48.757725 11350 leader_election.cc:290] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:48.759617 11322 ts_tablet_manager.cc:1434] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:05:48.759968 11319 ts_tablet_manager.cc:1434] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 13:05:48.760977 11322 tablet_bootstrap.cc:492] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711: Bootstrap starting.
I20260709 13:05:48.761047 11205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6eb9aaa1494f99a0b3cb2a61935c2f" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f" is_pre_election: true
I20260709 13:05:48.761682 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6eb9aaa1494f99a0b3cb2a61935c2f" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711" is_pre_election: true
I20260709 13:05:48.762483 11279 raft_consensus.cc:2468] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 353726f02e9c439792e790fb0958a80d in term 0.
I20260709 13:05:48.761873 11205 raft_consensus.cc:2468] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 353726f02e9c439792e790fb0958a80d in term 0.
I20260709 13:05:48.763480 11321 tablet_bootstrap.cc:492] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f: No bootstrap required, opened a new log
I20260709 13:05:48.764003 11321 ts_tablet_manager.cc:1403] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.002s
I20260709 13:05:48.764595 11094 leader_election.cc:304] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:48.773504 11338 raft_consensus.cc:2804] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.774020 11338 raft_consensus.cc:493] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.776877 11338 raft_consensus.cc:3060] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.777742 11130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5323137cf0c94b33bbde48d8b7ce1e01" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d" is_pre_election: true
I20260709 13:05:48.768538 11322 tablet_bootstrap.cc:654] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.778599 11130 raft_consensus.cc:2468] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 0.
I20260709 13:05:48.780490 11242 leader_election.cc:304] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:48.777591 11321 raft_consensus.cc:359] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.761080 11319 tablet_bootstrap.cc:492] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:48.781734 11130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970302250f4e4b4cb92307cc423adceb" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d" is_pre_election: true
I20260709 13:05:48.782059 11328 raft_consensus.cc:2804] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.782603 11328 raft_consensus.cc:493] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.782997 11328 raft_consensus.cc:3060] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.783039 11321 raft_consensus.cc:385] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.783545 11321 raft_consensus.cc:740] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.784480 11321 consensus_queue.cc:260] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.786373 11338 raft_consensus.cc:515] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.787079 11321 ts_tablet_manager.cc:1434] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:05:48.788254 11321 tablet_bootstrap.cc:492] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f: Bootstrap starting.
I20260709 13:05:48.789151 11319 tablet_bootstrap.cc:654] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.789681 11205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6eb9aaa1494f99a0b3cb2a61935c2f" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:48.790323 11205 raft_consensus.cc:3060] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.796839 11328 raft_consensus.cc:515] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.799520 11205 raft_consensus.cc:2468] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 353726f02e9c439792e790fb0958a80d in term 1.
I20260709 13:05:48.801960 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6eb9aaa1494f99a0b3cb2a61935c2f" candidate_uuid: "353726f02e9c439792e790fb0958a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:48.802752 11279 raft_consensus.cc:3060] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.804208 11321 tablet_bootstrap.cc:654] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.811132 11279 raft_consensus.cc:2468] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 353726f02e9c439792e790fb0958a80d in term 1.
I20260709 13:05:48.811460 11328 leader_election.cc:290] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:48.812935 11130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5323137cf0c94b33bbde48d8b7ce1e01" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:48.813155 11204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5323137cf0c94b33bbde48d8b7ce1e01" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:48.816572 11130 raft_consensus.cc:3060] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.814340 11094 leader_election.cc:304] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:48.818413 11350 raft_consensus.cc:2804] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.819152 11350 raft_consensus.cc:697] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 1 LEADER]: Becoming Leader. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Running, Role: LEADER
I20260709 13:05:48.820271 11350 consensus_queue.cc:237] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
W20260709 13:05:48.824785 11243 leader_election.cc:343] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.814970 11338 leader_election.cc:290] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [CANDIDATE]: Term 1 election: Requested vote from peers bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:48.831225 11322 tablet_bootstrap.cc:492] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711: No bootstrap required, opened a new log
I20260709 13:05:48.831941 11322 ts_tablet_manager.cc:1403] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711: Time spent bootstrapping tablet: real 0.071s	user 0.013s	sys 0.016s
I20260709 13:05:48.834407 11319 tablet_bootstrap.cc:492] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d: No bootstrap required, opened a new log
I20260709 13:05:48.834975 11319 ts_tablet_manager.cc:1403] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.074s	user 0.031s	sys 0.019s
I20260709 13:05:48.834841 11322 raft_consensus.cc:359] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.839094 11322 raft_consensus.cc:385] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.839468 11322 raft_consensus.cc:740] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.840520 11322 consensus_queue.cc:260] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.841768 11319 raft_consensus.cc:359] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.842592 11319 raft_consensus.cc:385] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.842870 11319 raft_consensus.cc:740] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.842892 11322 ts_tablet_manager.cc:1434] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:05:48.843772 11319 consensus_queue.cc:260] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.845683 11321 tablet_bootstrap.cc:492] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f: No bootstrap required, opened a new log
I20260709 13:05:48.845259 11322 tablet_bootstrap.cc:492] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711: Bootstrap starting.
I20260709 13:05:48.846412 11321 ts_tablet_manager.cc:1403] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f: Time spent bootstrapping tablet: real 0.058s	user 0.020s	sys 0.018s
I20260709 13:05:48.848259 11328 raft_consensus.cc:493] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.848834 11328 raft_consensus.cc:515] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.849408 11319 ts_tablet_manager.cc:1434] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 13:05:48.850562 11319 tablet_bootstrap.cc:492] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:48.836537 11130 raft_consensus.cc:2468] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 1.
I20260709 13:05:48.852442 11129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8ea549dae19466b9e30bde1c7b4721d" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d" is_pre_election: true
I20260709 13:05:48.845546 11027 catalog_manager.cc:5697] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d reported cstate change: term changed from 0 to 1, leader changed from <none> to 353726f02e9c439792e790fb0958a80d (127.0.107.193). New cstate: current_term: 1 leader_uuid: "353726f02e9c439792e790fb0958a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:48.853569 11242 leader_election.cc:304] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:48.855250 11328 leader_election.cc:290] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:48.856488 11204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8ea549dae19466b9e30bde1c7b4721d" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f" is_pre_election: true
I20260709 13:05:48.858037 11328 raft_consensus.cc:2804] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.858670 11328 raft_consensus.cc:697] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 LEADER]: Becoming Leader. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
W20260709 13:05:48.860047 11243 leader_election.cc:343] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.859802 11328 consensus_queue.cc:237] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.861853 11321 raft_consensus.cc:359] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.862694 11321 raft_consensus.cc:385] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.863029 11321 raft_consensus.cc:740] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.863919 11321 consensus_queue.cc:260] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.852510 11322 tablet_bootstrap.cc:654] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.868044 11327 raft_consensus.cc:493] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.868748 11327 raft_consensus.cc:515] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.857053 11319 tablet_bootstrap.cc:654] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:48.875748 11242 leader_election.cc:343] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.876334 11242 leader_election.cc:304] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711; no voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:48.877441 11328 raft_consensus.cc:2749] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:48.882985 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf50a2fc30148e3b1b5b7e732cb4c1e" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711" is_pre_election: true
I20260709 13:05:48.883831 11279 raft_consensus.cc:2468] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 0.
I20260709 13:05:48.885522 11169 leader_election.cc:304] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:48.885149 11027 catalog_manager.cc:5697] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:48.886406 11319 tablet_bootstrap.cc:492] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d: No bootstrap required, opened a new log
I20260709 13:05:48.886974 11319 ts_tablet_manager.cc:1403] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.001s
I20260709 13:05:48.887348 11130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf50a2fc30148e3b1b5b7e732cb4c1e" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d" is_pre_election: true
I20260709 13:05:48.887064 11327 leader_election.cc:290] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:48.888219 11130 raft_consensus.cc:2468] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 0.
I20260709 13:05:48.888408 11327 raft_consensus.cc:2804] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.889106 11327 raft_consensus.cc:493] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.889587 11327 raft_consensus.cc:3060] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.890074 11319 raft_consensus.cc:359] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.890878 11319 raft_consensus.cc:385] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.891199 11319 raft_consensus.cc:740] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.900607 11321 ts_tablet_manager.cc:1434] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f: Time spent starting tablet: real 0.054s	user 0.009s	sys 0.000s
I20260709 13:05:48.900643 11319 consensus_queue.cc:260] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.902221 11321 tablet_bootstrap.cc:492] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f: Bootstrap starting.
I20260709 13:05:48.907682 11327 raft_consensus.cc:515] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.909132 11372 raft_consensus.cc:493] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.908291 11319 ts_tablet_manager.cc:1434] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:05:48.909776 11372 raft_consensus.cc:515] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.912505 11372 leader_election.cc:290] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:48.912563 11350 consensus_queue.cc:1048] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:48.914220 11129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56f50814ef548078489152c9af6ee90" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d" is_pre_election: true
I20260709 13:05:48.915047 11129 raft_consensus.cc:2468] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 0.
I20260709 13:05:48.915956 11322 tablet_bootstrap.cc:492] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711: No bootstrap required, opened a new log
I20260709 13:05:48.916237 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56f50814ef548078489152c9af6ee90" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711" is_pre_election: true
I20260709 13:05:48.916555 11322 ts_tablet_manager.cc:1403] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711: Time spent bootstrapping tablet: real 0.072s	user 0.012s	sys 0.012s
I20260709 13:05:48.916857 11168 leader_election.cc:304] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:48.912349 11130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf50a2fc30148e3b1b5b7e732cb4c1e" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d"
W20260709 13:05:48.918196 11169 leader_election.cc:343] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:48.918321 11130 raft_consensus.cc:3060] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.919862 11322 raft_consensus.cc:359] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.920665 11322 raft_consensus.cc:385] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.920951 11322 raft_consensus.cc:740] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.921720 11322 consensus_queue.cc:260] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.924115 11322 ts_tablet_manager.cc:1434] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:05:48.925166 11322 tablet_bootstrap.cc:492] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711: Bootstrap starting.
I20260709 13:05:48.928740 11130 raft_consensus.cc:2468] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 1.
I20260709 13:05:48.931562 11321 tablet_bootstrap.cc:654] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.930670 11327 leader_election.cc:290] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:48.932399 11168 leader_election.cc:304] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:48.932523 11327 raft_consensus.cc:2804] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.929569 11319 tablet_bootstrap.cc:492] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:48.933137 11327 raft_consensus.cc:493] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.933682 11327 raft_consensus.cc:3060] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.934608 11278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf50a2fc30148e3b1b5b7e732cb4c1e" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:48.935401 11278 raft_consensus.cc:3060] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.939308 11372 raft_consensus.cc:493] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.931672 11322 tablet_bootstrap.cc:654] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.947479 11374 raft_consensus.cc:2804] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:48.940267 11372 raft_consensus.cc:515] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.953306 11321 tablet_bootstrap.cc:492] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f: No bootstrap required, opened a new log
I20260709 13:05:48.954138 11327 raft_consensus.cc:515] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.960804 11321 ts_tablet_manager.cc:1403] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f: Time spent bootstrapping tablet: real 0.059s	user 0.014s	sys 0.006s
I20260709 13:05:48.961668 11371 raft_consensus.cc:493] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.962263 11371 raft_consensus.cc:515] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.964730 11130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d" is_pre_election: true
I20260709 13:05:48.957649 11374 raft_consensus.cc:697] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Becoming Leader. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:48.966466 11130 raft_consensus.cc:2468] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 0.
I20260709 13:05:48.959913 11278 raft_consensus.cc:2468] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 1.
I20260709 13:05:48.968523 11129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56f50814ef548078489152c9af6ee90" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:48.967494 11372 leader_election.cc:290] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:48.967784 11327 leader_election.cc:290] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:48.980636 11319 tablet_bootstrap.cc:654] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:48.982236 11367 raft_consensus.cc:493] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.974654 11129 raft_consensus.cc:3060] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.975657 11168 leader_election.cc:304] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:48.990139 11130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56f50814ef548078489152c9af6ee90" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d" is_pre_election: true
I20260709 13:05:48.991088 11321 raft_consensus.cc:359] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.991848 11321 raft_consensus.cc:385] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:48.992115 11321 raft_consensus.cc:740] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Initialized, Role: FOLLOWER
I20260709 13:05:48.985642 11367 raft_consensus.cc:515] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.992830 11321 consensus_queue.cc:260] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.993417 11204 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.994881 11367 leader_election.cc:290] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:48.995694 11130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d" is_pre_election: true
I20260709 13:05:48.996394 11377 raft_consensus.cc:2804] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:48.996987 11377 raft_consensus.cc:493] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.996958 11130 raft_consensus.cc:2468] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 0.
I20260709 13:05:48.997537 11377 raft_consensus.cc:3060] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:48.997727 11371 leader_election.cc:290] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:49.000977 11321 ts_tablet_manager.cc:1434] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.001s
I20260709 13:05:49.002084 11321 tablet_bootstrap.cc:492] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f: Bootstrap starting.
I20260709 13:05:49.002979 11205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f" is_pre_election: true
I20260709 13:05:49.004140 11203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56f50814ef548078489152c9af6ee90" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f" is_pre_election: true
I20260709 13:05:49.004693 11242 leader_election.cc:304] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: 
I20260709 13:05:49.005064 11203 raft_consensus.cc:2393] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 93009b2e2d7441f4a9d48b4d094b3711 in current term 1: Already voted for candidate bc100c804e924ea6b35a60fc507e1d6f in this term.
I20260709 13:05:49.009428 11372 raft_consensus.cc:493] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.010078 11372 raft_consensus.cc:515] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.010075 11357 consensus_queue.cc:1048] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:49.011402 11243 leader_election.cc:304] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711; no voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:49.011947 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711" is_pre_election: true
I20260709 13:05:49.012665 11279 raft_consensus.cc:2468] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 0.
I20260709 13:05:49.013156 11367 raft_consensus.cc:2804] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:49.014114 11367 raft_consensus.cc:493] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.013274 11130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81596d1743546d89f983c09be2c83ea" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d" is_pre_election: true
I20260709 13:05:49.014577 11367 raft_consensus.cc:3060] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.015051 11130 raft_consensus.cc:2468] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 0.
I20260709 13:05:49.013660 11371 raft_consensus.cc:3060] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.016291 11168 leader_election.cc:304] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:49.016607 11322 tablet_bootstrap.cc:492] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711: No bootstrap required, opened a new log
I20260709 13:05:49.017302 11322 ts_tablet_manager.cc:1403] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711: Time spent bootstrapping tablet: real 0.092s	user 0.028s	sys 0.019s
I20260709 13:05:49.018836 11372 leader_election.cc:290] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:49.021306 11129 raft_consensus.cc:2468] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 1.
I20260709 13:05:49.022971 11168 leader_election.cc:304] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:49.017168 11327 raft_consensus.cc:2804] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:49.023414 11322 raft_consensus.cc:359] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.023866 11372 raft_consensus.cc:2804] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:49.023936 11327 raft_consensus.cc:493] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.024163 11322 raft_consensus.cc:385] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:49.024570 11327 raft_consensus.cc:3060] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.024580 11322 raft_consensus.cc:740] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.027657 11322 consensus_queue.cc:260] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:48.974457 11374 consensus_queue.cc:237] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.036026 11321 tablet_bootstrap.cc:654] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:49.036679 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81596d1743546d89f983c09be2c83ea" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711" is_pre_election: true
I20260709 13:05:49.037734 11279 raft_consensus.cc:2468] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 0.
I20260709 13:05:49.029315 11278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56f50814ef548078489152c9af6ee90" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:49.042146 11327 raft_consensus.cc:515] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.054185 11129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81596d1743546d89f983c09be2c83ea" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:49.055055 11129 raft_consensus.cc:3060] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.055392 11371 raft_consensus.cc:2749] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:49.056289 11367 raft_consensus.cc:515] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.059191 11372 raft_consensus.cc:697] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Becoming Leader. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:49.062441 11319 tablet_bootstrap.cc:492] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d: No bootstrap required, opened a new log
I20260709 13:05:49.063050 11319 ts_tablet_manager.cc:1403] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.153s	user 0.018s	sys 0.010s
I20260709 13:05:49.065863 11319 raft_consensus.cc:359] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.066543 11319 raft_consensus.cc:385] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:49.066789 11319 raft_consensus.cc:740] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.067492 11319 consensus_queue.cc:260] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.068974 11322 ts_tablet_manager.cc:1434] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.001s
I20260709 13:05:49.073627 11028 catalog_manager.cc:5697] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f reported cstate change: term changed from 0 to 1, leader changed from <none> to bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.079950 11319 ts_tablet_manager.cc:1434] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:05:49.080358 11278 raft_consensus.cc:2468] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 1.
I20260709 13:05:49.080868 11372 consensus_queue.cc:237] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.083029 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81596d1743546d89f983c09be2c83ea" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:49.084509 11279 raft_consensus.cc:3060] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.086994 11130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:49.087321 11203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:49.087713 11130 raft_consensus.cc:3060] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.087569 11377 raft_consensus.cc:515] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.089030 11205 raft_consensus.cc:2393] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 93009b2e2d7441f4a9d48b4d094b3711 in current term 1: Already voted for candidate bc100c804e924ea6b35a60fc507e1d6f in this term.
I20260709 13:05:49.083613 11367 leader_election.cc:290] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:49.096876 11279 raft_consensus.cc:2468] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 1.
I20260709 13:05:49.098292 11169 leader_election.cc:304] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:49.082445 11327 leader_election.cc:290] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:49.099217 11374 raft_consensus.cc:2804] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:49.099889 11374 raft_consensus.cc:697] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Becoming Leader. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:49.100719 11374 consensus_queue.cc:237] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.102240 11129 raft_consensus.cc:2468] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 1.
I20260709 13:05:49.103371 11130 raft_consensus.cc:2468] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 1.
I20260709 13:05:49.104902 11242 leader_election.cc:304] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:49.106189 11367 raft_consensus.cc:2804] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:49.106856 11367 raft_consensus.cc:697] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 LEADER]: Becoming Leader. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:49.107821 11367 consensus_queue.cc:237] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.108136 11321 tablet_bootstrap.cc:492] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f: No bootstrap required, opened a new log
I20260709 13:05:49.108690 11321 ts_tablet_manager.cc:1403] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f: Time spent bootstrapping tablet: real 0.107s	user 0.004s	sys 0.008s
I20260709 13:05:49.112133 11377 leader_election.cc:290] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:49.112902 11130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:49.113186 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:49.113794 11130 raft_consensus.cc:2393] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bc100c804e924ea6b35a60fc507e1d6f in current term 1: Already voted for candidate 93009b2e2d7441f4a9d48b4d094b3711 in this term.
I20260709 13:05:49.115182 11168 leader_election.cc:304] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc100c804e924ea6b35a60fc507e1d6f; no voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711
I20260709 13:05:49.115065 11321 raft_consensus.cc:359] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.115973 11321 raft_consensus.cc:385] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:49.116285 11321 raft_consensus.cc:740] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Initialized, Role: FOLLOWER
I20260709 13:05:49.116500 11377 raft_consensus.cc:2749] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:05:49.117126 11321 consensus_queue.cc:260] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.119210 11321 ts_tablet_manager.cc:1434] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:05:49.121537 11028 catalog_manager.cc:5697] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.122895 11377 raft_consensus.cc:493] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.125968 11377 raft_consensus.cc:515] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.128957 11377 leader_election.cc:290] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:49.129650 11350 consensus_queue.cc:1048] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:49.131498 11130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8ea549dae19466b9e30bde1c7b4721d" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d" is_pre_election: true
I20260709 13:05:49.132278 11130 raft_consensus.cc:2468] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 0.
I20260709 13:05:49.133391 11168 leader_election.cc:304] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:49.131084 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8ea549dae19466b9e30bde1c7b4721d" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711" is_pre_election: true
I20260709 13:05:49.134272 11279 raft_consensus.cc:2468] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 0.
I20260709 13:05:49.134372 11377 raft_consensus.cc:2804] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:49.134905 11377 raft_consensus.cc:493] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.135280 11377 raft_consensus.cc:3060] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.139547 11026 catalog_manager.cc:5697] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f reported cstate change: term changed from 0 to 1, leader changed from <none> to bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.141741 11026 catalog_manager.cc:5697] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f reported cstate change: term changed from 0 to 1, leader changed from <none> to bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.142479 11377 raft_consensus.cc:515] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.145612 11130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8ea549dae19466b9e30bde1c7b4721d" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:49.146454 11377 leader_election.cc:290] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:49.146276 11130 raft_consensus.cc:3060] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.147104 11278 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8ea549dae19466b9e30bde1c7b4721d" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:49.147677 11278 raft_consensus.cc:3060] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.151510 11338 consensus_queue.cc:1048] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:49.154781 11278 raft_consensus.cc:2468] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 1.
I20260709 13:05:49.156950 11169 leader_election.cc:304] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:49.157958 11377 raft_consensus.cc:2804] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:49.156850 11130 raft_consensus.cc:2468] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 1.
I20260709 13:05:49.166611 11377 raft_consensus.cc:697] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Becoming Leader. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:49.167526 11377 consensus_queue.cc:237] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:49.181085 11026 catalog_manager.cc:5697] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f reported cstate change: term changed from 0 to 1, leader changed from <none> to bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.239606 11350 consensus_queue.cc:1048] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:49.253674 11338 consensus_queue.cc:1048] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:49.269322 11367 consensus_queue.cc:1048] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:49.279839 11204 raft_consensus.cc:3060] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:49.290329 11367 consensus_queue.cc:1048] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:49.351840 11338 consensus_queue.cc:1048] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:49.358632   431 ts_manager.cc:295] Set tserver state for 353726f02e9c439792e790fb0958a80d to MAINTENANCE_MODE
I20260709 13:05:49.358639 11338 consensus_queue.cc:1048] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:49.359768   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:05:49.395946 11075 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:49.403541   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:49.404534   431 tablet_replica.cc:333] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d: stopping tablet replica
I20260709 13:05:49.405349   431 raft_consensus.cc:2243] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.405795   431 raft_consensus.cc:2272] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.407826   431 tablet_replica.cc:333] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d: stopping tablet replica
I20260709 13:05:49.408435   431 raft_consensus.cc:2243] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.408831   431 raft_consensus.cc:2272] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.410691   431 tablet_replica.cc:333] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d: stopping tablet replica
I20260709 13:05:49.411369   431 raft_consensus.cc:2243] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.411718   431 raft_consensus.cc:2272] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.413385   431 tablet_replica.cc:333] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d: stopping tablet replica
I20260709 13:05:49.413933   431 raft_consensus.cc:2243] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.414904   431 raft_consensus.cc:2272] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.417284   431 tablet_replica.cc:333] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d: stopping tablet replica
I20260709 13:05:49.417833   431 raft_consensus.cc:2243] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.418606   431 raft_consensus.cc:2272] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.420826   431 tablet_replica.cc:333] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d: stopping tablet replica
I20260709 13:05:49.421314   431 raft_consensus.cc:2243] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.422036   431 raft_consensus.cc:2272] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.424095   431 tablet_replica.cc:333] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d: stopping tablet replica
I20260709 13:05:49.424638   431 raft_consensus.cc:2243] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.424968   431 raft_consensus.cc:2272] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.426815   431 tablet_replica.cc:333] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d: stopping tablet replica
I20260709 13:05:49.427459   431 raft_consensus.cc:2243] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.427838   431 raft_consensus.cc:2272] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.430153   431 tablet_replica.cc:333] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d: stopping tablet replica
I20260709 13:05:49.430629   431 raft_consensus.cc:2243] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:49.431017   431 raft_consensus.cc:2272] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.432687   431 tablet_replica.cc:333] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d: stopping tablet replica
I20260709 13:05:49.433107   431 raft_consensus.cc:2243] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:49.433866   431 raft_consensus.cc:2272] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:49.464193   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:05:49.509492 11367 consensus_queue.cc:1048] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:49.519598 11242 consensus_peers.cc:597] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.548465 11377 consensus_queue.cc:1048] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:49.567660 11168 consensus_peers.cc:597] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.582186 11168 consensus_peers.cc:597] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:49.591681 11168 consensus_peers.cc:597] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.609930 11372 consensus_queue.cc:1048] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:05:49.613142 11168 consensus_peers.cc:597] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:49.617980 11372 consensus_queue.cc:1048] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:49.677155 11374 consensus_queue.cc:1048] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:05:49.869932 11242 consensus_peers.cc:597] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:50.397079 11075 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:50.467651 11367 raft_consensus.cc:493] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 353726f02e9c439792e790fb0958a80d)
I20260709 13:05:50.468299 11367 raft_consensus.cc:515] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:50.470077 11367 leader_election.cc:290] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:50.471313 11204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f" is_pre_election: true
W20260709 13:05:50.471832 11242 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111) [suppressed 145 similar messages]
W20260709 13:05:50.475032 11242 leader_election.cc:336] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111)
I20260709 13:05:50.475423 11242 leader_election.cc:304] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711; no voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:50.476168 11367 raft_consensus.cc:2749] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:50.514902 11374 raft_consensus.cc:493] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 353726f02e9c439792e790fb0958a80d)
I20260709 13:05:50.515561 11374 raft_consensus.cc:515] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:50.517170 11374 leader_election.cc:290] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:50.518297 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711" is_pre_election: true
I20260709 13:05:50.519065 11279 raft_consensus.cc:2468] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 1.
I20260709 13:05:50.520275 11169 leader_election.cc:304] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:50.521036 11374 raft_consensus.cc:2804] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.521442 11374 raft_consensus.cc:493] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 353726f02e9c439792e790fb0958a80d)
I20260709 13:05:50.521930 11374 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:50.522022 11168 leader_election.cc:336] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111)
I20260709 13:05:50.527652 11374 raft_consensus.cc:515] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:50.529338 11374 leader_election.cc:290] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:50.530526 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:50.531214 11279 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:50.533041 11168 leader_election.cc:336] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111)
I20260709 13:05:50.536593 11279 raft_consensus.cc:2468] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 2.
I20260709 13:05:50.537693 11169 leader_election.cc:304] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 353726f02e9c439792e790fb0958a80d
I20260709 13:05:50.538358 11374 raft_consensus.cc:2804] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:50.539317 11374 raft_consensus.cc:697] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Becoming Leader. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:50.540025 11374 consensus_queue.cc:237] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:50.547058 11027 catalog_manager.cc:5697] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f reported cstate change: term changed from 1 to 2, leader changed from 353726f02e9c439792e790fb0958a80d (127.0.107.193) to bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:50.650147 11367 raft_consensus.cc:493] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 353726f02e9c439792e790fb0958a80d)
I20260709 13:05:50.650759 11367 raft_consensus.cc:515] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:50.652549 11367 leader_election.cc:290] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675), 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527)
I20260709 13:05:50.653403 11204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970302250f4e4b4cb92307cc423adceb" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f" is_pre_election: true
I20260709 13:05:50.656628 11374 raft_consensus.cc:493] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 353726f02e9c439792e790fb0958a80d)
W20260709 13:05:50.656790 11242 leader_election.cc:336] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111)
I20260709 13:05:50.657163 11374 raft_consensus.cc:515] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:50.657363 11242 leader_election.cc:304] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711; no voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:50.658178 11367 raft_consensus.cc:2749] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:50.659227 11374 leader_election.cc:290] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727), 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527)
I20260709 13:05:50.659866 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970302250f4e4b4cb92307cc423adceb" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711" is_pre_election: true
I20260709 13:05:50.660431 11279 raft_consensus.cc:2468] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 1.
I20260709 13:05:50.661397 11169 leader_election.cc:304] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:50.662238 11374 raft_consensus.cc:2804] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.662573 11374 raft_consensus.cc:493] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 353726f02e9c439792e790fb0958a80d)
I20260709 13:05:50.662878 11374 raft_consensus.cc:3060] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:50.663298 11168 leader_election.cc:336] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111)
I20260709 13:05:50.668114 11374 raft_consensus.cc:515] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:50.670373 11374 leader_election.cc:290] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 election: Requested vote from peers 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727), 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527)
I20260709 13:05:50.671177 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970302250f4e4b4cb92307cc423adceb" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:50.671777 11279 raft_consensus.cc:3060] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:50.674783 11168 leader_election.cc:336] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111)
I20260709 13:05:50.677758 11279 raft_consensus.cc:2468] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 2.
I20260709 13:05:50.678845 11169 leader_election.cc:304] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 353726f02e9c439792e790fb0958a80d
I20260709 13:05:50.679713 11374 raft_consensus.cc:2804] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:50.680203 11374 raft_consensus.cc:697] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Becoming Leader. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:50.680994 11374 consensus_queue.cc:237] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:50.688299 11027 catalog_manager.cc:5697] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f reported cstate change: term changed from 1 to 2, leader changed from 353726f02e9c439792e790fb0958a80d (127.0.107.193) to bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:50.752651 11374 raft_consensus.cc:493] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 353726f02e9c439792e790fb0958a80d)
I20260709 13:05:50.753208 11374 raft_consensus.cc:515] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:50.755038 11374 leader_election.cc:290] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:50.756194 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebae99d48338401f8a5c420bd5695aa5" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711" is_pre_election: true
I20260709 13:05:50.757726 11367 raft_consensus.cc:493] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 353726f02e9c439792e790fb0958a80d)
I20260709 13:05:50.758287 11367 raft_consensus.cc:515] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
W20260709 13:05:50.759333 11168 leader_election.cc:336] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111)
I20260709 13:05:50.759907 11168 leader_election.cc:304] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc100c804e924ea6b35a60fc507e1d6f; no voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711
I20260709 13:05:50.760427 11367 leader_election.cc:290] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:50.761399 11374 raft_consensus.cc:2749] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:50.761497 11204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebae99d48338401f8a5c420bd5695aa5" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f" is_pre_election: true
I20260709 13:05:50.762208 11204 raft_consensus.cc:2468] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 1.
I20260709 13:05:50.763334 11243 leader_election.cc:304] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
W20260709 13:05:50.764312 11242 leader_election.cc:336] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111)
I20260709 13:05:50.764324 11367 raft_consensus.cc:2804] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:50.764848 11367 raft_consensus.cc:493] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 353726f02e9c439792e790fb0958a80d)
I20260709 13:05:50.765228 11367 raft_consensus.cc:3060] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.770247 11367 raft_consensus.cc:515] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:50.771875 11367 leader_election.cc:290] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:50.772909 11204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebae99d48338401f8a5c420bd5695aa5" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:50.773483 11204 raft_consensus.cc:3060] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:50.775547 11242 leader_election.cc:336] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111)
I20260709 13:05:50.778448 11204 raft_consensus.cc:2468] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 2.
I20260709 13:05:50.779536 11243 leader_election.cc:304] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 353726f02e9c439792e790fb0958a80d
I20260709 13:05:50.780262 11367 raft_consensus.cc:2804] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:50.781136 11367 raft_consensus.cc:697] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Becoming Leader. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:50.781909 11367 consensus_queue.cc:237] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:50.788998 11027 catalog_manager.cc:5697] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 reported cstate change: term changed from 1 to 2, leader changed from 353726f02e9c439792e790fb0958a80d (127.0.107.193) to 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:50.865731 11367 raft_consensus.cc:493] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 353726f02e9c439792e790fb0958a80d)
I20260709 13:05:50.866336 11367 raft_consensus.cc:515] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:50.868340 11367 leader_election.cc:290] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:50.869518 11204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6eb9aaa1494f99a0b3cb2a61935c2f" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f" is_pre_election: true
W20260709 13:05:50.872167 11242 leader_election.cc:336] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111)
I20260709 13:05:50.872558 11242 leader_election.cc:304] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711; no voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:50.873253 11367 raft_consensus.cc:2749] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:50.877282 11374 raft_consensus.cc:493] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 353726f02e9c439792e790fb0958a80d)
I20260709 13:05:50.877812 11374 raft_consensus.cc:515] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:50.879606 11374 leader_election.cc:290] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:50.880926 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6eb9aaa1494f99a0b3cb2a61935c2f" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711" is_pre_election: true
I20260709 13:05:50.881771 11279 raft_consensus.cc:2468] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 1.
I20260709 13:05:50.883123 11169 leader_election.cc:304] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:50.883980 11374 raft_consensus.cc:2804] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:05:50.884012 11168 leader_election.cc:336] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111)
I20260709 13:05:50.884506 11374 raft_consensus.cc:493] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 353726f02e9c439792e790fb0958a80d)
I20260709 13:05:50.884927 11374 raft_consensus.cc:3060] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:50.890470 11374 raft_consensus.cc:515] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:50.892454 11374 leader_election.cc:290] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:50.893661 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6eb9aaa1494f99a0b3cb2a61935c2f" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:50.894346 11279 raft_consensus.cc:3060] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:50.896206 11168 leader_election.cc:336] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111)
I20260709 13:05:50.900369 11279 raft_consensus.cc:2468] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 2.
I20260709 13:05:50.901546 11169 leader_election.cc:304] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 353726f02e9c439792e790fb0958a80d
I20260709 13:05:50.902285 11374 raft_consensus.cc:2804] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:50.902859 11374 raft_consensus.cc:697] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Becoming Leader. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:50.903627 11374 consensus_queue.cc:237] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:50.911453 11027 catalog_manager.cc:5697] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f reported cstate change: term changed from 1 to 2, leader changed from 353726f02e9c439792e790fb0958a80d (127.0.107.193) to bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:51.065629 11168 consensus_peers.cc:597] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:51.088415 11279 raft_consensus.cc:1275] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.089952 11407 consensus_queue.cc:1048] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:51.108175 11168 consensus_peers.cc:597] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:51.171968 11204 raft_consensus.cc:1275] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.173381 11367 consensus_queue.cc:1048] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:51.191039 11242 consensus_peers.cc:597] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:51.198009 11278 raft_consensus.cc:1275] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.199647 11374 consensus_queue.cc:1048] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:51.395409 11168 consensus_peers.cc:597] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:51.398298 11075 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:51.497545 11278 raft_consensus.cc:1275] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:51.499058 11418 consensus_queue.cc:1048] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:05:51.905191 11168 consensus_peers.cc:597] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:51.949136 11168 consensus_peers.cc:597] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:52.043810 11168 consensus_peers.cc:597] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:52.101729 11242 consensus_peers.cc:597] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:52.255725 11242 consensus_peers.cc:597] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:52.287205 11168 consensus_peers.cc:597] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:52.399564 11075 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:05:53.400736 11075 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:05:53.505577 11168 consensus_peers.cc:597] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:53.579984 11242 consensus_peers.cc:597] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:53.766135 11168 consensus_peers.cc:597] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:53.801498 11168 consensus_peers.cc:597] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:05:54.378041 11168 consensus_peers.cc:597] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:54.394302 11168 consensus_peers.cc:597] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:54.401734 11075 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:54.484603   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:54.489673 11242 consensus_peers.cc:597] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:05:54.493304 11429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:54.494153 11428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:54.494851 11431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:54.496219   431 server_base.cc:1061] running on GCE node
I20260709 13:05:54.498100   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:54.498350   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:54.498543   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602354498520 us; error 0 us; skew 500 ppm
I20260709 13:05:54.499121   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:54.501847   431 webserver.cc:533] Webserver started at http://127.0.107.193:36023/ using document root <none> and password file <none>
I20260709 13:05:54.502367   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:54.502576   431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:05:54.506219   431 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:05:54.510345 11436 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:54.511462   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 13:05:54.511818   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-0-root
uuid: "353726f02e9c439792e790fb0958a80d"
format_stamp: "Formatted at 2026-07-09 13:05:47 on dist-test-slave-cq4t"
I20260709 13:05:54.512133   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602014632082-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 13:05:54.541548 11168 consensus_peers.cc:597] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:54.542605   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:54.544184   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:54.546067   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:54.551314 11443 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:05:54.606624   431 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:05:54.607034   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.058s	user 0.000s	sys 0.004s
I20260709 13:05:54.607295   431 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:05:54.612169 11443 tablet_bootstrap.cc:492] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:54.623904 11443 tablet_bootstrap.cc:492] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.624652 11443 tablet_bootstrap.cc:492] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d: Bootstrap complete.
I20260709 13:05:54.625270 11443 ts_tablet_manager.cc:1403] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:05:54.627388 11443 raft_consensus.cc:359] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.627933 11443 raft_consensus.cc:740] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.628625 11443 consensus_queue.cc:260] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.630863 11443 ts_tablet_manager.cc:1434] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:05:54.631711 11443 tablet_bootstrap.cc:492] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:54.644454 11443 tablet_bootstrap.cc:492] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.645328 11443 tablet_bootstrap.cc:492] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d: Bootstrap complete.
I20260709 13:05:54.645924 11443 ts_tablet_manager.cc:1403] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.014s	user 0.005s	sys 0.006s
I20260709 13:05:54.648116 11443 raft_consensus.cc:359] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.648737 11443 raft_consensus.cc:740] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.649556   431 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:05:54.649343 11443 consensus_queue.cc:260] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.649921   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.043s	user 0.038s	sys 0.002s
I20260709 13:05:54.651145 11443 ts_tablet_manager.cc:1434] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.002s
I20260709 13:05:54.651916 11443 tablet_bootstrap.cc:492] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:54.665387 11443 tablet_bootstrap.cc:492] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.666476 11443 tablet_bootstrap.cc:492] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d: Bootstrap complete.
I20260709 13:05:54.667176 11443 ts_tablet_manager.cc:1403] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.007s
I20260709 13:05:54.670433 11443 raft_consensus.cc:359] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.671269 11443 raft_consensus.cc:740] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.672078 11443 consensus_queue.cc:260] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.674324 11443 ts_tablet_manager.cc:1434] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:05:54.675352 11443 tablet_bootstrap.cc:492] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:54.689718 11443 tablet_bootstrap.cc:492] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.690416 11443 tablet_bootstrap.cc:492] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d: Bootstrap complete.
I20260709 13:05:54.691076 11443 ts_tablet_manager.cc:1403] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.004s
I20260709 13:05:54.693284 11443 raft_consensus.cc:359] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.693830 11443 raft_consensus.cc:740] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.694500 11443 consensus_queue.cc:260] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.696223 11443 ts_tablet_manager.cc:1434] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:54.696863 11443 tablet_bootstrap.cc:492] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:54.708787 11443 tablet_bootstrap.cc:492] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.709601 11443 tablet_bootstrap.cc:492] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d: Bootstrap complete.
I20260709 13:05:54.710167 11443 ts_tablet_manager.cc:1403] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.001s
I20260709 13:05:54.710533   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:33527
I20260709 13:05:54.710652 11506 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:33527 every 8 connection(s)
I20260709 13:05:54.712381 11443 raft_consensus.cc:359] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:54.712968 11443 raft_consensus.cc:740] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.713712 11443 consensus_queue.cc:260] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:54.715478 11443 ts_tablet_manager.cc:1434] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.002s
I20260709 13:05:54.716180 11443 tablet_bootstrap.cc:492] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:54.717542   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:54.744164 11443 tablet_bootstrap.cc:492] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.744927 11443 tablet_bootstrap.cc:492] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d: Bootstrap complete.
I20260709 13:05:54.745532 11443 ts_tablet_manager.cc:1403] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.008s
I20260709 13:05:54.747995 11443 raft_consensus.cc:359] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.748788 11443 raft_consensus.cc:740] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.749854 11443 consensus_queue.cc:260] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
W20260709 13:05:54.750200 11242 consensus_peers.cc:597] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:05:54.750974 11507 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39169
I20260709 13:05:54.751469 11507 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:54.752053 11443 ts_tablet_manager.cc:1434] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:05:54.752503 11205 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a6eb9aaa1494f99a0b3cb2a61935c2f"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
mode: GRACEFUL
new_leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:55628
I20260709 13:05:54.752991 11443 tablet_bootstrap.cc:492] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:54.753376 11205 raft_consensus.cc:606] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Received request to transfer leadership to TS 93009b2e2d7441f4a9d48b4d094b3711
I20260709 13:05:54.752488 11507 heartbeater.cc:507] Master 127.0.107.254:39169 requested a full tablet report, sending...
I20260709 13:05:54.756559 11205 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "970302250f4e4b4cb92307cc423adceb"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
mode: GRACEFUL
new_leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:55628
I20260709 13:05:54.757162 11205 raft_consensus.cc:606] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Received request to transfer leadership to TS 93009b2e2d7441f4a9d48b4d094b3711
I20260709 13:05:54.760317 11205 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d56f50814ef548078489152c9af6ee90"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
mode: GRACEFUL
new_leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:55628
I20260709 13:05:54.760701 11026 ts_manager.cc:194] Re-registered known tserver with Master: 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527)
I20260709 13:05:54.761005 11205 raft_consensus.cc:606] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Received request to transfer leadership to TS 93009b2e2d7441f4a9d48b4d094b3711
I20260709 13:05:54.762620   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:54.763281   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:54.764173   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:54.767241 11443 tablet_bootstrap.cc:492] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.768138 11443 tablet_bootstrap.cc:492] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d: Bootstrap complete.
I20260709 13:05:54.768787 11443 ts_tablet_manager.cc:1403] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 13:05:54.771376 11443 raft_consensus.cc:359] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.772125 11443 raft_consensus.cc:740] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.772815 11443 consensus_queue.cc:260] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.773893 11026 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41576
I20260709 13:05:54.775224 11443 ts_tablet_manager.cc:1434] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:05:54.776144 11443 tablet_bootstrap.cc:492] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:54.777812 11507 heartbeater.cc:499] Master 127.0.107.254:39169 was elected leader, sending a full tablet report...
I20260709 13:05:54.791334 11443 tablet_bootstrap.cc:492] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.792375 11443 tablet_bootstrap.cc:492] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d: Bootstrap complete.
I20260709 13:05:54.793026 11443 ts_tablet_manager.cc:1403] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:05:54.795523 11443 raft_consensus.cc:359] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.796209 11443 raft_consensus.cc:740] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.796927 11443 consensus_queue.cc:260] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.798818 11443 ts_tablet_manager.cc:1434] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:05:54.799661 11443 tablet_bootstrap.cc:492] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:54.811841 11443 tablet_bootstrap.cc:492] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.812577 11443 tablet_bootstrap.cc:492] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d: Bootstrap complete.
I20260709 13:05:54.813253 11443 ts_tablet_manager.cc:1403] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:05:54.813697 11481 raft_consensus.cc:3060] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:54.815845 11443 raft_consensus.cc:359] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.816584 11443 raft_consensus.cc:740] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.817317 11443 consensus_queue.cc:260] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.819450 11443 ts_tablet_manager.cc:1434] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:05:54.820333 11443 tablet_bootstrap.cc:492] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d: Bootstrap starting.
I20260709 13:05:54.838150 11443 tablet_bootstrap.cc:492] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:54.839094 11443 tablet_bootstrap.cc:492] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d: Bootstrap complete.
I20260709 13:05:54.839815 11443 ts_tablet_manager.cc:1403] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.007s
I20260709 13:05:54.842139 11443 raft_consensus.cc:359] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.842840 11443 raft_consensus.cc:740] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353726f02e9c439792e790fb0958a80d, State: Initialized, Role: FOLLOWER
I20260709 13:05:54.843552 11443 consensus_queue.cc:260] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.845600 11443 ts_tablet_manager.cc:1434] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:05:54.919942 11444 raft_consensus.cc:993] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f: : Instructing follower 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:54.920575 11427 raft_consensus.cc:1081] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Signalling peer 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:54.922676 11279 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d56f50814ef548078489152c9af6ee90"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:45600
I20260709 13:05:54.923476 11279 raft_consensus.cc:493] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:54.923879 11279 raft_consensus.cc:3060] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:54.930114 11279 raft_consensus.cc:515] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.932710 11279 leader_election.cc:290] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:54.933100 11481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56f50814ef548078489152c9af6ee90" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:54.933368 11205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56f50814ef548078489152c9af6ee90" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:54.933871 11479 raft_consensus.cc:3060] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:54.934024 11205 raft_consensus.cc:3055] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:54.933864 11481 raft_consensus.cc:3060] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:54.934775 11205 raft_consensus.cc:740] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:54.935889 11205 consensus_queue.cc:260] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.937282 11205 raft_consensus.cc:3060] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:54.942304 11481 raft_consensus.cc:2468] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 2.
I20260709 13:05:54.943869 11242 leader_election.cc:304] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: 
I20260709 13:05:54.944849 11414 raft_consensus.cc:2804] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:54.945050 11205 raft_consensus.cc:2468] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 2.
I20260709 13:05:54.947108 11414 raft_consensus.cc:697] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Becoming Leader. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:54.948081 11414 consensus_queue.cc:237] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:54.951666 11479 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:54.958830 11026 catalog_manager.cc:5697] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 reported cstate change: term changed from 1 to 2, leader changed from bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194) to 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:55.029242 11444 raft_consensus.cc:993] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f: : Instructing follower 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:55.029758 11427 raft_consensus.cc:1081] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Signalling peer 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:55.031559 11279 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a6eb9aaa1494f99a0b3cb2a61935c2f"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:45600
I20260709 13:05:55.032174 11279 raft_consensus.cc:493] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:55.032590 11279 raft_consensus.cc:3060] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:55.038226 11279 raft_consensus.cc:515] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:55.040714 11279 leader_election.cc:290] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 3 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:55.041147 11479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6eb9aaa1494f99a0b3cb2a61935c2f" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:55.041249 11444 raft_consensus.cc:993] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f: : Instructing follower 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:55.041319 11205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a6eb9aaa1494f99a0b3cb2a61935c2f" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:55.041985 11479 raft_consensus.cc:3060] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:05:55.042124 11444 raft_consensus.cc:1081] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Signalling peer 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:55.042202 11205 raft_consensus.cc:3055] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:55.042963 11205 raft_consensus.cc:740] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:55.044263 11279 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "970302250f4e4b4cb92307cc423adceb"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:45600
I20260709 13:05:55.043926 11205 consensus_queue.cc:260] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:55.044827 11279 raft_consensus.cc:493] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:55.045177 11279 raft_consensus.cc:3060] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:55.045710 11205 raft_consensus.cc:3060] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:55.050401 11479 raft_consensus.cc:2468] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 3.
I20260709 13:05:55.051798 11242 leader_election.cc:304] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: 
I20260709 13:05:55.052037 11279 raft_consensus.cc:515] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:55.052767 11414 raft_consensus.cc:2804] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:55.053309 11414 raft_consensus.cc:697] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 LEADER]: Becoming Leader. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:55.053756 11205 raft_consensus.cc:2468] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 3.
I20260709 13:05:55.054718 11279 leader_election.cc:290] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 3 election: Requested vote from peers bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675), 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527)
I20260709 13:05:55.054276 11414 consensus_queue.cc:237] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:55.055778 11205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970302250f4e4b4cb92307cc423adceb" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:55.056349 11479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "970302250f4e4b4cb92307cc423adceb" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:55.056604 11205 raft_consensus.cc:3055] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:55.057010 11205 raft_consensus.cc:740] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:55.057089 11479 raft_consensus.cc:3060] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:55.058033 11205 consensus_queue.cc:260] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:55.059496 11205 raft_consensus.cc:3060] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:55.064533 11026 catalog_manager.cc:5697] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 reported cstate change: term changed from 2 to 3, leader changed from bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194) to 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:55.065610 11479 raft_consensus.cc:2468] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 3.
I20260709 13:05:55.067054 11242 leader_election.cc:304] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: 
I20260709 13:05:55.067780 11205 raft_consensus.cc:2468] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 3.
I20260709 13:05:55.068046 11414 raft_consensus.cc:2804] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:55.068737 11414 raft_consensus.cc:697] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 LEADER]: Becoming Leader. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:55.069659 11414 consensus_queue.cc:237] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } }
I20260709 13:05:55.079058 11026 catalog_manager.cc:5697] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 reported cstate change: term changed from 2 to 3, leader changed from bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194) to 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:55.402812 11075 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:55.432385 11479 raft_consensus.cc:1275] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:55.433972 11535 consensus_queue.cc:1048] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:55.446456 11205 raft_consensus.cc:1275] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:55.448190 11535 consensus_queue.cc:1048] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:55.459074 11479 raft_consensus.cc:1275] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:55.460506 11535 consensus_queue.cc:1048] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:55.469874 11205 raft_consensus.cc:1275] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:55.471468 11414 consensus_queue.cc:1048] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:55.525945 11479 raft_consensus.cc:1275] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:55.527958 11524 consensus_queue.cc:1048] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:55.542515 11205 raft_consensus.cc:1275] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:55.547431 11414 consensus_queue.cc:1048] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:55.813946   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:55.834879 11279 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5323137cf0c94b33bbde48d8b7ce1e01"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
mode: GRACEFUL
new_leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
 from {username='slave'} at 127.0.0.1:33822
I20260709 13:05:55.835820 11279 raft_consensus.cc:606] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 LEADER]: Received request to transfer leadership to TS bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:55.838572 11205 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8ea549dae19466b9e30bde1c7b4721d"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
mode: GRACEFUL
new_leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:55628
I20260709 13:05:55.839208 11205 raft_consensus.cc:606] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Received request to transfer leadership to TS 93009b2e2d7441f4a9d48b4d094b3711
I20260709 13:05:55.842425 11279 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
mode: GRACEFUL
new_leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
 from {username='slave'} at 127.0.0.1:33822
I20260709 13:05:55.842996 11279 raft_consensus.cc:606] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 LEADER]: Received request to transfer leadership to TS bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:55.844239   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:55.844827   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:55.845479   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:56.004726 11544 raft_consensus.cc:993] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f: : Instructing follower 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:56.005205 11444 raft_consensus.cc:1081] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Signalling peer 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:56.007079 11279 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8ea549dae19466b9e30bde1c7b4721d"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:45600
I20260709 13:05:56.007711 11279 raft_consensus.cc:493] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:56.008015 11279 raft_consensus.cc:3060] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.013295 11279 raft_consensus.cc:515] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:56.015213 11279 leader_election.cc:290] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:56.016186 11479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8ea549dae19466b9e30bde1c7b4721d" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:56.016458 11205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8ea549dae19466b9e30bde1c7b4721d" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:56.016921 11479 raft_consensus.cc:3060] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.017045 11205 raft_consensus.cc:3055] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:56.017494 11205 raft_consensus.cc:740] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:56.018350 11205 consensus_queue.cc:260] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:56.019637 11205 raft_consensus.cc:3060] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.025291 11479 raft_consensus.cc:2468] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 2.
I20260709 13:05:56.026952 11242 leader_election.cc:304] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: 
I20260709 13:05:56.027681 11205 raft_consensus.cc:2468] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 2.
I20260709 13:05:56.027761 11414 raft_consensus.cc:2804] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:56.028399 11414 raft_consensus.cc:697] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Becoming Leader. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:56.029256 11414 consensus_queue.cc:237] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:56.037139 11026 catalog_manager.cc:5697] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 reported cstate change: term changed from 1 to 2, leader changed from bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194) to 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.222769 11535 raft_consensus.cc:993] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711: : Instructing follower bc100c804e924ea6b35a60fc507e1d6f to start an election
I20260709 13:05:56.223297 11535 raft_consensus.cc:1081] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 LEADER]: Signalling peer bc100c804e924ea6b35a60fc507e1d6f to start an election
I20260709 13:05:56.225097 11205 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5323137cf0c94b33bbde48d8b7ce1e01"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
 from {username='slave'} at 127.0.0.1:53778
I20260709 13:05:56.225769 11205 raft_consensus.cc:493] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:56.226184 11205 raft_consensus.cc:3060] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.232649 11205 raft_consensus.cc:515] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:56.234567 11205 leader_election.cc:290] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:56.235502 11481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5323137cf0c94b33bbde48d8b7ce1e01" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:56.235852 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5323137cf0c94b33bbde48d8b7ce1e01" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:56.236231 11481 raft_consensus.cc:3060] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.236526 11279 raft_consensus.cc:3055] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:56.236856 11279 raft_consensus.cc:740] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:56.237699 11279 consensus_queue.cc:260] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:56.238833 11279 raft_consensus.cc:3060] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.242689 11481 raft_consensus.cc:2468] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 2.
I20260709 13:05:56.243933 11168 leader_election.cc:304] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:56.244621 11444 raft_consensus.cc:2804] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:56.245090 11444 raft_consensus.cc:697] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Becoming Leader. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:56.246042 11279 raft_consensus.cc:2468] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 2.
I20260709 13:05:56.245927 11444 consensus_queue.cc:237] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:56.253271 11026 catalog_manager.cc:5697] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f reported cstate change: term changed from 1 to 2, leader changed from 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195) to bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.283715 11414 raft_consensus.cc:993] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711: : Instructing follower bc100c804e924ea6b35a60fc507e1d6f to start an election
I20260709 13:05:56.284214 11535 raft_consensus.cc:1081] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 LEADER]: Signalling peer bc100c804e924ea6b35a60fc507e1d6f to start an election
I20260709 13:05:56.285560 11205 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
 from {username='slave'} at 127.0.0.1:53778
I20260709 13:05:56.286074 11205 raft_consensus.cc:493] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:56.286391 11205 raft_consensus.cc:3060] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.291618 11205 raft_consensus.cc:515] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:56.293669 11205 leader_election.cc:290] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:56.294972 11481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:56.295171 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:56.295784 11481 raft_consensus.cc:3060] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.295845 11279 raft_consensus.cc:3055] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:56.296396 11279 raft_consensus.cc:740] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:56.297181 11279 consensus_queue.cc:260] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:56.298426 11279 raft_consensus.cc:3060] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:56.302834 11481 raft_consensus.cc:2468] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 2.
I20260709 13:05:56.303392 11279 raft_consensus.cc:2468] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 2.
I20260709 13:05:56.304262 11168 leader_election.cc:304] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:56.305321 11544 raft_consensus.cc:2804] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:56.305873 11544 raft_consensus.cc:697] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Becoming Leader. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:56.306646 11544 consensus_queue.cc:237] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:56.314796 11026 catalog_manager.cc:5697] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f reported cstate change: term changed from 1 to 2, leader changed from 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195) to bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194). New cstate: current_term: 2 leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:56.403924 11075 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:05:56.615427 11059 debug-util.cc:398] Leaking SignalData structure 0x7b0800064e00 after lost signal to thread 434
W20260709 13:05:56.616654 11059 debug-util.cc:398] Leaking SignalData structure 0x7b08003203a0 after lost signal to thread 11062
W20260709 13:05:56.617789 11059 debug-util.cc:398] Leaking SignalData structure 0x7b08000b3540 after lost signal to thread 11229
W20260709 13:05:56.619642 11059 debug-util.cc:398] Leaking SignalData structure 0x7b08003672a0 after lost signal to thread 11506
I20260709 13:05:56.630808 11481 raft_consensus.cc:1275] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:56.632715 11535 consensus_queue.cc:1048] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:56.647799 11205 raft_consensus.cc:1275] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:56.649902 11557 consensus_queue.cc:1048] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:56.711301 11479 raft_consensus.cc:1275] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:56.712760 11544 consensus_queue.cc:1048] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:56.723927 11279 raft_consensus.cc:1275] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:56.725880 11563 consensus_queue.cc:1048] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:56.826511 11479 raft_consensus.cc:1275] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:56.828040 11444 consensus_queue.cc:1048] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:56.839517 11279 raft_consensus.cc:1275] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:56.841359 11544 consensus_queue.cc:1048] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:56.853643   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:56.866725 11205 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
mode: GRACEFUL
new_leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:55628
I20260709 13:05:56.867290 11205 raft_consensus.cc:606] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Received request to transfer leadership to TS 93009b2e2d7441f4a9d48b4d094b3711
I20260709 13:05:56.870342 11279 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d56f50814ef548078489152c9af6ee90"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
mode: GRACEFUL
new_leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
 from {username='slave'} at 127.0.0.1:33822
I20260709 13:05:56.871035 11279 raft_consensus.cc:606] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Received request to transfer leadership to TS bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:56.873869 11205 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
mode: GRACEFUL
new_leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:55628
I20260709 13:05:56.874425 11205 raft_consensus.cc:606] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Received request to transfer leadership to TS 93009b2e2d7441f4a9d48b4d094b3711
I20260709 13:05:56.875775   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:56.876484   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:56.877257   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:57.183262 11567 raft_consensus.cc:993] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711: : Instructing follower bc100c804e924ea6b35a60fc507e1d6f to start an election
I20260709 13:05:57.183914 11557 raft_consensus.cc:1081] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Signalling peer bc100c804e924ea6b35a60fc507e1d6f to start an election
I20260709 13:05:57.185999 11205 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d56f50814ef548078489152c9af6ee90"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
 from {username='slave'} at 127.0.0.1:53778
I20260709 13:05:57.186661 11205 raft_consensus.cc:493] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.187234 11205 raft_consensus.cc:3060] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.195937 11205 raft_consensus.cc:515] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:57.200500 11479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56f50814ef548078489152c9af6ee90" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:57.201247 11479 raft_consensus.cc:3060] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.204665 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56f50814ef548078489152c9af6ee90" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:57.205458 11279 raft_consensus.cc:3055] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:57.205828 11279 raft_consensus.cc:740] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:57.206583 11279 consensus_queue.cc:260] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:57.207612 11279 raft_consensus.cc:3060] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.208267 11205 leader_election.cc:290] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 3 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:57.211131 11479 raft_consensus.cc:2468] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 3.
I20260709 13:05:57.212813 11168 leader_election.cc:304] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:57.213869 11544 raft_consensus.cc:2804] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:57.214469 11544 raft_consensus.cc:697] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 3 LEADER]: Becoming Leader. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:57.215425 11544 consensus_queue.cc:237] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:57.215931 11279 raft_consensus.cc:2468] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 3.
I20260709 13:05:57.223670 11026 catalog_manager.cc:5697] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f reported cstate change: term changed from 2 to 3, leader changed from 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195) to bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194). New cstate: current_term: 3 leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.230086 11562 raft_consensus.cc:993] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f: : Instructing follower 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:57.230686 11562 raft_consensus.cc:1081] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Signalling peer 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:57.232770 11279 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:45600
I20260709 13:05:57.233428 11279 raft_consensus.cc:493] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.233883 11279 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.240078 11279 raft_consensus.cc:515] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:57.241775 11279 leader_election.cc:290] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 3 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:57.242965 11479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:57.243152 11204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:57.243719 11479 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.243899 11204 raft_consensus.cc:3055] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:57.244365 11204 raft_consensus.cc:740] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:57.245256 11204 consensus_queue.cc:260] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:57.246505 11204 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.250970 11479 raft_consensus.cc:2468] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 3.
I20260709 13:05:57.252519 11242 leader_election.cc:304] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: 
I20260709 13:05:57.253273 11567 raft_consensus.cc:2804] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:57.253762 11567 raft_consensus.cc:697] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 LEADER]: Becoming Leader. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:57.254556 11567 consensus_queue.cc:237] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:57.254951 11204 raft_consensus.cc:2468] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 3.
I20260709 13:05:57.264098 11026 catalog_manager.cc:5697] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 reported cstate change: term changed from 2 to 3, leader changed from bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194) to 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.379405 11563 raft_consensus.cc:993] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f: : Instructing follower 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:57.380540 11562 raft_consensus.cc:1081] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Signalling peer 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:57.383422 11279 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:45600
I20260709 13:05:57.384364 11279 raft_consensus.cc:493] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.385036 11279 raft_consensus.cc:3060] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.394261 11279 raft_consensus.cc:515] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:57.396416 11279 leader_election.cc:290] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 3 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:57.398056 11481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:57.398527 11204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28402092b5d42a6a0dde934ed6d0154" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:57.399065 11481 raft_consensus.cc:3060] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.399425 11204 raft_consensus.cc:3055] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:57.399845 11204 raft_consensus.cc:740] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:57.401083 11204 consensus_queue.cc:260] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:57.403110 11204 raft_consensus.cc:3060] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:57.405540 11075 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:57.408453 11481 raft_consensus.cc:2468] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 3.
I20260709 13:05:57.409878 11242 leader_election.cc:304] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: 
I20260709 13:05:57.410899 11567 raft_consensus.cc:2804] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:57.411437 11567 raft_consensus.cc:697] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 LEADER]: Becoming Leader. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:57.412909 11567 consensus_queue.cc:237] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:57.416220 11204 raft_consensus.cc:2468] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 3.
I20260709 13:05:57.423993 11026 catalog_manager.cc:5697] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 reported cstate change: term changed from 2 to 3, leader changed from bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194) to 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.603180 11481 raft_consensus.cc:1275] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:57.604869 11562 consensus_queue.cc:1048] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:57.626248 11279 raft_consensus.cc:1275] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:57.629575 11544 consensus_queue.cc:1048] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:57.653021 11481 raft_consensus.cc:1275] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:57.654358 11567 consensus_queue.cc:1048] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:57.667770 11204 raft_consensus.cc:1275] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:57.670001 11568 consensus_queue.cc:1048] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:57.793340 11481 raft_consensus.cc:1275] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:57.794837 11568 consensus_queue.cc:1048] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:57.805428 11204 raft_consensus.cc:1275] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:57.807811 11568 consensus_queue.cc:1048] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:57.888191   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:57.898979 11279 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
mode: GRACEFUL
new_leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
 from {username='slave'} at 127.0.0.1:33822
I20260709 13:05:57.899772 11279 raft_consensus.cc:606] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 LEADER]: Received request to transfer leadership to TS bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:57.903318 11204 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbf50a2fc30148e3b1b5b7e732cb4c1e"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
mode: GRACEFUL
new_leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:55628
I20260709 13:05:57.904160 11204 raft_consensus.cc:606] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Received request to transfer leadership to TS 93009b2e2d7441f4a9d48b4d094b3711
I20260709 13:05:57.907995 11279 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ebae99d48338401f8a5c420bd5695aa5"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
mode: GRACEFUL
new_leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
 from {username='slave'} at 127.0.0.1:33822
I20260709 13:05:57.908659 11279 raft_consensus.cc:606] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Received request to transfer leadership to TS bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:57.910094   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:57.910713   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:57.911588   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:57.979187 11568 raft_consensus.cc:993] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711: : Instructing follower bc100c804e924ea6b35a60fc507e1d6f to start an election
I20260709 13:05:57.979895 11557 raft_consensus.cc:1081] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Signalling peer bc100c804e924ea6b35a60fc507e1d6f to start an election
I20260709 13:05:57.981982 11204 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ebae99d48338401f8a5c420bd5695aa5"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
 from {username='slave'} at 127.0.0.1:53778
I20260709 13:05:57.982528 11204 raft_consensus.cc:493] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:57.983119 11204 raft_consensus.cc:3060] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.989121 11204 raft_consensus.cc:515] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:57.991170 11204 leader_election.cc:290] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 3 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:57.992548 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebae99d48338401f8a5c420bd5695aa5" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:57.992657 11481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebae99d48338401f8a5c420bd5695aa5" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:57.993197 11279 raft_consensus.cc:3055] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:57.993500 11481 raft_consensus.cc:3060] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:57.993564 11279 raft_consensus.cc:740] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:57.994786 11279 consensus_queue.cc:260] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:57.996469 11279 raft_consensus.cc:3060] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:58.002754 11279 raft_consensus.cc:2468] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 3.
I20260709 13:05:58.004226 11169 leader_election.cc:304] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:58.005075 11563 raft_consensus.cc:2804] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:58.005563 11563 raft_consensus.cc:697] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 3 LEADER]: Becoming Leader. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:58.006558 11481 raft_consensus.cc:2468] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 3.
I20260709 13:05:58.006651 11563 consensus_queue.cc:237] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:58.019564 11028 catalog_manager.cc:5697] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f reported cstate change: term changed from 2 to 3, leader changed from 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195) to bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194). New cstate: current_term: 3 leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:58.240909 11567 raft_consensus.cc:993] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711: : Instructing follower bc100c804e924ea6b35a60fc507e1d6f to start an election
I20260709 13:05:58.241511 11557 raft_consensus.cc:1081] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 LEADER]: Signalling peer bc100c804e924ea6b35a60fc507e1d6f to start an election
I20260709 13:05:58.243304 11204 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
 from {username='slave'} at 127.0.0.1:53778
I20260709 13:05:58.244035 11204 raft_consensus.cc:493] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:58.244457 11204 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:58.253712 11204 raft_consensus.cc:515] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:58.256124 11204 leader_election.cc:290] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 4 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:58.257269 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:58.257138 11481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:58.258044 11279 raft_consensus.cc:3055] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:58.258140 11481 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:58.258364 11279 raft_consensus.cc:740] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:58.259451 11279 consensus_queue.cc:260] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:58.260394 11279 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:58.266004 11279 raft_consensus.cc:2468] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 4.
I20260709 13:05:58.266870 11481 raft_consensus.cc:2468] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 4.
I20260709 13:05:58.267421 11169 leader_election.cc:304] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:58.268524 11563 raft_consensus.cc:2804] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:58.269044 11563 raft_consensus.cc:697] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 4 LEADER]: Becoming Leader. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:58.269996 11563 consensus_queue.cc:237] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:58.280069 11028 catalog_manager.cc:5697] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f reported cstate change: term changed from 3 to 4, leader changed from 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195) to bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194). New cstate: current_term: 4 leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:58.406853 11075 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:58.421396 11544 raft_consensus.cc:993] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f: : Instructing follower 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:58.422134 11544 raft_consensus.cc:1081] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Signalling peer 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:58.424391 11279 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbf50a2fc30148e3b1b5b7e732cb4c1e"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:45600
I20260709 13:05:58.424899 11279 raft_consensus.cc:493] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:58.425248 11279 raft_consensus.cc:3060] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.431391 11279 raft_consensus.cc:515] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:58.432917 11279 leader_election.cc:290] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:58.434134 11204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf50a2fc30148e3b1b5b7e732cb4c1e" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:58.434757 11204 raft_consensus.cc:3055] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:58.434593 11481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf50a2fc30148e3b1b5b7e732cb4c1e" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:58.435213 11204 raft_consensus.cc:740] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:58.435482 11481 raft_consensus.cc:3060] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.436295 11204 consensus_queue.cc:260] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:58.437827 11204 raft_consensus.cc:3060] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:58.443847 11481 raft_consensus.cc:2468] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 2.
I20260709 13:05:58.445211 11242 leader_election.cc:304] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: 
I20260709 13:05:58.445570 11204 raft_consensus.cc:2468] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 2.
I20260709 13:05:58.446143 11567 raft_consensus.cc:2804] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:58.446947 11567 raft_consensus.cc:697] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Becoming Leader. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:58.448164 11567 consensus_queue.cc:237] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:58.457691 11028 catalog_manager.cc:5697] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 reported cstate change: term changed from 1 to 2, leader changed from bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194) to 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:58.512853 11279 raft_consensus.cc:1275] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:58.514668 11544 consensus_queue.cc:1048] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:58.531874 11481 raft_consensus.cc:1275] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:58.533505 11562 consensus_queue.cc:1048] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:58.777159 11279 raft_consensus.cc:1275] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 4 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:58.778376 11544 consensus_queue.cc:1048] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:58.785655 11481 raft_consensus.cc:1275] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 4 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:58.787807 11544 consensus_queue.cc:1048] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:58.845619 11204 raft_consensus.cc:1275] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.847057 11567 consensus_queue.cc:1048] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:58.864657 11481 raft_consensus.cc:1275] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:58.868629 11567 consensus_queue.cc:1048] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:58.919020   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:58.927094 11204 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
mode: GRACEFUL
new_leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:55628
I20260709 13:05:58.927714 11204 raft_consensus.cc:606] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 4 LEADER]: Received request to transfer leadership to TS 93009b2e2d7441f4a9d48b4d094b3711
I20260709 13:05:58.930840 11204 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a81596d1743546d89f983c09be2c83ea"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
mode: GRACEFUL
new_leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:55628
I20260709 13:05:58.931481 11204 raft_consensus.cc:606] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Received request to transfer leadership to TS 93009b2e2d7441f4a9d48b4d094b3711
I20260709 13:05:58.934681 11279 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbf50a2fc30148e3b1b5b7e732cb4c1e"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
mode: GRACEFUL
new_leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
 from {username='slave'} at 127.0.0.1:33822
I20260709 13:05:58.935276 11279 raft_consensus.cc:606] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Received request to transfer leadership to TS bc100c804e924ea6b35a60fc507e1d6f
I20260709 13:05:58.936697   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:58.937347   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:58.938086   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:59.186241 11562 raft_consensus.cc:993] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f: : Instructing follower 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:59.186753 11544 raft_consensus.cc:1081] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 4 LEADER]: Signalling peer 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:59.188619 11279 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:45600
I20260709 13:05:59.189323 11279 raft_consensus.cc:493] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:59.189795 11279 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.197311 11279 raft_consensus.cc:515] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:59.199790 11279 leader_election.cc:290] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 5 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:59.200984 11204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:59.201751 11204 raft_consensus.cc:3055] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:59.202178 11204 raft_consensus.cc:740] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:59.202885 11204 consensus_queue.cc:260] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:59.203887 11204 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.203639 11479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35015db7e03c4af6a10a7a81b575dea9" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:59.204478 11479 raft_consensus.cc:3060] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.209628 11204 raft_consensus.cc:2468] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 5.
I20260709 13:05:59.211097 11243 leader_election.cc:304] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:59.211869 11597 raft_consensus.cc:2804] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:59.212350 11597 raft_consensus.cc:697] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 5 LEADER]: Becoming Leader. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:59.213202 11597 consensus_queue.cc:237] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:59.214502 11479 raft_consensus.cc:2468] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 5.
I20260709 13:05:59.220898 11026 catalog_manager.cc:5697] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 reported cstate change: term changed from 4 to 5, leader changed from bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194) to 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195). New cstate: current_term: 5 leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:59.298615 11562 raft_consensus.cc:993] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f: : Instructing follower 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:59.299222 11562 raft_consensus.cc:1081] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Signalling peer 93009b2e2d7441f4a9d48b4d094b3711 to start an election
I20260709 13:05:59.300861 11279 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a81596d1743546d89f983c09be2c83ea"
dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
 from {username='slave'} at 127.0.0.1:45600
I20260709 13:05:59.301347 11279 raft_consensus.cc:493] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:59.301707 11279 raft_consensus.cc:3060] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.307037 11279 raft_consensus.cc:515] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:59.308549 11279 leader_election.cc:290] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675)
I20260709 13:05:59.309715 11479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81596d1743546d89f983c09be2c83ea" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:59.309710 11204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81596d1743546d89f983c09be2c83ea" candidate_uuid: "93009b2e2d7441f4a9d48b4d094b3711" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
I20260709 13:05:59.310695 11479 raft_consensus.cc:3060] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.310705 11204 raft_consensus.cc:3055] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:59.311303 11204 raft_consensus.cc:740] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:59.312278 11204 consensus_queue.cc:260] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:59.313694 11204 raft_consensus.cc:3060] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:59.318889 11479 raft_consensus.cc:2468] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 2.
I20260709 13:05:59.320366 11242 leader_election.cc:304] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353726f02e9c439792e790fb0958a80d, 93009b2e2d7441f4a9d48b4d094b3711; no voters: 
I20260709 13:05:59.321292 11597 raft_consensus.cc:2804] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:59.321172 11204 raft_consensus.cc:2468] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93009b2e2d7441f4a9d48b4d094b3711 in term 2.
I20260709 13:05:59.321973 11597 raft_consensus.cc:697] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Becoming Leader. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:59.323063 11597 consensus_queue.cc:237] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:59.331945 11026 catalog_manager.cc:5697] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 reported cstate change: term changed from 1 to 2, leader changed from bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194) to 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "93009b2e2d7441f4a9d48b4d094b3711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:59.356081 11597 raft_consensus.cc:993] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711: : Instructing follower bc100c804e924ea6b35a60fc507e1d6f to start an election
I20260709 13:05:59.356603 11557 raft_consensus.cc:1081] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Signalling peer bc100c804e924ea6b35a60fc507e1d6f to start an election
I20260709 13:05:59.358438 11204 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbf50a2fc30148e3b1b5b7e732cb4c1e"
dest_uuid: "bc100c804e924ea6b35a60fc507e1d6f"
 from {username='slave'} at 127.0.0.1:53778
I20260709 13:05:59.359073 11204 raft_consensus.cc:493] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:59.359355 11204 raft_consensus.cc:3060] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:59.364209 11204 raft_consensus.cc:515] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:59.365828 11204 leader_election.cc:290] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 3 election: Requested vote from peers 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527), 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195:40727)
I20260709 13:05:59.367089 11279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf50a2fc30148e3b1b5b7e732cb4c1e" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "93009b2e2d7441f4a9d48b4d094b3711"
I20260709 13:05:59.367156 11479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbf50a2fc30148e3b1b5b7e732cb4c1e" candidate_uuid: "bc100c804e924ea6b35a60fc507e1d6f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "353726f02e9c439792e790fb0958a80d"
I20260709 13:05:59.367689 11279 raft_consensus.cc:3055] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:59.367964 11479 raft_consensus.cc:3060] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:59.368083 11279 raft_consensus.cc:740] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 93009b2e2d7441f4a9d48b4d094b3711, State: Running, Role: LEADER
I20260709 13:05:59.368947 11279 consensus_queue.cc:260] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:59.369870 11279 raft_consensus.cc:3060] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:59.375263 11279 raft_consensus.cc:2468] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 3.
I20260709 13:05:59.376915 11169 leader_election.cc:304] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93009b2e2d7441f4a9d48b4d094b3711, bc100c804e924ea6b35a60fc507e1d6f; no voters: 
I20260709 13:05:59.377457 11479 raft_consensus.cc:2468] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc100c804e924ea6b35a60fc507e1d6f in term 3.
I20260709 13:05:59.377672 11562 raft_consensus.cc:2804] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:59.378317 11562 raft_consensus.cc:697] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 3 LEADER]: Becoming Leader. State: Replica: bc100c804e924ea6b35a60fc507e1d6f, State: Running, Role: LEADER
I20260709 13:05:59.379166 11562 consensus_queue.cc:237] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } }
I20260709 13:05:59.386812 11026 catalog_manager.cc:5697] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f reported cstate change: term changed from 2 to 3, leader changed from 93009b2e2d7441f4a9d48b4d094b3711 (127.0.107.195) to bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194). New cstate: current_term: 3 leader_uuid: "bc100c804e924ea6b35a60fc507e1d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:59.407932 11075 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:05:59.715175 11479 raft_consensus.cc:1275] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 5 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:05:59.716747 11557 consensus_queue.cc:1048] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:59.731299 11204 raft_consensus.cc:1275] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 5 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:05:59.735035 11557 consensus_queue.cc:1048] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:59.761045 11479 raft_consensus.cc:1275] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.763463 11597 consensus_queue.cc:1048] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:59.774494 11204 raft_consensus.cc:1275] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Refusing update from remote peer 93009b2e2d7441f4a9d48b4d094b3711: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:59.783687 11479 raft_consensus.cc:1275] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:59.784389 11611 consensus_queue.cc:1048] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc100c804e924ea6b35a60fc507e1d6f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 37675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:59.785848 11562 consensus_queue.cc:1048] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "353726f02e9c439792e790fb0958a80d" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:59.806082 11279 raft_consensus.cc:1275] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Refusing update from remote peer bc100c804e924ea6b35a60fc507e1d6f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:59.809191 11563 consensus_queue.cc:1048] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93009b2e2d7441f4a9d48b4d094b3711" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:05:59.838472 11242 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:33527, user_credentials={real_user=slave}} blocked reactor thread for 63580.9us
I20260709 13:05:59.947474   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:59.988415   431 tablet_server.cc:179] TabletServer@127.0.107.193:33527 shutting down...
I20260709 13:06:00.035216   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:00.036208   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:00.037240   431 raft_consensus.cc:2243] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.037869   431 raft_consensus.cc:2272] T d56f50814ef548078489152c9af6ee90 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.041143   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:00.041937   431 raft_consensus.cc:2243] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.042487   431 raft_consensus.cc:2272] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.046522   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:00.047384   431 raft_consensus.cc:2243] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.047983   431 raft_consensus.cc:2272] T 5323137cf0c94b33bbde48d8b7ce1e01 P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.051285   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:00.052146   431 raft_consensus.cc:2243] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.052915   431 raft_consensus.cc:2272] T d8ea549dae19466b9e30bde1c7b4721d P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.055805   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:00.056420   431 raft_consensus.cc:2243] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.056981   431 raft_consensus.cc:2272] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.059825   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:00.060467   431 raft_consensus.cc:2243] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.061021   431 raft_consensus.cc:2272] T e28402092b5d42a6a0dde934ed6d0154 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.063815   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:00.064451   431 raft_consensus.cc:2243] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.064898   431 raft_consensus.cc:2272] T ebae99d48338401f8a5c420bd5695aa5 P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.067325   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:00.067920   431 raft_consensus.cc:2243] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.068400   431 raft_consensus.cc:2272] T 35015db7e03c4af6a10a7a81b575dea9 P 353726f02e9c439792e790fb0958a80d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.071929   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:00.072597   431 raft_consensus.cc:2243] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.073192   431 raft_consensus.cc:2272] T 970302250f4e4b4cb92307cc423adceb P 353726f02e9c439792e790fb0958a80d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.075860   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:06:00.076515   431 raft_consensus.cc:2243] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.076982   431 raft_consensus.cc:2272] T a81596d1743546d89f983c09be2c83ea P 353726f02e9c439792e790fb0958a80d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:00.101086 11242 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111) [suppressed 93 similar messages]
W20260709 13:06:00.105288 11242 consensus_peers.cc:597] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:00.114517   431 tablet_server.cc:196] TabletServer@127.0.107.193:33527 shutdown complete.
W20260709 13:06:00.144412 11242 consensus_peers.cc:597] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:00.151505   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:06:00.188093 11243 consensus_peers.cc:597] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675): Couldn't send request to peer bc100c804e924ea6b35a60fc507e1d6f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:00.193440 11243 consensus_peers.cc:597] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675): Couldn't send request to peer bc100c804e924ea6b35a60fc507e1d6f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:00.197635 11242 consensus_peers.cc:597] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:00.203200 11168 consensus_peers.cc:597] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:00.205124   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:00.206130   431 tablet_replica.cc:333] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f: stopping tablet replica
I20260709 13:06:00.207304   431 raft_consensus.cc:2243] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:00.208647   431 raft_consensus.cc:2272] T 5323137cf0c94b33bbde48d8b7ce1e01 P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.212664   431 tablet_replica.cc:333] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f: stopping tablet replica
I20260709 13:06:00.213676   431 raft_consensus.cc:2243] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.214418   431 raft_consensus.cc:2272] T d8ea549dae19466b9e30bde1c7b4721d P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.217826   431 tablet_replica.cc:333] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f: stopping tablet replica
I20260709 13:06:00.218761   431 raft_consensus.cc:2243] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:00.220009   431 raft_consensus.cc:2272] T d56f50814ef548078489152c9af6ee90 P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.223608   431 tablet_replica.cc:333] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f: stopping tablet replica
I20260709 13:06:00.224503   431 raft_consensus.cc:2243] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.225092   431 raft_consensus.cc:2272] T 35015db7e03c4af6a10a7a81b575dea9 P bc100c804e924ea6b35a60fc507e1d6f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.228807   431 tablet_replica.cc:333] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f: stopping tablet replica
I20260709 13:06:00.229437   431 raft_consensus.cc:2243] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.229892   431 raft_consensus.cc:2272] T e28402092b5d42a6a0dde934ed6d0154 P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.232595   431 tablet_replica.cc:333] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f: stopping tablet replica
I20260709 13:06:00.233220   431 raft_consensus.cc:2243] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.233870   431 raft_consensus.cc:2272] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.236455   431 tablet_replica.cc:333] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f: stopping tablet replica
I20260709 13:06:00.237120   431 raft_consensus.cc:2243] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:00.237914   431 raft_consensus.cc:2272] T ebae99d48338401f8a5c420bd5695aa5 P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.240593   431 tablet_replica.cc:333] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f: stopping tablet replica
I20260709 13:06:00.241225   431 raft_consensus.cc:2243] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.241677   431 raft_consensus.cc:2272] T a81596d1743546d89f983c09be2c83ea P bc100c804e924ea6b35a60fc507e1d6f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.245224   431 tablet_replica.cc:333] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f: stopping tablet replica
I20260709 13:06:00.245896   431 raft_consensus.cc:2243] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.246491   431 raft_consensus.cc:2272] T 970302250f4e4b4cb92307cc423adceb P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.250123   431 tablet_replica.cc:333] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f: stopping tablet replica
I20260709 13:06:00.250854   431 raft_consensus.cc:2243] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:00.251849   431 raft_consensus.cc:2272] T bbf50a2fc30148e3b1b5b7e732cb4c1e P bc100c804e924ea6b35a60fc507e1d6f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.296064   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:06:00.315876 11242 consensus_peers.cc:597] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:00.323518 11243 consensus_peers.cc:597] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675): Couldn't send request to peer bc100c804e924ea6b35a60fc507e1d6f. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37675: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:00.345672   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:06:00.359344 11243 consensus_peers.cc:597] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675): Couldn't send request to peer bc100c804e924ea6b35a60fc507e1d6f. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37675: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:00.360107 11243 consensus_peers.cc:597] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer bc100c804e924ea6b35a60fc507e1d6f (127.0.107.194:37675): Couldn't send request to peer bc100c804e924ea6b35a60fc507e1d6f. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:37675: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:00.367996 11242 consensus_peers.cc:597] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 -> Peer 353726f02e9c439792e790fb0958a80d (127.0.107.193:33527): Couldn't send request to peer 353726f02e9c439792e790fb0958a80d. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:00.393504   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:00.394413   431 tablet_replica.cc:333] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711: stopping tablet replica
I20260709 13:06:00.395576   431 raft_consensus.cc:2243] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.396226   431 raft_consensus.cc:2272] T d56f50814ef548078489152c9af6ee90 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.398515   431 tablet_replica.cc:333] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711: stopping tablet replica
I20260709 13:06:00.399200   431 raft_consensus.cc:2243] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.399643   431 raft_consensus.cc:2272] T bbf50a2fc30148e3b1b5b7e732cb4c1e P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.402452   431 tablet_replica.cc:333] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711: stopping tablet replica
I20260709 13:06:00.403201   431 raft_consensus.cc:2243] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:00.404106   431 raft_consensus.cc:2272] T d8ea549dae19466b9e30bde1c7b4721d P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.406656   431 tablet_replica.cc:333] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711: stopping tablet replica
I20260709 13:06:00.407356   431 raft_consensus.cc:2243] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.407857   431 raft_consensus.cc:2272] T 5323137cf0c94b33bbde48d8b7ce1e01 P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:00.408937 11075 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:06:00.410580   431 tablet_replica.cc:333] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711: stopping tablet replica
I20260709 13:06:00.411278   431 raft_consensus.cc:2243] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:00.412128   431 raft_consensus.cc:2272] T a81596d1743546d89f983c09be2c83ea P 93009b2e2d7441f4a9d48b4d094b3711 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.415242   431 tablet_replica.cc:333] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711: stopping tablet replica
I20260709 13:06:00.415855   431 raft_consensus.cc:2243] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:00.416769   431 raft_consensus.cc:2272] T 970302250f4e4b4cb92307cc423adceb P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.419266   431 tablet_replica.cc:333] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711: stopping tablet replica
I20260709 13:06:00.419932   431 raft_consensus.cc:2243] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:00.420894   431 raft_consensus.cc:2272] T 5a6eb9aaa1494f99a0b3cb2a61935c2f P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.423517   431 tablet_replica.cc:333] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711: stopping tablet replica
I20260709 13:06:00.424103   431 raft_consensus.cc:2243] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:00.424849   431 raft_consensus.cc:2272] T e28402092b5d42a6a0dde934ed6d0154 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.427393   431 tablet_replica.cc:333] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711: stopping tablet replica
I20260709 13:06:00.428015   431 raft_consensus.cc:2243] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:00.428531   431 raft_consensus.cc:2272] T ebae99d48338401f8a5c420bd5695aa5 P 93009b2e2d7441f4a9d48b4d094b3711 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.430564   431 tablet_replica.cc:333] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711: stopping tablet replica
I20260709 13:06:00.431197   431 raft_consensus.cc:2243] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:06:00.431978   431 raft_consensus.cc:2272] T 35015db7e03c4af6a10a7a81b575dea9 P 93009b2e2d7441f4a9d48b4d094b3711 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.467893   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:06:00.495573   431 master.cc:562] Master@127.0.107.254:39169 shutting down...
I20260709 13:06:00.523932   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:00.524514   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:00.524838   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P d8dfab7811414a58a2bda1e4ab50887a: stopping tablet replica
I20260709 13:06:00.567086   431 master.cc:584] Master@127.0.107.254:39169 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13405 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:06:00.616822   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:39099
I20260709 13:06:00.618207   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:00.624861 11621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:00.624948 11620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:00.626250 11623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:00.628274   431 server_base.cc:1061] running on GCE node
I20260709 13:06:00.629618   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:00.629832   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:00.629944   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602360629934 us; error 0 us; skew 500 ppm
I20260709 13:06:00.630414   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:00.633473   431 webserver.cc:533] Webserver started at http://127.0.107.254:43191/ using document root <none> and password file <none>
I20260709 13:06:00.634019   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:00.634184   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:00.634438   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:00.635757   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/master-0-root/instance:
uuid: "7bba317fff1648debe515449d96b3233"
format_stamp: "Formatted at 2026-07-09 13:06:00 on dist-test-slave-cq4t"
I20260709 13:06:00.640722   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:06:00.644712 11628 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:00.645540   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:06:00.645901   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/master-0-root
uuid: "7bba317fff1648debe515449d96b3233"
format_stamp: "Formatted at 2026-07-09 13:06:00 on dist-test-slave-cq4t"
I20260709 13:06:00.646207   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:00.657020   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:00.658303   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:00.700721   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:39099
I20260709 13:06:00.700843 11679 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:39099 every 8 connection(s)
I20260709 13:06:00.704748 11680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:00.716660 11680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233: Bootstrap starting.
I20260709 13:06:00.721624 11680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:00.726524 11680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233: No bootstrap required, opened a new log
I20260709 13:06:00.728818 11680 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bba317fff1648debe515449d96b3233" member_type: VOTER }
I20260709 13:06:00.729216 11680 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:00.729506 11680 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bba317fff1648debe515449d96b3233, State: Initialized, Role: FOLLOWER
I20260709 13:06:00.730167 11680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bba317fff1648debe515449d96b3233" member_type: VOTER }
I20260709 13:06:00.730597 11680 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:00.730785 11680 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:00.731020 11680 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:00.736176 11680 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bba317fff1648debe515449d96b3233" member_type: VOTER }
I20260709 13:06:00.736758 11680 leader_election.cc:304] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bba317fff1648debe515449d96b3233; no voters: 
I20260709 13:06:00.737977 11680 leader_election.cc:290] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:00.738361 11683 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:00.739841 11683 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [term 1 LEADER]: Becoming Leader. State: Replica: 7bba317fff1648debe515449d96b3233, State: Running, Role: LEADER
I20260709 13:06:00.740658 11683 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bba317fff1648debe515449d96b3233" member_type: VOTER }
I20260709 13:06:00.741418 11680 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:00.745669 11684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bba317fff1648debe515449d96b3233" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bba317fff1648debe515449d96b3233" member_type: VOTER } }
I20260709 13:06:00.746481 11684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:00.748690 11689 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:00.748535 11685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bba317fff1648debe515449d96b3233. Latest consensus state: current_term: 1 leader_uuid: "7bba317fff1648debe515449d96b3233" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bba317fff1648debe515449d96b3233" member_type: VOTER } }
I20260709 13:06:00.749258 11685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:00.754458 11689 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:00.762559   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:00.765802 11689 catalog_manager.cc:1383] Generated new cluster ID: 7a1b95ff2ec54ebd8bd30deb4e77fa44
I20260709 13:06:00.766201 11689 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:00.785566 11689 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:00.786984 11689 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:00.807185 11689 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233: Generated new TSK 0
I20260709 13:06:00.807855 11689 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:00.829643   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:00.836198 11701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:00.837410 11702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:00.839490 11704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:00.839946   431 server_base.cc:1061] running on GCE node
I20260709 13:06:00.840945   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:00.841138   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:00.841259   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602360841248 us; error 0 us; skew 500 ppm
I20260709 13:06:00.841763   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:00.844579   431 webserver.cc:533] Webserver started at http://127.0.107.193:35997/ using document root <none> and password file <none>
I20260709 13:06:00.845055   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:00.845221   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:00.845434   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:00.846549   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-0-root/instance:
uuid: "349cc403b3024572a5f1a322ca85ed24"
format_stamp: "Formatted at 2026-07-09 13:06:00 on dist-test-slave-cq4t"
I20260709 13:06:00.851573   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.003s
I20260709 13:06:00.855294 11709 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:00.856314   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:06:00.856592   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-0-root
uuid: "349cc403b3024572a5f1a322ca85ed24"
format_stamp: "Formatted at 2026-07-09 13:06:00 on dist-test-slave-cq4t"
I20260709 13:06:00.856875   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:00.877933   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:00.879276   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:00.880947   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:00.883615   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:00.883827   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:00.884166   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:00.884343   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:00.933101   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:45555
I20260709 13:06:00.933243 11771 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:45555 every 8 connection(s)
I20260709 13:06:00.938757   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:00.947098 11776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:00.947238 11777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:00.952612   431 server_base.cc:1061] running on GCE node
W20260709 13:06:00.953156 11779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:00.953894 11772 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39099
I20260709 13:06:00.954408 11772 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:00.954463   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:00.954886   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:00.955135   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602360955115 us; error 0 us; skew 500 ppm
I20260709 13:06:00.955454 11772 heartbeater.cc:507] Master 127.0.107.254:39099 requested a full tablet report, sending...
I20260709 13:06:00.955927   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:00.958194 11645 ts_manager.cc:194] Registered new tserver with Master: 349cc403b3024572a5f1a322ca85ed24 (127.0.107.193:45555)
I20260709 13:06:00.959427   431 webserver.cc:533] Webserver started at http://127.0.107.194:33027/ using document root <none> and password file <none>
I20260709 13:06:00.960186   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:00.960426   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:00.960721   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:00.961052 11645 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50870
I20260709 13:06:00.962342   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-1-root/instance:
uuid: "af15c44e874c442aab0a32438663b35f"
format_stamp: "Formatted at 2026-07-09 13:06:00 on dist-test-slave-cq4t"
I20260709 13:06:00.967392   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:06:00.971264 11784 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:00.972301   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:00.972719   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-1-root
uuid: "af15c44e874c442aab0a32438663b35f"
format_stamp: "Formatted at 2026-07-09 13:06:00 on dist-test-slave-cq4t"
I20260709 13:06:00.973034   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:00.989570   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:00.991003   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:00.992726   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:00.995370   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:00.995579   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:00.995900   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:00.996093   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:01.044605   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:39131
I20260709 13:06:01.044942 11846 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:39131 every 8 connection(s)
I20260709 13:06:01.049552   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:01.056807 11850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:01.057796 11851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:01.060006   431 server_base.cc:1061] running on GCE node
W20260709 13:06:01.061007 11854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:01.062250   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:01.062588   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:01.062824   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602361062781 us; error 0 us; skew 500 ppm
I20260709 13:06:01.062880 11847 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39099
I20260709 13:06:01.063406 11847 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:01.063542   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:01.064216 11847 heartbeater.cc:507] Master 127.0.107.254:39099 requested a full tablet report, sending...
I20260709 13:06:01.066555   431 webserver.cc:533] Webserver started at http://127.0.107.195:32837/ using document root <none> and password file <none>
I20260709 13:06:01.066854 11645 ts_manager.cc:194] Registered new tserver with Master: af15c44e874c442aab0a32438663b35f (127.0.107.194:39131)
I20260709 13:06:01.067325   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:01.067628   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:01.067950   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:01.068884 11645 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50872
I20260709 13:06:01.069620   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-2-root/instance:
uuid: "800ee909a7ad492eb71c4cccb4e24e2f"
format_stamp: "Formatted at 2026-07-09 13:06:01 on dist-test-slave-cq4t"
I20260709 13:06:01.074579   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:06:01.078408 11858 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:01.079435   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:01.079752   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-2-root
uuid: "800ee909a7ad492eb71c4cccb4e24e2f"
format_stamp: "Formatted at 2026-07-09 13:06:01 on dist-test-slave-cq4t"
I20260709 13:06:01.080072   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602014632082-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:01.112298   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:01.113739   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:01.115586   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:01.118036   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:01.118295   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:01.118571   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:01.118753   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:01.166833   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:39013
I20260709 13:06:01.166997 11920 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:39013 every 8 connection(s)
I20260709 13:06:01.182855 11921 heartbeater.cc:344] Connected to a master server at 127.0.107.254:39099
I20260709 13:06:01.183427 11921 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:01.184194 11921 heartbeater.cc:507] Master 127.0.107.254:39099 requested a full tablet report, sending...
I20260709 13:06:01.186247 11645 ts_manager.cc:194] Registered new tserver with Master: 800ee909a7ad492eb71c4cccb4e24e2f (127.0.107.195:39013)
I20260709 13:06:01.187059   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.0162605s
I20260709 13:06:01.187903 11645 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50878
I20260709 13:06:01.216213 11645 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50882:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:01.218475 11645 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:01.271620 11886 tablet_service.cc:1511] Processing CreateTablet for tablet fa5453391722496dba2413e300b7658e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=101075c38bd34e818e428ddb8bac9d64]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:01.273392 11886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa5453391722496dba2413e300b7658e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.277505 11885 tablet_service.cc:1511] Processing CreateTablet for tablet 64126e3ded0a41b5b8626e7b49fdeed2 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=101075c38bd34e818e428ddb8bac9d64]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:01.279242 11885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64126e3ded0a41b5b8626e7b49fdeed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.285817 11737 tablet_service.cc:1511] Processing CreateTablet for tablet fa5453391722496dba2413e300b7658e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=101075c38bd34e818e428ddb8bac9d64]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:01.287483 11737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa5453391722496dba2413e300b7658e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.300216 11812 tablet_service.cc:1511] Processing CreateTablet for tablet fa5453391722496dba2413e300b7658e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=101075c38bd34e818e428ddb8bac9d64]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:01.301550 11736 tablet_service.cc:1511] Processing CreateTablet for tablet 64126e3ded0a41b5b8626e7b49fdeed2 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=101075c38bd34e818e428ddb8bac9d64]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:01.302088 11812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa5453391722496dba2413e300b7658e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.303090 11736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64126e3ded0a41b5b8626e7b49fdeed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.308187 11811 tablet_service.cc:1511] Processing CreateTablet for tablet 64126e3ded0a41b5b8626e7b49fdeed2 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=101075c38bd34e818e428ddb8bac9d64]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:01.310249 11811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64126e3ded0a41b5b8626e7b49fdeed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.328274 11936 tablet_bootstrap.cc:492] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f: Bootstrap starting.
I20260709 13:06:01.340252 11937 tablet_bootstrap.cc:492] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24: Bootstrap starting.
I20260709 13:06:01.343152 11936 tablet_bootstrap.cc:654] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.344105 11938 tablet_bootstrap.cc:492] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f: Bootstrap starting.
I20260709 13:06:01.350152 11938 tablet_bootstrap.cc:654] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.352181 11937 tablet_bootstrap.cc:654] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.353420 11936 tablet_bootstrap.cc:492] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f: No bootstrap required, opened a new log
I20260709 13:06:01.353994 11936 ts_tablet_manager.cc:1403] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.008s
I20260709 13:06:01.358336 11936 raft_consensus.cc:359] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.359194 11936 raft_consensus.cc:385] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.359495 11937 tablet_bootstrap.cc:492] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24: No bootstrap required, opened a new log
I20260709 13:06:01.359529 11936 raft_consensus.cc:740] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 800ee909a7ad492eb71c4cccb4e24e2f, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.360190 11937 ts_tablet_manager.cc:1403] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.000s
I20260709 13:06:01.360608 11936 consensus_queue.cc:260] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.361289 11938 tablet_bootstrap.cc:492] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f: No bootstrap required, opened a new log
I20260709 13:06:01.361845 11938 ts_tablet_manager.cc:1403] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f: Time spent bootstrapping tablet: real 0.018s	user 0.005s	sys 0.009s
I20260709 13:06:01.363307 11921 heartbeater.cc:499] Master 127.0.107.254:39099 was elected leader, sending a full tablet report...
I20260709 13:06:01.363783 11936 ts_tablet_manager.cc:1434] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 13:06:01.364813 11936 tablet_bootstrap.cc:492] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f: Bootstrap starting.
I20260709 13:06:01.364679 11938 raft_consensus.cc:359] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } } peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } }
I20260709 13:06:01.365468 11938 raft_consensus.cc:385] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.365854 11938 raft_consensus.cc:740] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af15c44e874c442aab0a32438663b35f, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.366027 11937 raft_consensus.cc:359] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } } peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } }
I20260709 13:06:01.366982 11937 raft_consensus.cc:385] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.366909 11938 consensus_queue.cc:260] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } } peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } }
I20260709 13:06:01.367550 11937 raft_consensus.cc:740] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349cc403b3024572a5f1a322ca85ed24, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.368732 11937 consensus_queue.cc:260] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } } peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } }
I20260709 13:06:01.373596 11936 tablet_bootstrap.cc:654] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.375849 11938 ts_tablet_manager.cc:1434] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.006s
I20260709 13:06:01.376958 11847 heartbeater.cc:499] Master 127.0.107.254:39099 was elected leader, sending a full tablet report...
I20260709 13:06:01.378376 11772 heartbeater.cc:499] Master 127.0.107.254:39099 was elected leader, sending a full tablet report...
I20260709 13:06:01.377103 11938 tablet_bootstrap.cc:492] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f: Bootstrap starting.
I20260709 13:06:01.385943 11938 tablet_bootstrap.cc:654] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.389253 11937 ts_tablet_manager.cc:1434] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.005s
I20260709 13:06:01.390973 11937 tablet_bootstrap.cc:492] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24: Bootstrap starting.
I20260709 13:06:01.400578 11937 tablet_bootstrap.cc:654] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.420320 11938 tablet_bootstrap.cc:492] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f: No bootstrap required, opened a new log
I20260709 13:06:01.420472 11936 tablet_bootstrap.cc:492] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f: No bootstrap required, opened a new log
I20260709 13:06:01.420874 11938 ts_tablet_manager.cc:1403] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f: Time spent bootstrapping tablet: real 0.044s	user 0.010s	sys 0.028s
I20260709 13:06:01.421100 11936 ts_tablet_manager.cc:1403] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f: Time spent bootstrapping tablet: real 0.057s	user 0.021s	sys 0.032s
I20260709 13:06:01.421324 11937 tablet_bootstrap.cc:492] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24: No bootstrap required, opened a new log
I20260709 13:06:01.421886 11937 ts_tablet_manager.cc:1403] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.008s
I20260709 13:06:01.423851 11938 raft_consensus.cc:359] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.424535 11938 raft_consensus.cc:385] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.424450 11936 raft_consensus.cc:359] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } } peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } }
I20260709 13:06:01.425028 11938 raft_consensus.cc:740] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af15c44e874c442aab0a32438663b35f, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.425032 11937 raft_consensus.cc:359] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.425549 11936 raft_consensus.cc:385] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.425875 11937 raft_consensus.cc:385] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.425907 11936 raft_consensus.cc:740] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 800ee909a7ad492eb71c4cccb4e24e2f, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.426373 11937 raft_consensus.cc:740] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349cc403b3024572a5f1a322ca85ed24, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.426183 11938 consensus_queue.cc:260] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.426855 11936 consensus_queue.cc:260] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } } peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } }
I20260709 13:06:01.427464 11937 consensus_queue.cc:260] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.429215 11936 ts_tablet_manager.cc:1434] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 13:06:01.429723 11937 ts_tablet_manager.cc:1434] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:06:01.431918 11938 ts_tablet_manager.cc:1434] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:06:01.487718 11943 raft_consensus.cc:493] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.488409 11943 raft_consensus.cc:515] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } } peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } }
I20260709 13:06:01.491142 11943 leader_election.cc:290] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 800ee909a7ad492eb71c4cccb4e24e2f (127.0.107.195:39013), 349cc403b3024572a5f1a322ca85ed24 (127.0.107.193:45555)
I20260709 13:06:01.501395 11896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5453391722496dba2413e300b7658e" candidate_uuid: "af15c44e874c442aab0a32438663b35f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" is_pre_election: true
I20260709 13:06:01.502239 11896 raft_consensus.cc:2468] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af15c44e874c442aab0a32438663b35f in term 0.
I20260709 13:06:01.503464 11786 leader_election.cc:304] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 800ee909a7ad492eb71c4cccb4e24e2f, af15c44e874c442aab0a32438663b35f; no voters: 
I20260709 13:06:01.504091 11747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5453391722496dba2413e300b7658e" candidate_uuid: "af15c44e874c442aab0a32438663b35f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349cc403b3024572a5f1a322ca85ed24" is_pre_election: true
I20260709 13:06:01.504352 11943 raft_consensus.cc:2804] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:01.504839 11747 raft_consensus.cc:2468] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af15c44e874c442aab0a32438663b35f in term 0.
I20260709 13:06:01.504913 11943 raft_consensus.cc:493] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.505363 11943 raft_consensus.cc:3060] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:01.511348 11943 raft_consensus.cc:515] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } } peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } }
I20260709 13:06:01.513491 11943 leader_election.cc:290] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [CANDIDATE]: Term 1 election: Requested vote from peers 800ee909a7ad492eb71c4cccb4e24e2f (127.0.107.195:39013), 349cc403b3024572a5f1a322ca85ed24 (127.0.107.193:45555)
I20260709 13:06:01.514653 11747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5453391722496dba2413e300b7658e" candidate_uuid: "af15c44e874c442aab0a32438663b35f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349cc403b3024572a5f1a322ca85ed24"
I20260709 13:06:01.514559 11896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5453391722496dba2413e300b7658e" candidate_uuid: "af15c44e874c442aab0a32438663b35f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "800ee909a7ad492eb71c4cccb4e24e2f"
I20260709 13:06:01.515484 11747 raft_consensus.cc:3060] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:01.515565 11896 raft_consensus.cc:3060] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:01.521724 11896 raft_consensus.cc:2468] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af15c44e874c442aab0a32438663b35f in term 1.
I20260709 13:06:01.521710 11747 raft_consensus.cc:2468] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af15c44e874c442aab0a32438663b35f in term 1.
I20260709 13:06:01.523106 11786 leader_election.cc:304] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 800ee909a7ad492eb71c4cccb4e24e2f, af15c44e874c442aab0a32438663b35f; no voters: 
I20260709 13:06:01.524219 11943 raft_consensus.cc:2804] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:01.525279 11943 raft_consensus.cc:697] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 1 LEADER]: Becoming Leader. State: Replica: af15c44e874c442aab0a32438663b35f, State: Running, Role: LEADER
I20260709 13:06:01.526070 11943 consensus_queue.cc:237] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } } peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } }
I20260709 13:06:01.533792 11644 catalog_manager.cc:5697] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f reported cstate change: term changed from 0 to 1, leader changed from <none> to af15c44e874c442aab0a32438663b35f (127.0.107.194). New cstate: current_term: 1 leader_uuid: "af15c44e874c442aab0a32438663b35f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:01.603318 11944 raft_consensus.cc:493] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.603845 11944 raft_consensus.cc:515] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.606019 11944 leader_election.cc:290] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af15c44e874c442aab0a32438663b35f (127.0.107.194:39131), 800ee909a7ad492eb71c4cccb4e24e2f (127.0.107.195:39013)
I20260709 13:06:01.617622 11896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64126e3ded0a41b5b8626e7b49fdeed2" candidate_uuid: "349cc403b3024572a5f1a322ca85ed24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" is_pre_election: true
I20260709 13:06:01.617722 11822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64126e3ded0a41b5b8626e7b49fdeed2" candidate_uuid: "349cc403b3024572a5f1a322ca85ed24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af15c44e874c442aab0a32438663b35f" is_pre_election: true
I20260709 13:06:01.618464 11896 raft_consensus.cc:2468] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349cc403b3024572a5f1a322ca85ed24 in term 0.
I20260709 13:06:01.618507 11822 raft_consensus.cc:2468] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349cc403b3024572a5f1a322ca85ed24 in term 0.
I20260709 13:06:01.619692 11711 leader_election.cc:304] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349cc403b3024572a5f1a322ca85ed24, 800ee909a7ad492eb71c4cccb4e24e2f; no voters: 
I20260709 13:06:01.620484 11944 raft_consensus.cc:2804] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:01.620841 11944 raft_consensus.cc:493] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.621258 11944 raft_consensus.cc:3060] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:01.626952 11944 raft_consensus.cc:515] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.628895 11944 leader_election.cc:290] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [CANDIDATE]: Term 1 election: Requested vote from peers af15c44e874c442aab0a32438663b35f (127.0.107.194:39131), 800ee909a7ad492eb71c4cccb4e24e2f (127.0.107.195:39013)
I20260709 13:06:01.629787 11822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64126e3ded0a41b5b8626e7b49fdeed2" candidate_uuid: "349cc403b3024572a5f1a322ca85ed24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af15c44e874c442aab0a32438663b35f"
I20260709 13:06:01.630103 11896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64126e3ded0a41b5b8626e7b49fdeed2" candidate_uuid: "349cc403b3024572a5f1a322ca85ed24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "800ee909a7ad492eb71c4cccb4e24e2f"
I20260709 13:06:01.630509 11822 raft_consensus.cc:3060] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:01.630658 11896 raft_consensus.cc:3060] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:01.638273 11822 raft_consensus.cc:2468] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349cc403b3024572a5f1a322ca85ed24 in term 1.
I20260709 13:06:01.638273 11896 raft_consensus.cc:2468] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349cc403b3024572a5f1a322ca85ed24 in term 1.
I20260709 13:06:01.639809 11713 leader_election.cc:304] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349cc403b3024572a5f1a322ca85ed24, af15c44e874c442aab0a32438663b35f; no voters: 
I20260709 13:06:01.640614 11944 raft_consensus.cc:2804] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:01.641710 11944 raft_consensus.cc:697] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 1 LEADER]: Becoming Leader. State: Replica: 349cc403b3024572a5f1a322ca85ed24, State: Running, Role: LEADER
I20260709 13:06:01.642515 11944 consensus_queue.cc:237] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.649518 11644 catalog_manager.cc:5697] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 349cc403b3024572a5f1a322ca85ed24 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "349cc403b3024572a5f1a322ca85ed24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:01.773975 11644 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50886:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:01.776324 11644 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:01.804191 11736 tablet_service.cc:1511] Processing CreateTablet for tablet 640a7be311ef41d69cf98cf7505507bb (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=717b8a78243f4a268e322b0de4297d9c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:01.805583 11736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 640a7be311ef41d69cf98cf7505507bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.805712 11811 tablet_service.cc:1511] Processing CreateTablet for tablet 640a7be311ef41d69cf98cf7505507bb (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=717b8a78243f4a268e322b0de4297d9c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:01.807216 11811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 640a7be311ef41d69cf98cf7505507bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.807292 11886 tablet_service.cc:1511] Processing CreateTablet for tablet 640a7be311ef41d69cf98cf7505507bb (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=717b8a78243f4a268e322b0de4297d9c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:06:01.808625 11886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 640a7be311ef41d69cf98cf7505507bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.811146 11812 tablet_service.cc:1511] Processing CreateTablet for tablet bd2ecbfc736d493599e915d2b97f5583 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=717b8a78243f4a268e322b0de4297d9c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:01.812628 11812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2ecbfc736d493599e915d2b97f5583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.816677 11737 tablet_service.cc:1511] Processing CreateTablet for tablet bd2ecbfc736d493599e915d2b97f5583 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=717b8a78243f4a268e322b0de4297d9c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:01.818085 11737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2ecbfc736d493599e915d2b97f5583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.824551 11937 tablet_bootstrap.cc:492] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24: Bootstrap starting.
I20260709 13:06:01.825053 11885 tablet_service.cc:1511] Processing CreateTablet for tablet bd2ecbfc736d493599e915d2b97f5583 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=717b8a78243f4a268e322b0de4297d9c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:06:01.826496 11885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2ecbfc736d493599e915d2b97f5583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.832293 11938 tablet_bootstrap.cc:492] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f: Bootstrap starting.
I20260709 13:06:01.837010 11937 tablet_bootstrap.cc:654] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.840842 11938 tablet_bootstrap.cc:654] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.856864 11936 tablet_bootstrap.cc:492] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f: Bootstrap starting.
I20260709 13:06:01.860899 11938 tablet_bootstrap.cc:492] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f: No bootstrap required, opened a new log
I20260709 13:06:01.861059 11937 tablet_bootstrap.cc:492] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24: No bootstrap required, opened a new log
I20260709 13:06:01.861756 11938 ts_tablet_manager.cc:1403] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.014s
I20260709 13:06:01.861908 11937 ts_tablet_manager.cc:1403] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.017s
I20260709 13:06:01.865075 11936 tablet_bootstrap.cc:654] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.865322 11938 raft_consensus.cc:359] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.866372 11938 raft_consensus.cc:385] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.866822 11938 raft_consensus.cc:740] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af15c44e874c442aab0a32438663b35f, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.866655 11937 raft_consensus.cc:359] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.867841 11937 raft_consensus.cc:385] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.868253 11937 raft_consensus.cc:740] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349cc403b3024572a5f1a322ca85ed24, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.867853 11938 consensus_queue.cc:260] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.869158 11937 consensus_queue.cc:260] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.871204 11938 ts_tablet_manager.cc:1434] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:06:01.871860 11937 ts_tablet_manager.cc:1434] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:06:01.872539 11938 tablet_bootstrap.cc:492] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f: Bootstrap starting.
I20260709 13:06:01.873183 11937 tablet_bootstrap.cc:492] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24: Bootstrap starting.
I20260709 13:06:01.879997 11937 tablet_bootstrap.cc:654] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.880872 11938 tablet_bootstrap.cc:654] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.885694 11936 tablet_bootstrap.cc:492] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f: No bootstrap required, opened a new log
I20260709 13:06:01.886629 11936 ts_tablet_manager.cc:1403] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.001s
I20260709 13:06:01.894659 11936 raft_consensus.cc:359] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.896128 11936 raft_consensus.cc:385] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.896682 11936 raft_consensus.cc:740] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 800ee909a7ad492eb71c4cccb4e24e2f, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.897946 11936 consensus_queue.cc:260] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.901140 11936 ts_tablet_manager.cc:1434] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:06:01.902441 11936 tablet_bootstrap.cc:492] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f: Bootstrap starting.
I20260709 13:06:01.907418 11937 tablet_bootstrap.cc:492] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24: No bootstrap required, opened a new log
I20260709 13:06:01.907431 11938 tablet_bootstrap.cc:492] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f: No bootstrap required, opened a new log
I20260709 13:06:01.908396 11937 ts_tablet_manager.cc:1403] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.013s
I20260709 13:06:01.908421 11938 ts_tablet_manager.cc:1403] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.015s
I20260709 13:06:01.912176 11937 raft_consensus.cc:359] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.912420 11938 raft_consensus.cc:359] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.913278 11937 raft_consensus.cc:385] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.913338 11938 raft_consensus.cc:385] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.913900 11937 raft_consensus.cc:740] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349cc403b3024572a5f1a322ca85ed24, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.913995 11938 raft_consensus.cc:740] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af15c44e874c442aab0a32438663b35f, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.914427 11936 tablet_bootstrap.cc:654] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:01.915232 11937 consensus_queue.cc:260] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.915110 11938 consensus_queue.cc:260] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.918869 11937 ts_tablet_manager.cc:1434] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:06:01.920921 11938 ts_tablet_manager.cc:1434] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:06:01.923136 11936 tablet_bootstrap.cc:492] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f: No bootstrap required, opened a new log
I20260709 13:06:01.923836 11936 ts_tablet_manager.cc:1403] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.004s
I20260709 13:06:01.927353 11936 raft_consensus.cc:359] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.928293 11936 raft_consensus.cc:385] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:01.928692 11936 raft_consensus.cc:740] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 800ee909a7ad492eb71c4cccb4e24e2f, State: Initialized, Role: FOLLOWER
I20260709 13:06:01.929636 11936 consensus_queue.cc:260] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.932384 11936 ts_tablet_manager.cc:1434] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:01.973855 11943 consensus_queue.cc:1048] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [LEADER]: Connected to new peer: Peer: permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:01.986676 11949 consensus_queue.cc:1048] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [LEADER]: Connected to new peer: Peer: permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:01.988190 11942 raft_consensus.cc:493] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.988920 11942 raft_consensus.cc:515] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:01.992435 11942 leader_election.cc:290] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 349cc403b3024572a5f1a322ca85ed24 (127.0.107.193:45555), af15c44e874c442aab0a32438663b35f (127.0.107.194:39131)
I20260709 13:06:02.019104 11747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2ecbfc736d493599e915d2b97f5583" candidate_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349cc403b3024572a5f1a322ca85ed24" is_pre_election: true
I20260709 13:06:02.020382 11747 raft_consensus.cc:2468] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 800ee909a7ad492eb71c4cccb4e24e2f in term 0.
I20260709 13:06:02.021867 11862 leader_election.cc:304] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349cc403b3024572a5f1a322ca85ed24, 800ee909a7ad492eb71c4cccb4e24e2f; no voters: 
I20260709 13:06:02.023455 11942 raft_consensus.cc:2804] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.023869 11942 raft_consensus.cc:493] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.024245 11942 raft_consensus.cc:3060] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.025403 11822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2ecbfc736d493599e915d2b97f5583" candidate_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af15c44e874c442aab0a32438663b35f" is_pre_election: true
I20260709 13:06:02.026103 11822 raft_consensus.cc:2468] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 800ee909a7ad492eb71c4cccb4e24e2f in term 0.
I20260709 13:06:02.030778 11944 raft_consensus.cc:493] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.031492 11944 raft_consensus.cc:515] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:02.032459 11942 raft_consensus.cc:515] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:02.033993 11944 leader_election.cc:290] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af15c44e874c442aab0a32438663b35f (127.0.107.194:39131), 800ee909a7ad492eb71c4cccb4e24e2f (127.0.107.195:39013)
I20260709 13:06:02.034626 11942 leader_election.cc:290] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [CANDIDATE]: Term 1 election: Requested vote from peers 349cc403b3024572a5f1a322ca85ed24 (127.0.107.193:45555), af15c44e874c442aab0a32438663b35f (127.0.107.194:39131)
I20260709 13:06:02.035113 11822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640a7be311ef41d69cf98cf7505507bb" candidate_uuid: "349cc403b3024572a5f1a322ca85ed24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af15c44e874c442aab0a32438663b35f" is_pre_election: true
I20260709 13:06:02.035531 11747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2ecbfc736d493599e915d2b97f5583" candidate_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349cc403b3024572a5f1a322ca85ed24"
I20260709 13:06:02.035864 11822 raft_consensus.cc:2468] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349cc403b3024572a5f1a322ca85ed24 in term 0.
I20260709 13:06:02.036334 11747 raft_consensus.cc:3060] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.036616 11821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2ecbfc736d493599e915d2b97f5583" candidate_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af15c44e874c442aab0a32438663b35f"
I20260709 13:06:02.037285 11821 raft_consensus.cc:3060] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.037348 11713 leader_election.cc:304] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349cc403b3024572a5f1a322ca85ed24, af15c44e874c442aab0a32438663b35f; no voters: 
I20260709 13:06:02.038317 11944 raft_consensus.cc:2804] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.038596 11944 raft_consensus.cc:493] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.035531 11896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640a7be311ef41d69cf98cf7505507bb" candidate_uuid: "349cc403b3024572a5f1a322ca85ed24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" is_pre_election: true
I20260709 13:06:02.038820 11944 raft_consensus.cc:3060] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.039577 11896 raft_consensus.cc:2468] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349cc403b3024572a5f1a322ca85ed24 in term 0.
I20260709 13:06:02.044569 11747 raft_consensus.cc:2468] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 800ee909a7ad492eb71c4cccb4e24e2f in term 1.
I20260709 13:06:02.045329 11821 raft_consensus.cc:2468] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 800ee909a7ad492eb71c4cccb4e24e2f in term 1.
I20260709 13:06:02.045233 11944 raft_consensus.cc:515] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:02.046281 11862 leader_election.cc:304] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349cc403b3024572a5f1a322ca85ed24, 800ee909a7ad492eb71c4cccb4e24e2f; no voters: 
I20260709 13:06:02.047163 11942 raft_consensus.cc:2804] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.047828 11942 raft_consensus.cc:697] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 LEADER]: Becoming Leader. State: Replica: 800ee909a7ad492eb71c4cccb4e24e2f, State: Running, Role: LEADER
I20260709 13:06:02.048964 11942 consensus_queue.cc:237] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:02.049547 11944 leader_election.cc:290] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [CANDIDATE]: Term 1 election: Requested vote from peers af15c44e874c442aab0a32438663b35f (127.0.107.194:39131), 800ee909a7ad492eb71c4cccb4e24e2f (127.0.107.195:39013)
I20260709 13:06:02.050287 11821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640a7be311ef41d69cf98cf7505507bb" candidate_uuid: "349cc403b3024572a5f1a322ca85ed24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af15c44e874c442aab0a32438663b35f"
I20260709 13:06:02.050544 11896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "640a7be311ef41d69cf98cf7505507bb" candidate_uuid: "349cc403b3024572a5f1a322ca85ed24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "800ee909a7ad492eb71c4cccb4e24e2f"
I20260709 13:06:02.051318 11821 raft_consensus.cc:3060] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.051402 11896 raft_consensus.cc:3060] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.058089 11821 raft_consensus.cc:2468] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349cc403b3024572a5f1a322ca85ed24 in term 1.
I20260709 13:06:02.059253 11713 leader_election.cc:304] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349cc403b3024572a5f1a322ca85ed24, af15c44e874c442aab0a32438663b35f; no voters: 
I20260709 13:06:02.060001 11944 raft_consensus.cc:2804] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.060509 11944 raft_consensus.cc:697] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [term 1 LEADER]: Becoming Leader. State: Replica: 349cc403b3024572a5f1a322ca85ed24, State: Running, Role: LEADER
I20260709 13:06:02.060119 11896 raft_consensus.cc:2468] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349cc403b3024572a5f1a322ca85ed24 in term 1.
I20260709 13:06:02.060973 11644 catalog_manager.cc:5697] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 800ee909a7ad492eb71c4cccb4e24e2f (127.0.107.195). New cstate: current_term: 1 leader_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:02.061406 11944 consensus_queue.cc:237] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:02.066747 11953 consensus_queue.cc:1048] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:02.068248 11978 consensus_queue.cc:1048] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:02.074419 11643 catalog_manager.cc:5697] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 349cc403b3024572a5f1a322ca85ed24 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "349cc403b3024572a5f1a322ca85ed24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:02.111249 11821 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa5453391722496dba2413e300b7658e"
dest_uuid: "af15c44e874c442aab0a32438663b35f"
mode: GRACEFUL
new_leader_uuid: "349cc403b3024572a5f1a322ca85ed24"
 from {username='slave'} at 127.0.0.1:46736
I20260709 13:06:02.111866 11821 raft_consensus.cc:606] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 1 LEADER]: Received request to transfer leadership to TS 349cc403b3024572a5f1a322ca85ed24
I20260709 13:06:02.126849 11747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64126e3ded0a41b5b8626e7b49fdeed2"
dest_uuid: "349cc403b3024572a5f1a322ca85ed24"
mode: GRACEFUL
new_leader_uuid: "800ee909a7ad492eb71c4cccb4e24e2f"
 from {username='slave'} at 127.0.0.1:57044
I20260709 13:06:02.127504 11747 raft_consensus.cc:606] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 1 LEADER]: Received request to transfer leadership to TS 800ee909a7ad492eb71c4cccb4e24e2f
I20260709 13:06:02.147785 11896 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd2ecbfc736d493599e915d2b97f5583"
dest_uuid: "800ee909a7ad492eb71c4cccb4e24e2f"
mode: GRACEFUL
new_leader_uuid: "349cc403b3024572a5f1a322ca85ed24"
 from {username='slave'} at 127.0.0.1:47316
I20260709 13:06:02.148382 11896 raft_consensus.cc:606] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 LEADER]: Received request to transfer leadership to TS 349cc403b3024572a5f1a322ca85ed24
I20260709 13:06:02.445513 11942 consensus_queue.cc:1048] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:02.457094 11985 consensus_queue.cc:1048] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:02.467941 11985 raft_consensus.cc:993] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f: : Instructing follower 349cc403b3024572a5f1a322ca85ed24 to start an election
I20260709 13:06:02.468457 11985 raft_consensus.cc:1081] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 LEADER]: Signalling peer 349cc403b3024572a5f1a322ca85ed24 to start an election
I20260709 13:06:02.470072 11747 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd2ecbfc736d493599e915d2b97f5583"
dest_uuid: "349cc403b3024572a5f1a322ca85ed24"
 from {username='slave'} at 127.0.0.1:57030
I20260709 13:06:02.470734 11747 raft_consensus.cc:493] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:02.471122 11747 raft_consensus.cc:3060] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:02.477792 11747 raft_consensus.cc:515] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:02.480187 11747 leader_election.cc:290] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [CANDIDATE]: Term 2 election: Requested vote from peers af15c44e874c442aab0a32438663b35f (127.0.107.194:39131), 800ee909a7ad492eb71c4cccb4e24e2f (127.0.107.195:39013)
I20260709 13:06:02.480984 11821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2ecbfc736d493599e915d2b97f5583" candidate_uuid: "349cc403b3024572a5f1a322ca85ed24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "af15c44e874c442aab0a32438663b35f"
I20260709 13:06:02.481277 11896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2ecbfc736d493599e915d2b97f5583" candidate_uuid: "349cc403b3024572a5f1a322ca85ed24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "800ee909a7ad492eb71c4cccb4e24e2f"
I20260709 13:06:02.481819 11821 raft_consensus.cc:3060] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:02.481954 11896 raft_consensus.cc:3055] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:02.482398 11896 raft_consensus.cc:740] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 800ee909a7ad492eb71c4cccb4e24e2f, State: Running, Role: LEADER
I20260709 13:06:02.483093 11896 consensus_queue.cc:260] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:02.484058 11896 raft_consensus.cc:3060] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:02.490484 11821 raft_consensus.cc:2468] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349cc403b3024572a5f1a322ca85ed24 in term 2.
I20260709 13:06:02.492234 11713 leader_election.cc:304] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349cc403b3024572a5f1a322ca85ed24, af15c44e874c442aab0a32438663b35f; no voters: 
I20260709 13:06:02.493340 11981 raft_consensus.cc:2804] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:02.493913 11981 raft_consensus.cc:697] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 2 LEADER]: Becoming Leader. State: Replica: 349cc403b3024572a5f1a322ca85ed24, State: Running, Role: LEADER
I20260709 13:06:02.494029 11896 raft_consensus.cc:2468] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349cc403b3024572a5f1a322ca85ed24 in term 2.
I20260709 13:06:02.494966 11981 consensus_queue.cc:237] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:02.501271 11949 raft_consensus.cc:993] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f: : Instructing follower 349cc403b3024572a5f1a322ca85ed24 to start an election
I20260709 13:06:02.501899 11987 raft_consensus.cc:1081] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 1 LEADER]: Signalling peer 349cc403b3024572a5f1a322ca85ed24 to start an election
I20260709 13:06:02.503612 11747 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa5453391722496dba2413e300b7658e"
dest_uuid: "349cc403b3024572a5f1a322ca85ed24"
 from {username='slave'} at 127.0.0.1:57024
I20260709 13:06:02.504195 11747 raft_consensus.cc:493] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:02.504559 11747 raft_consensus.cc:3060] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:02.505012 11643 catalog_manager.cc:5697] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 reported cstate change: term changed from 1 to 2, leader changed from 800ee909a7ad492eb71c4cccb4e24e2f (127.0.107.195) to 349cc403b3024572a5f1a322ca85ed24 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "349cc403b3024572a5f1a322ca85ed24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:02.511674 11747 raft_consensus.cc:515] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } } peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } }
I20260709 13:06:02.514299 11747 leader_election.cc:290] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [CANDIDATE]: Term 2 election: Requested vote from peers 800ee909a7ad492eb71c4cccb4e24e2f (127.0.107.195:39013), af15c44e874c442aab0a32438663b35f (127.0.107.194:39131)
I20260709 13:06:02.515329 11896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5453391722496dba2413e300b7658e" candidate_uuid: "349cc403b3024572a5f1a322ca85ed24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "800ee909a7ad492eb71c4cccb4e24e2f"
I20260709 13:06:02.515583 11821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5453391722496dba2413e300b7658e" candidate_uuid: "349cc403b3024572a5f1a322ca85ed24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "af15c44e874c442aab0a32438663b35f"
I20260709 13:06:02.516105 11896 raft_consensus.cc:3060] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:02.516307 11821 raft_consensus.cc:3055] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:02.516798 11821 raft_consensus.cc:740] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: af15c44e874c442aab0a32438663b35f, State: Running, Role: LEADER
I20260709 13:06:02.517602 11821 consensus_queue.cc:260] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } } peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } }
I20260709 13:06:02.519001 11821 raft_consensus.cc:3060] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:02.522363 11896 raft_consensus.cc:2468] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349cc403b3024572a5f1a322ca85ed24 in term 2.
I20260709 13:06:02.523927 11711 leader_election.cc:304] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349cc403b3024572a5f1a322ca85ed24, 800ee909a7ad492eb71c4cccb4e24e2f; no voters: 
I20260709 13:06:02.524739 11981 raft_consensus.cc:2804] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:02.525306 11981 raft_consensus.cc:697] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [term 2 LEADER]: Becoming Leader. State: Replica: 349cc403b3024572a5f1a322ca85ed24, State: Running, Role: LEADER
I20260709 13:06:02.525398 11821 raft_consensus.cc:2468] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349cc403b3024572a5f1a322ca85ed24 in term 2.
I20260709 13:06:02.526329 11981 consensus_queue.cc:237] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } } peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } }
I20260709 13:06:02.533159 11643 catalog_manager.cc:5697] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 reported cstate change: term changed from 1 to 2, leader changed from af15c44e874c442aab0a32438663b35f (127.0.107.194) to 349cc403b3024572a5f1a322ca85ed24 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "349cc403b3024572a5f1a322ca85ed24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:02.551072 11981 consensus_queue.cc:1048] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:02.563663 11981 consensus_queue.cc:1048] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:02.605897 11944 raft_consensus.cc:993] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24: : Instructing follower 800ee909a7ad492eb71c4cccb4e24e2f to start an election
I20260709 13:06:02.606454 11944 raft_consensus.cc:1081] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 1 LEADER]: Signalling peer 800ee909a7ad492eb71c4cccb4e24e2f to start an election
I20260709 13:06:02.608053 11896 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64126e3ded0a41b5b8626e7b49fdeed2"
dest_uuid: "800ee909a7ad492eb71c4cccb4e24e2f"
 from {username='slave'} at 127.0.0.1:59748
I20260709 13:06:02.608609 11896 raft_consensus.cc:493] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:02.608932 11896 raft_consensus.cc:3060] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:02.615204 11896 raft_consensus.cc:515] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:02.616925 11896 leader_election.cc:290] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [CANDIDATE]: Term 2 election: Requested vote from peers 349cc403b3024572a5f1a322ca85ed24 (127.0.107.193:45555), af15c44e874c442aab0a32438663b35f (127.0.107.194:39131)
I20260709 13:06:02.617945 11747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64126e3ded0a41b5b8626e7b49fdeed2" candidate_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "349cc403b3024572a5f1a322ca85ed24"
I20260709 13:06:02.618269 11821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64126e3ded0a41b5b8626e7b49fdeed2" candidate_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "af15c44e874c442aab0a32438663b35f"
I20260709 13:06:02.619081 11747 raft_consensus.cc:3055] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:02.619341 11821 raft_consensus.cc:3060] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:02.619493 11747 raft_consensus.cc:740] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 349cc403b3024572a5f1a322ca85ed24, State: Running, Role: LEADER
I20260709 13:06:02.620654 11747 consensus_queue.cc:260] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:02.622165 11747 raft_consensus.cc:3060] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:02.626962 11821 raft_consensus.cc:2468] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 800ee909a7ad492eb71c4cccb4e24e2f in term 2.
I20260709 13:06:02.628429 11862 leader_election.cc:304] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 800ee909a7ad492eb71c4cccb4e24e2f, af15c44e874c442aab0a32438663b35f; no voters: 
I20260709 13:06:02.629575 11985 raft_consensus.cc:2804] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:02.629863 11747 raft_consensus.cc:2468] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 800ee909a7ad492eb71c4cccb4e24e2f in term 2.
I20260709 13:06:02.630072 11985 raft_consensus.cc:697] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [term 2 LEADER]: Becoming Leader. State: Replica: 800ee909a7ad492eb71c4cccb4e24e2f, State: Running, Role: LEADER
I20260709 13:06:02.631089 11985 consensus_queue.cc:237] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:02.638164 11643 catalog_manager.cc:5697] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f reported cstate change: term changed from 1 to 2, leader changed from 349cc403b3024572a5f1a322ca85ed24 (127.0.107.193) to 800ee909a7ad492eb71c4cccb4e24e2f (127.0.107.195). New cstate: current_term: 2 leader_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:02.919225   431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:06:02.921166   431 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:06:02.921624   431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:06:02.935918 11747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd2ecbfc736d493599e915d2b97f5583"
dest_uuid: "349cc403b3024572a5f1a322ca85ed24"
mode: GRACEFUL
new_leader_uuid: "af15c44e874c442aab0a32438663b35f"
 from {username='slave'} at 127.0.0.1:40416
I20260709 13:06:02.936659 11747 raft_consensus.cc:606] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 2 LEADER]: Received request to transfer leadership to TS af15c44e874c442aab0a32438663b35f
I20260709 13:06:02.937960   431 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:06:02.938365 11896 raft_consensus.cc:1275] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 2 FOLLOWER]: Refusing update from remote peer 349cc403b3024572a5f1a322ca85ed24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:02.939950 11944 consensus_queue.cc:1048] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:02.948909 11821 raft_consensus.cc:1275] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 2 FOLLOWER]: Refusing update from remote peer 349cc403b3024572a5f1a322ca85ed24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:02.951126 11944 consensus_queue.cc:1048] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:02.961019 11981 raft_consensus.cc:993] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24: : Instructing follower af15c44e874c442aab0a32438663b35f to start an election
I20260709 13:06:02.961858 11944 raft_consensus.cc:1081] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 2 LEADER]: Signalling peer af15c44e874c442aab0a32438663b35f to start an election
I20260709 13:06:02.963562 11821 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd2ecbfc736d493599e915d2b97f5583"
dest_uuid: "af15c44e874c442aab0a32438663b35f"
 from {username='slave'} at 127.0.0.1:46718
I20260709 13:06:02.964372 11821 raft_consensus.cc:493] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:02.964761 11821 raft_consensus.cc:3060] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:02.970829 11821 raft_consensus.cc:515] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:02.972947 11821 leader_election.cc:290] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [CANDIDATE]: Term 3 election: Requested vote from peers 349cc403b3024572a5f1a322ca85ed24 (127.0.107.193:45555), 800ee909a7ad492eb71c4cccb4e24e2f (127.0.107.195:39013)
I20260709 13:06:02.973716 11747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2ecbfc736d493599e915d2b97f5583" candidate_uuid: "af15c44e874c442aab0a32438663b35f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "349cc403b3024572a5f1a322ca85ed24"
I20260709 13:06:02.974014 11896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2ecbfc736d493599e915d2b97f5583" candidate_uuid: "af15c44e874c442aab0a32438663b35f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "800ee909a7ad492eb71c4cccb4e24e2f"
I20260709 13:06:02.974421 11747 raft_consensus.cc:3055] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:02.974687 11896 raft_consensus.cc:3060] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:02.975051 11747 raft_consensus.cc:740] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 349cc403b3024572a5f1a322ca85ed24, State: Running, Role: LEADER
I20260709 13:06:02.976045 11747 consensus_queue.cc:260] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:02.977344 11747 raft_consensus.cc:3060] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:02.982475 11896 raft_consensus.cc:2468] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af15c44e874c442aab0a32438663b35f in term 3.
I20260709 13:06:02.983724 11786 leader_election.cc:304] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 800ee909a7ad492eb71c4cccb4e24e2f, af15c44e874c442aab0a32438663b35f; no voters: 
I20260709 13:06:02.984669 11987 raft_consensus.cc:2804] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:02.984549 11747 raft_consensus.cc:2468] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af15c44e874c442aab0a32438663b35f in term 3.
I20260709 13:06:02.985304 11987 raft_consensus.cc:697] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 3 LEADER]: Becoming Leader. State: Replica: af15c44e874c442aab0a32438663b35f, State: Running, Role: LEADER
I20260709 13:06:02.986184 11987 consensus_queue.cc:237] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } }
I20260709 13:06:02.993583 11644 catalog_manager.cc:5697] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f reported cstate change: term changed from 2 to 3, leader changed from 349cc403b3024572a5f1a322ca85ed24 (127.0.107.193) to af15c44e874c442aab0a32438663b35f (127.0.107.194). New cstate: current_term: 3 leader_uuid: "af15c44e874c442aab0a32438663b35f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349cc403b3024572a5f1a322ca85ed24" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 45555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.001377 11821 raft_consensus.cc:1275] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 2 FOLLOWER]: Refusing update from remote peer 349cc403b3024572a5f1a322ca85ed24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:03.002990 11944 consensus_queue.cc:1048] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af15c44e874c442aab0a32438663b35f" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:03.009172   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:06:03.023466 11896 raft_consensus.cc:1275] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f [term 2 FOLLOWER]: Refusing update from remote peer 349cc403b3024572a5f1a322ca85ed24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:03.026087 11978 consensus_queue.cc:1048] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "800ee909a7ad492eb71c4cccb4e24e2f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 39013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:03.044905   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:03.045650   431 tablet_replica.cc:333] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24: stopping tablet replica
I20260709 13:06:03.046309   431 raft_consensus.cc:2243] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.046784   431 raft_consensus.cc:2272] T bd2ecbfc736d493599e915d2b97f5583 P 349cc403b3024572a5f1a322ca85ed24 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.049790   431 tablet_replica.cc:333] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24: stopping tablet replica
I20260709 13:06:03.050665   431 raft_consensus.cc:2243] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.051539   431 raft_consensus.cc:2272] T 640a7be311ef41d69cf98cf7505507bb P 349cc403b3024572a5f1a322ca85ed24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.053998   431 tablet_replica.cc:333] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24: stopping tablet replica
I20260709 13:06:03.054502   431 raft_consensus.cc:2243] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.054840   431 raft_consensus.cc:2272] T 64126e3ded0a41b5b8626e7b49fdeed2 P 349cc403b3024572a5f1a322ca85ed24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.057092   431 tablet_replica.cc:333] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24: stopping tablet replica
I20260709 13:06:03.057603   431 raft_consensus.cc:2243] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.058429   431 raft_consensus.cc:2272] T fa5453391722496dba2413e300b7658e P 349cc403b3024572a5f1a322ca85ed24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.084380   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:06:03.097232   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 13:06:03.118024   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:03.119063   431 tablet_replica.cc:333] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f: stopping tablet replica
I20260709 13:06:03.119807   431 raft_consensus.cc:2243] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.120348   431 raft_consensus.cc:2272] T 640a7be311ef41d69cf98cf7505507bb P af15c44e874c442aab0a32438663b35f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.123230   431 tablet_replica.cc:333] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f: stopping tablet replica
I20260709 13:06:03.123798   431 raft_consensus.cc:2243] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.124447   431 pending_rounds.cc:70] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f: Trying to abort 1 pending ops.
I20260709 13:06:03.124657   431 pending_rounds.cc:77] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305635278801285120 op_type: NO_OP noop_request { }
I20260709 13:06:03.124972   431 raft_consensus.cc:2889] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:06:03.125264   431 raft_consensus.cc:2272] T bd2ecbfc736d493599e915d2b97f5583 P af15c44e874c442aab0a32438663b35f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.127772   431 tablet_replica.cc:333] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f: stopping tablet replica
I20260709 13:06:03.128329   431 raft_consensus.cc:2243] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.128787   431 raft_consensus.cc:2272] T 64126e3ded0a41b5b8626e7b49fdeed2 P af15c44e874c442aab0a32438663b35f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.130641   431 tablet_replica.cc:333] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f: stopping tablet replica
I20260709 13:06:03.131238   431 raft_consensus.cc:2243] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.131675   431 raft_consensus.cc:2272] T fa5453391722496dba2413e300b7658e P af15c44e874c442aab0a32438663b35f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.145834   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:06:03.159526   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:06:03.179690   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:03.180281   431 tablet_replica.cc:333] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f: stopping tablet replica
I20260709 13:06:03.180795   431 raft_consensus.cc:2243] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.181277   431 raft_consensus.cc:2272] T bd2ecbfc736d493599e915d2b97f5583 P 800ee909a7ad492eb71c4cccb4e24e2f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.183597   431 tablet_replica.cc:333] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f: stopping tablet replica
I20260709 13:06:03.184080   431 raft_consensus.cc:2243] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.184465   431 raft_consensus.cc:2272] T 640a7be311ef41d69cf98cf7505507bb P 800ee909a7ad492eb71c4cccb4e24e2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.186721   431 tablet_replica.cc:333] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f: stopping tablet replica
I20260709 13:06:03.187290   431 raft_consensus.cc:2243] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:03.187688   431 raft_consensus.cc:2272] T fa5453391722496dba2413e300b7658e P 800ee909a7ad492eb71c4cccb4e24e2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.189987   431 tablet_replica.cc:333] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f: stopping tablet replica
I20260709 13:06:03.190454   431 raft_consensus.cc:2243] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.191100   431 pending_rounds.cc:70] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f: Trying to abort 1 pending ops.
I20260709 13:06:03.191255   431 pending_rounds.cc:77] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305635277346430976 op_type: NO_OP noop_request { }
I20260709 13:06:03.191483   431 raft_consensus.cc:2889] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:06:03.191710   431 raft_consensus.cc:2272] T 64126e3ded0a41b5b8626e7b49fdeed2 P 800ee909a7ad492eb71c4cccb4e24e2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.215024   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:06:03.228771   431 master.cc:562] Master@127.0.107.254:39099 shutting down...
I20260709 13:06:03.244602   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:03.245162   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:03.245492   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7bba317fff1648debe515449d96b3233: stopping tablet replica
I20260709 13:06:03.265018   431 master.cc:584] Master@127.0.107.254:39099 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2678 ms)
[----------] 6 tests from LeaderRebalancerTest (348475 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:06:03.295591   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:42411
I20260709 13:06:03.296808   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:03.303469 11995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:03.303416 11996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:03.304612 11998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:03.307154   431 server_base.cc:1061] running on GCE node
I20260709 13:06:03.308032   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:03.308283   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:03.308460   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602363308446 us; error 0 us; skew 500 ppm
I20260709 13:06:03.309041   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:03.311661   431 webserver.cc:533] Webserver started at http://127.0.107.254:43385/ using document root <none> and password file <none>
I20260709 13:06:03.312182   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:03.312372   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:03.312664   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:03.313762   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/master-0-root/instance:
uuid: "634b2b70487747899d3404a63bf5c87c"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-cq4t"
I20260709 13:06:03.318755   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:06:03.323078 12003 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.323937   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 13:06:03.324316   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/master-0-root
uuid: "634b2b70487747899d3404a63bf5c87c"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-cq4t"
I20260709 13:06:03.324731   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:03.349269   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:03.350410   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:03.393699   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:42411
I20260709 13:06:03.393823 12054 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:42411 every 8 connection(s)
I20260709 13:06:03.398265 12055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.410540 12055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c: Bootstrap starting.
I20260709 13:06:03.415614 12055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:03.420542 12055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c: No bootstrap required, opened a new log
I20260709 13:06:03.422998 12055 raft_consensus.cc:359] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634b2b70487747899d3404a63bf5c87c" member_type: VOTER }
I20260709 13:06:03.423460 12055 raft_consensus.cc:385] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:03.423661 12055 raft_consensus.cc:740] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634b2b70487747899d3404a63bf5c87c, State: Initialized, Role: FOLLOWER
I20260709 13:06:03.424305 12055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634b2b70487747899d3404a63bf5c87c" member_type: VOTER }
I20260709 13:06:03.424785 12055 raft_consensus.cc:399] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:03.424993 12055 raft_consensus.cc:493] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:03.425230 12055 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:03.430516 12055 raft_consensus.cc:515] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634b2b70487747899d3404a63bf5c87c" member_type: VOTER }
I20260709 13:06:03.431164 12055 leader_election.cc:304] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 634b2b70487747899d3404a63bf5c87c; no voters: 
I20260709 13:06:03.432364 12055 leader_election.cc:290] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:03.432724 12058 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:03.434392 12058 raft_consensus.cc:697] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [term 1 LEADER]: Becoming Leader. State: Replica: 634b2b70487747899d3404a63bf5c87c, State: Running, Role: LEADER
I20260709 13:06:03.435156 12058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634b2b70487747899d3404a63bf5c87c" member_type: VOTER }
I20260709 13:06:03.435811 12055 sys_catalog.cc:565] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:03.438513 12059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "634b2b70487747899d3404a63bf5c87c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634b2b70487747899d3404a63bf5c87c" member_type: VOTER } }
I20260709 13:06:03.438535 12060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 634b2b70487747899d3404a63bf5c87c. Latest consensus state: current_term: 1 leader_uuid: "634b2b70487747899d3404a63bf5c87c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634b2b70487747899d3404a63bf5c87c" member_type: VOTER } }
I20260709 13:06:03.439474 12060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:03.439512 12059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:03.449491 12064 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:03.455498 12064 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:03.456609   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:03.465075 12064 catalog_manager.cc:1383] Generated new cluster ID: cc3fb1d69c1c4cc6885df8a5bdf8fa9e
I20260709 13:06:03.465441 12064 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:03.484576 12064 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:03.485925 12064 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:03.503000 12064 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c: Generated new TSK 0
I20260709 13:06:03.503814 12064 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:03.523936   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:03.530375 12076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:03.531241 12077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:03.533182 12079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:03.533203   431 server_base.cc:1061] running on GCE node
I20260709 13:06:03.534578   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:03.534806   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:03.535024   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602363535009 us; error 0 us; skew 500 ppm
I20260709 13:06:03.535523   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:03.538183   431 webserver.cc:533] Webserver started at http://127.0.107.193:41869/ using document root <none> and password file <none>
I20260709 13:06:03.538638   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:03.538791   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:03.539100   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:03.540112   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-0-root/instance:
uuid: "2780688fce76495ca9c1ff6e8e998046"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-cq4t"
I20260709 13:06:03.544690   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:06:03.548758 12084 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.549666   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:06:03.550007   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-0-root
uuid: "2780688fce76495ca9c1ff6e8e998046"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-cq4t"
I20260709 13:06:03.550311   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:03.576745   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:03.577920   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:03.579612   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:03.582113   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:03.582341   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.582615   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:03.582788   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.624346   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:35857
I20260709 13:06:03.624497 12146 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:35857 every 8 connection(s)
I20260709 13:06:03.629714   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:03.638533 12151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:03.640743 12152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:03.642259   431 server_base.cc:1061] running on GCE node
W20260709 13:06:03.644325 12154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:03.645177 12147 heartbeater.cc:344] Connected to a master server at 127.0.107.254:42411
I20260709 13:06:03.645589   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:06:03.645682 12147 heartbeater.cc:461] Registering TS with master...
W20260709 13:06:03.646054   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:03.646263   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602363646246 us; error 0 us; skew 500 ppm
I20260709 13:06:03.646822 12147 heartbeater.cc:507] Master 127.0.107.254:42411 requested a full tablet report, sending...
I20260709 13:06:03.647086   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:03.649652 12020 ts_manager.cc:194] Registered new tserver with Master: 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857)
I20260709 13:06:03.649812   431 webserver.cc:533] Webserver started at http://127.0.107.194:46479/ using document root <none> and password file <none>
I20260709 13:06:03.650525   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:03.650738   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:03.651053   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:03.651757 12020 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52552
I20260709 13:06:03.652351   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-1-root/instance:
uuid: "52d9dc2493034c6cbd8196d41f879ce8"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-cq4t"
I20260709 13:06:03.657559   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:06:03.661873 12159 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.662824   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:06:03.663293   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-1-root
uuid: "52d9dc2493034c6cbd8196d41f879ce8"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-cq4t"
I20260709 13:06:03.663658   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:03.682742   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:03.683962   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:03.685463   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:03.687794   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:03.688030   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.688278   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:03.688459   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.730213   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:34543
I20260709 13:06:03.730410 12221 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:34543 every 8 connection(s)
I20260709 13:06:03.735291   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:03.742786 12225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:03.744815 12226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:03.745463 12228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:03.746479   431 server_base.cc:1061] running on GCE node
I20260709 13:06:03.747556   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:03.747864   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:03.748087   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602363748070 us; error 0 us; skew 500 ppm
I20260709 13:06:03.748795   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:03.749141 12222 heartbeater.cc:344] Connected to a master server at 127.0.107.254:42411
I20260709 13:06:03.749784 12222 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:03.750725 12222 heartbeater.cc:507] Master 127.0.107.254:42411 requested a full tablet report, sending...
I20260709 13:06:03.751693   431 webserver.cc:533] Webserver started at http://127.0.107.195:33801/ using document root <none> and password file <none>
I20260709 13:06:03.752440   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:03.752693   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:03.752972   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:03.753104 12020 ts_manager.cc:194] Registered new tserver with Master: 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:03.754341   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-2-root/instance:
uuid: "437a699518154aa580c1ba0868048c05"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-cq4t"
I20260709 13:06:03.754647 12020 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52554
I20260709 13:06:03.759346   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:06:03.763590 12233 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.764609   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:06:03.764986   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-2-root
uuid: "437a699518154aa580c1ba0868048c05"
format_stamp: "Formatted at 2026-07-09 13:06:03 on dist-test-slave-cq4t"
I20260709 13:06:03.765321   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602014632082-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:03.779919   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:03.781023   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:03.782487   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:03.784979   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:03.785221   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.785481   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:03.785706   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:03.829965   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:32769
I20260709 13:06:03.830080 12295 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:32769 every 8 connection(s)
I20260709 13:06:03.846323 12296 heartbeater.cc:344] Connected to a master server at 127.0.107.254:42411
I20260709 13:06:03.847082 12296 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:03.848097 12296 heartbeater.cc:507] Master 127.0.107.254:42411 requested a full tablet report, sending...
I20260709 13:06:03.850397 12020 ts_manager.cc:194] Registered new tserver with Master: 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:03.851132   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017618423s
I20260709 13:06:03.852368 12020 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52568
I20260709 13:06:03.880311 12020 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52580:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:03.883351 12020 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:03.954953 12187 tablet_service.cc:1511] Processing CreateTablet for tablet 9b4f009c4594491582878ae8b1f5f21b (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:03.956557 12187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b4f009c4594491582878ae8b1f5f21b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.963012 12185 tablet_service.cc:1511] Processing CreateTablet for tablet f38db537da264dd4adf4363864b96250 (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:03.964574 12185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f38db537da264dd4adf4363864b96250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.958195 12186 tablet_service.cc:1511] Processing CreateTablet for tablet d6bd5ec5c50c48b78c88d403817fbdae (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:03.966667 12186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6bd5ec5c50c48b78c88d403817fbdae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.968873 12184 tablet_service.cc:1511] Processing CreateTablet for tablet 8b89ecf901f543718a4f562df52c12ff (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:03.970361 12184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b89ecf901f543718a4f562df52c12ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.972352 12261 tablet_service.cc:1511] Processing CreateTablet for tablet 9b4f009c4594491582878ae8b1f5f21b (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:03.973773 12261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b4f009c4594491582878ae8b1f5f21b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.974051 12112 tablet_service.cc:1511] Processing CreateTablet for tablet 9b4f009c4594491582878ae8b1f5f21b (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:03.975541 12112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b4f009c4594491582878ae8b1f5f21b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.978010 12260 tablet_service.cc:1511] Processing CreateTablet for tablet d6bd5ec5c50c48b78c88d403817fbdae (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:03.978188 12183 tablet_service.cc:1511] Processing CreateTablet for tablet fd09f5dac3864c3da2319df9b220cc3f (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:03.979501 12260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6bd5ec5c50c48b78c88d403817fbdae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.979624 12183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd09f5dac3864c3da2319df9b220cc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.980480 12111 tablet_service.cc:1511] Processing CreateTablet for tablet d6bd5ec5c50c48b78c88d403817fbdae (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:03.981856 12111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6bd5ec5c50c48b78c88d403817fbdae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.984211 12259 tablet_service.cc:1511] Processing CreateTablet for tablet f38db537da264dd4adf4363864b96250 (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:03.984515 12182 tablet_service.cc:1511] Processing CreateTablet for tablet 63915eb22a6f42e3875c55499d3cafab (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:03.985834 12259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f38db537da264dd4adf4363864b96250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.985920 12182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63915eb22a6f42e3875c55499d3cafab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.989138 12258 tablet_service.cc:1511] Processing CreateTablet for tablet 8b89ecf901f543718a4f562df52c12ff (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:03.990667 12258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b89ecf901f543718a4f562df52c12ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.991568 12181 tablet_service.cc:1511] Processing CreateTablet for tablet 106eefe703c24ccc96acae1f610d7fdc (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:03.993136 12181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 106eefe703c24ccc96acae1f610d7fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.995649 12257 tablet_service.cc:1511] Processing CreateTablet for tablet fd09f5dac3864c3da2319df9b220cc3f (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:03.997197 12257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd09f5dac3864c3da2319df9b220cc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:03.996771 12180 tablet_service.cc:1511] Processing CreateTablet for tablet c16e10739a0e45b4ae22a2366e1162c3 (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:03.998338 12180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16e10739a0e45b4ae22a2366e1162c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.003005 12110 tablet_service.cc:1511] Processing CreateTablet for tablet f38db537da264dd4adf4363864b96250 (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:04.004608 12110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f38db537da264dd4adf4363864b96250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.005028 12179 tablet_service.cc:1511] Processing CreateTablet for tablet 3d0833359f734033bfe7eab9cc74666f (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:04.006417 12179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d0833359f734033bfe7eab9cc74666f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.009050 12256 tablet_service.cc:1511] Processing CreateTablet for tablet 63915eb22a6f42e3875c55499d3cafab (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:04.010532 12256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63915eb22a6f42e3875c55499d3cafab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.016265 12109 tablet_service.cc:1511] Processing CreateTablet for tablet 8b89ecf901f543718a4f562df52c12ff (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:04.018016 12109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b89ecf901f543718a4f562df52c12ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.021588 12108 tablet_service.cc:1511] Processing CreateTablet for tablet fd09f5dac3864c3da2319df9b220cc3f (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:04.024968 12255 tablet_service.cc:1511] Processing CreateTablet for tablet 106eefe703c24ccc96acae1f610d7fdc (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:04.025322 12254 tablet_service.cc:1511] Processing CreateTablet for tablet c16e10739a0e45b4ae22a2366e1162c3 (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:04.026598 12255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 106eefe703c24ccc96acae1f610d7fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.028203 12108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd09f5dac3864c3da2319df9b220cc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.029153 12253 tablet_service.cc:1511] Processing CreateTablet for tablet 3d0833359f734033bfe7eab9cc74666f (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:04.032414 12107 tablet_service.cc:1511] Processing CreateTablet for tablet 63915eb22a6f42e3875c55499d3cafab (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:04.036804 12253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d0833359f734033bfe7eab9cc74666f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.037120 12107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63915eb22a6f42e3875c55499d3cafab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.039011 12254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16e10739a0e45b4ae22a2366e1162c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.039232 12106 tablet_service.cc:1511] Processing CreateTablet for tablet 106eefe703c24ccc96acae1f610d7fdc (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:04.059689 12106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 106eefe703c24ccc96acae1f610d7fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.089241 12105 tablet_service.cc:1511] Processing CreateTablet for tablet c16e10739a0e45b4ae22a2366e1162c3 (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:04.092150 12104 tablet_service.cc:1511] Processing CreateTablet for tablet 3d0833359f734033bfe7eab9cc74666f (DEFAULT_TABLE table=test-workload [id=46cb2c1d972b4abbbbb7f26b4a68d394]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:04.090761 12105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16e10739a0e45b4ae22a2366e1162c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.095371 12104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d0833359f734033bfe7eab9cc74666f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:04.106678 12310 tablet_bootstrap.cc:492] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:04.125769 12310 tablet_bootstrap.cc:654] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.167510 12311 tablet_bootstrap.cc:492] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:04.178071 12310 tablet_bootstrap.cc:492] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:04.178668 12310 ts_tablet_manager.cc:1403] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.072s	user 0.009s	sys 0.016s
I20260709 13:06:04.181751 12310 raft_consensus.cc:359] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.182559 12310 raft_consensus.cc:385] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.182880 12310 raft_consensus.cc:740] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.187850 12310 consensus_queue.cc:260] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.195964 12313 tablet_bootstrap.cc:492] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:04.197503 12311 tablet_bootstrap.cc:654] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.203037 12222 heartbeater.cc:499] Master 127.0.107.254:42411 was elected leader, sending a full tablet report...
I20260709 13:06:04.206261 12311 tablet_bootstrap.cc:492] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:04.206362 12310 ts_tablet_manager.cc:1434] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.003s
I20260709 13:06:04.207050 12311 ts_tablet_manager.cc:1403] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.040s	user 0.007s	sys 0.005s
I20260709 13:06:04.207695 12310 tablet_bootstrap.cc:492] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:04.210103 12311 raft_consensus.cc:359] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.210975 12311 raft_consensus.cc:385] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.211329 12311 raft_consensus.cc:740] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.212241 12311 consensus_queue.cc:260] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.214942 12296 heartbeater.cc:499] Master 127.0.107.254:42411 was elected leader, sending a full tablet report...
I20260709 13:06:04.215682 12311 ts_tablet_manager.cc:1434] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:06:04.216806 12311 tablet_bootstrap.cc:492] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:04.217278 12310 tablet_bootstrap.cc:654] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.223165 12311 tablet_bootstrap.cc:654] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.232568 12313 tablet_bootstrap.cc:654] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.237322 12311 tablet_bootstrap.cc:492] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:04.238051 12311 ts_tablet_manager.cc:1403] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.009s
I20260709 13:06:04.240887 12311 raft_consensus.cc:359] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } }
I20260709 13:06:04.241218 12313 tablet_bootstrap.cc:492] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:04.241611 12311 raft_consensus.cc:385] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.241978 12313 ts_tablet_manager.cc:1403] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.000s
I20260709 13:06:04.241979 12311 raft_consensus.cc:740] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.243094 12311 consensus_queue.cc:260] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } }
I20260709 13:06:04.245224 12311 ts_tablet_manager.cc:1434] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:06:04.245944 12310 tablet_bootstrap.cc:492] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:04.246040 12311 tablet_bootstrap.cc:492] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:04.246816 12310 ts_tablet_manager.cc:1403] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.039s	user 0.007s	sys 0.011s
I20260709 13:06:04.248199 12313 raft_consensus.cc:359] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.249300 12313 raft_consensus.cc:385] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.251147 12313 raft_consensus.cc:740] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.250697 12310 raft_consensus.cc:359] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.251629 12310 raft_consensus.cc:385] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.251953 12310 raft_consensus.cc:740] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.252277 12313 consensus_queue.cc:260] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.252822 12311 tablet_bootstrap.cc:654] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.252853 12310 consensus_queue.cc:260] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.255597 12310 ts_tablet_manager.cc:1434] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:04.256624 12310 tablet_bootstrap.cc:492] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:04.260478 12311 tablet_bootstrap.cc:492] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:04.261092 12311 ts_tablet_manager.cc:1403] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 13:06:04.263191 12310 tablet_bootstrap.cc:654] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.264166 12311 raft_consensus.cc:359] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.265261 12311 raft_consensus.cc:385] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.265656 12311 raft_consensus.cc:740] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.266690 12311 consensus_queue.cc:260] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.267663 12316 raft_consensus.cc:493] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.268537 12316 raft_consensus.cc:515] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.281186 12316 leader_election.cc:290] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:04.315860 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd09f5dac3864c3da2319df9b220cc3f" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
W20260709 13:06:04.318497 12237 leader_election.cc:343] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.323210 12147 heartbeater.cc:499] Master 127.0.107.254:42411 was elected leader, sending a full tablet report...
I20260709 13:06:04.324107 12313 ts_tablet_manager.cc:1434] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.082s	user 0.001s	sys 0.032s
I20260709 13:06:04.325865 12313 tablet_bootstrap.cc:492] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:04.334869 12313 tablet_bootstrap.cc:654] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.350428 12311 ts_tablet_manager.cc:1434] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.089s	user 0.031s	sys 0.053s
I20260709 13:06:04.351254 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd09f5dac3864c3da2319df9b220cc3f" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:04.352087 12313 tablet_bootstrap.cc:492] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
W20260709 13:06:04.353009 12235 leader_election.cc:343] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.353591 12311 tablet_bootstrap.cc:492] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:04.354552 12310 tablet_bootstrap.cc:492] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:04.355235 12310 ts_tablet_manager.cc:1403] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.099s	user 0.032s	sys 0.058s
I20260709 13:06:04.353506 12235 leader_election.cc:304] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 437a699518154aa580c1ba0868048c05; no voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:04.359162 12310 raft_consensus.cc:359] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.360013 12310 raft_consensus.cc:385] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.360320 12313 ts_tablet_manager.cc:1403] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.007s
I20260709 13:06:04.360371 12316 raft_consensus.cc:2749] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:04.360771 12311 tablet_bootstrap.cc:654] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.361357 12316 raft_consensus.cc:493] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.361831 12316 raft_consensus.cc:515] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } }
I20260709 13:06:04.364192 12313 raft_consensus.cc:359] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.364751 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4f009c4594491582878ae8b1f5f21b" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:04.365729 12316 leader_election.cc:290] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857)
W20260709 13:06:04.366431 12235 leader_election.cc:343] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.366634 12313 raft_consensus.cc:385] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.366991 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4f009c4594491582878ae8b1f5f21b" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:04.360323 12310 raft_consensus.cc:740] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.367481 12313 raft_consensus.cc:740] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
W20260709 13:06:04.368836 12237 leader_election.cc:343] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.368611 12310 consensus_queue.cc:260] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.369396 12237 leader_election.cc:304] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 437a699518154aa580c1ba0868048c05; no voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:04.369053 12313 consensus_queue.cc:260] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.372419 12316 raft_consensus.cc:2749] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:04.372542 12311 tablet_bootstrap.cc:492] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:04.373059 12313 ts_tablet_manager.cc:1434] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 13:06:04.373243 12311 ts_tablet_manager.cc:1403] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.001s
I20260709 13:06:04.374222 12313 tablet_bootstrap.cc:492] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:04.374579 12310 ts_tablet_manager.cc:1434] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:06:04.375778 12310 tablet_bootstrap.cc:492] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:04.376387 12311 raft_consensus.cc:359] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.377328 12311 raft_consensus.cc:385] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.377611 12311 raft_consensus.cc:740] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.378594 12311 consensus_queue.cc:260] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.382763 12310 tablet_bootstrap.cc:654] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.382884 12311 ts_tablet_manager.cc:1434] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 13:06:04.384132 12311 tablet_bootstrap.cc:492] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:04.390236 12311 tablet_bootstrap.cc:654] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.394662 12310 tablet_bootstrap.cc:492] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:04.395246 12310 ts_tablet_manager.cc:1403] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 13:06:04.397344 12320 raft_consensus.cc:493] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.398150 12320 raft_consensus.cc:515] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.398888 12310 raft_consensus.cc:359] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.399860 12310 raft_consensus.cc:385] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.400157 12310 raft_consensus.cc:740] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.401042 12310 consensus_queue.cc:260] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.401885 12311 tablet_bootstrap.cc:492] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:04.402508 12311 ts_tablet_manager.cc:1403] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.003s
I20260709 13:06:04.404229 12320 leader_election.cc:290] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:04.406754 12311 raft_consensus.cc:359] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.408082 12311 raft_consensus.cc:385] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.408515 12311 raft_consensus.cc:740] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.409639 12311 consensus_queue.cc:260] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.413051 12311 ts_tablet_manager.cc:1434] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.005s
I20260709 13:06:04.413630 12310 ts_tablet_manager.cc:1434] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:06:04.414244 12311 tablet_bootstrap.cc:492] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:04.420477 12313 tablet_bootstrap.cc:654] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.424695 12314 raft_consensus.cc:493] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.425302 12314 raft_consensus.cc:515] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.422413 12311 tablet_bootstrap.cc:654] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.431041 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f38db537da264dd4adf4363864b96250" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:04.431486 12314 leader_election.cc:290] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:04.434391 12316 raft_consensus.cc:493] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.435534 12316 raft_consensus.cc:515] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.437543 12310 tablet_bootstrap.cc:492] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:04.445057 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6bd5ec5c50c48b78c88d403817fbdae" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
W20260709 13:06:04.445644 12086 leader_election.cc:343] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.445544 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6bd5ec5c50c48b78c88d403817fbdae" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:04.441452 12316 leader_election.cc:290] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
W20260709 13:06:04.447181 12235 leader_election.cc:343] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:04.448916 12237 leader_election.cc:343] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.449512 12237 leader_election.cc:304] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 437a699518154aa580c1ba0868048c05; no voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8
W20260709 13:06:04.449741 12068 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:04.451460 12316 raft_consensus.cc:2749] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:04.451375 12313 tablet_bootstrap.cc:492] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:04.452687 12311 tablet_bootstrap.cc:492] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:04.452926 12313 ts_tablet_manager.cc:1403] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.079s	user 0.010s	sys 0.010s
I20260709 13:06:04.453244 12311 ts_tablet_manager.cc:1403] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.003s
I20260709 13:06:04.455945 12310 tablet_bootstrap.cc:654] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.456444 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0833359f734033bfe7eab9cc74666f" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:04.456512 12313 raft_consensus.cc:359] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.457581 12313 raft_consensus.cc:385] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.457998 12313 raft_consensus.cc:740] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
W20260709 13:06:04.458360 12163 leader_election.cc:343] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.459062 12313 consensus_queue.cc:260] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.460868 12311 raft_consensus.cc:359] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.461982 12311 raft_consensus.cc:385] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.462369 12311 raft_consensus.cc:740] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.463714 12311 consensus_queue.cc:260] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.468052 12311 ts_tablet_manager.cc:1434] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.001s
I20260709 13:06:04.470638 12320 raft_consensus.cc:493] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.470875 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f38db537da264dd4adf4363864b96250" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:04.471697 12271 raft_consensus.cc:2468] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 0.
I20260709 13:06:04.472430 12311 tablet_bootstrap.cc:492] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:04.474583 12088 leader_election.cc:304] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:04.471352 12320 raft_consensus.cc:515] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.478438 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6bd5ec5c50c48b78c88d403817fbdae" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:04.480218 12311 tablet_bootstrap.cc:654] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.480436 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0833359f734033bfe7eab9cc74666f" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:04.480732 12320 leader_election.cc:290] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:04.481374 12320 raft_consensus.cc:2804] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:04.481776 12320 raft_consensus.cc:493] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.482175 12320 raft_consensus.cc:3060] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:04.482331 12163 leader_election.cc:343] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.482963 12163 leader_election.cc:304] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52d9dc2493034c6cbd8196d41f879ce8; no voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05
W20260709 13:06:04.483664 12086 leader_election.cc:343] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.484539 12314 raft_consensus.cc:2749] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:04.492645 12311 tablet_bootstrap.cc:492] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:04.492967 12313 ts_tablet_manager.cc:1434] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 13:06:04.493510 12311 ts_tablet_manager.cc:1403] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.005s
I20260709 13:06:04.495461 12313 tablet_bootstrap.cc:492] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:04.498075 12311 raft_consensus.cc:359] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.499191 12311 raft_consensus.cc:385] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.499598 12311 raft_consensus.cc:740] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.500684 12311 consensus_queue.cc:260] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.503533 12311 ts_tablet_manager.cc:1434] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.009s	user 0.012s	sys 0.000s
I20260709 13:06:04.504127 12313 tablet_bootstrap.cc:654] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.504069 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6bd5ec5c50c48b78c88d403817fbdae" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:04.504776 12311 tablet_bootstrap.cc:492] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:04.505008 12271 raft_consensus.cc:2468] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 0.
I20260709 13:06:04.506541 12088 leader_election.cc:304] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:04.507766 12337 raft_consensus.cc:2804] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:04.508344 12337 raft_consensus.cc:493] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.508778 12337 raft_consensus.cc:3060] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.512007 12311 tablet_bootstrap.cc:654] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.520058 12310 tablet_bootstrap.cc:492] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:04.520880 12310 ts_tablet_manager.cc:1403] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.084s	user 0.035s	sys 0.011s
I20260709 13:06:04.520313 12320 raft_consensus.cc:515] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.524649 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f38db537da264dd4adf4363864b96250" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:04.525631 12310 raft_consensus.cc:359] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.527046 12310 raft_consensus.cc:385] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:06:04.527362 12086 leader_election.cc:343] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.527474 12310 raft_consensus.cc:740] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.530833 12311 tablet_bootstrap.cc:492] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:04.531375 12311 ts_tablet_manager.cc:1403] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.011s
I20260709 13:06:04.531888 12320 leader_election.cc:290] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Requested vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:04.530009 12337 raft_consensus.cc:515] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.530100 12310 consensus_queue.cc:260] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.533109 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f38db537da264dd4adf4363864b96250" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:04.534204 12271 raft_consensus.cc:3060] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.533905 12311 raft_consensus.cc:359] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.534683 12311 raft_consensus.cc:385] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.534998 12311 raft_consensus.cc:740] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.535616 12310 ts_tablet_manager.cc:1434] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:06:04.535805 12311 consensus_queue.cc:260] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.536697 12310 tablet_bootstrap.cc:492] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:04.543766 12311 ts_tablet_manager.cc:1434] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:06:04.544909 12311 tablet_bootstrap.cc:492] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:04.545284 12310 tablet_bootstrap.cc:654] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.551841 12311 tablet_bootstrap.cc:654] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.558038 12310 tablet_bootstrap.cc:492] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:04.558772 12310 ts_tablet_manager.cc:1403] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.006s
I20260709 13:06:04.563100 12310 raft_consensus.cc:359] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.564224 12310 raft_consensus.cc:385] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.564626 12310 raft_consensus.cc:740] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.565584 12310 consensus_queue.cc:260] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.568141 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6bd5ec5c50c48b78c88d403817fbdae" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
W20260709 13:06:04.571158 12086 leader_election.cc:343] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.571347 12310 ts_tablet_manager.cc:1434] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.002s
I20260709 13:06:04.572937 12310 tablet_bootstrap.cc:492] T d6bd5ec5c50c48b78c88d403817fbdae P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:04.577594 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6bd5ec5c50c48b78c88d403817fbdae" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:04.578385 12270 raft_consensus.cc:3060] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.581485 12311 tablet_bootstrap.cc:492] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:04.582060 12311 ts_tablet_manager.cc:1403] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.015s
I20260709 13:06:04.582679 12310 tablet_bootstrap.cc:654] T d6bd5ec5c50c48b78c88d403817fbdae P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.583200 12316 raft_consensus.cc:493] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.583761 12316 raft_consensus.cc:515] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.585991 12316 leader_election.cc:290] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:04.587033 12271 raft_consensus.cc:2468] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 1.
I20260709 13:06:04.588001 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0833359f734033bfe7eab9cc74666f" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:04.588361 12088 leader_election.cc:304] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 52d9dc2493034c6cbd8196d41f879ce8
W20260709 13:06:04.589550 12237 leader_election.cc:343] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.589643 12320 raft_consensus.cc:2804] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.590821 12337 leader_election.cc:290] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Requested vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:04.590884 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0833359f734033bfe7eab9cc74666f" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:04.591790 12197 raft_consensus.cc:2468] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 0.
I20260709 13:06:04.592303 12270 raft_consensus.cc:2468] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 1.
I20260709 13:06:04.593099 12235 leader_election.cc:304] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 2780688fce76495ca9c1ff6e8e998046
I20260709 13:06:04.593904 12088 leader_election.cc:304] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:04.594388 12316 raft_consensus.cc:2804] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:04.595230 12316 raft_consensus.cc:493] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.595578 12316 raft_consensus.cc:3060] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.596350 12337 raft_consensus.cc:2804] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.597134 12337 raft_consensus.cc:697] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Becoming Leader. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:04.598382 12337 consensus_queue.cc:237] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.600196 12314 raft_consensus.cc:493] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.600948 12314 raft_consensus.cc:515] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.604691 12313 tablet_bootstrap.cc:492] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:04.604966 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16e10739a0e45b4ae22a2366e1162c3" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:04.605579 12313 ts_tablet_manager.cc:1403] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.110s	user 0.041s	sys 0.036s
W20260709 13:06:04.607118 12163 leader_election.cc:343] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.609654 12314 leader_election.cc:290] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:04.609764 12320 raft_consensus.cc:697] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Becoming Leader. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:04.610956 12320 consensus_queue.cc:237] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.612056 12313 raft_consensus.cc:359] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } }
I20260709 13:06:04.612196 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16e10739a0e45b4ae22a2366e1162c3" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:04.613202 12270 raft_consensus.cc:2468] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 0.
I20260709 13:06:04.614507 12163 leader_election.cc:304] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 2780688fce76495ca9c1ff6e8e998046
I20260709 13:06:04.616101 12314 raft_consensus.cc:2804] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:04.616660 12314 raft_consensus.cc:493] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.617090 12314 raft_consensus.cc:3060] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.618197 12311 raft_consensus.cc:359] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.619267 12311 raft_consensus.cc:385] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.619623 12311 raft_consensus.cc:740] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.623687 12311 consensus_queue.cc:260] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.624812 12313 raft_consensus.cc:385] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.628399 12313 raft_consensus.cc:740] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.629482 12313 consensus_queue.cc:260] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } }
I20260709 13:06:04.636831 12313 ts_tablet_manager.cc:1434] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.003s
I20260709 13:06:04.638098 12313 tablet_bootstrap.cc:492] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:04.634557 12314 raft_consensus.cc:515] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.637255 12016 catalog_manager.cc:5697] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "2780688fce76495ca9c1ff6e8e998046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:04.641944 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16e10739a0e45b4ae22a2366e1162c3" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
W20260709 13:06:04.643596 12163 leader_election.cc:343] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.626986 12316 raft_consensus.cc:515] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.645721 12313 tablet_bootstrap.cc:654] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.645920 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16e10739a0e45b4ae22a2366e1162c3" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:04.646855 12270 raft_consensus.cc:3060] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.649091 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0833359f734033bfe7eab9cc74666f" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
W20260709 13:06:04.651223 12237 leader_election.cc:343] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.645566 12311 ts_tablet_manager.cc:1434] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.063s	user 0.009s	sys 0.001s
I20260709 13:06:04.652907 12323 raft_consensus.cc:493] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.653620 12323 raft_consensus.cc:515] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.659029 12314 leader_election.cc:290] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:04.660044 12270 raft_consensus.cc:2468] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 1.
I20260709 13:06:04.661955 12163 leader_election.cc:304] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 2780688fce76495ca9c1ff6e8e998046
I20260709 13:06:04.663977 12314 raft_consensus.cc:2804] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.665552 12323 leader_election.cc:290] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:04.666292 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89ecf901f543718a4f562df52c12ff" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:04.667059 12197 raft_consensus.cc:2468] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 0.
I20260709 13:06:04.667644 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89ecf901f543718a4f562df52c12ff" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:04.668426 12235 leader_election.cc:304] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
W20260709 13:06:04.669224 12237 leader_election.cc:343] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.669376 12323 raft_consensus.cc:2804] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:04.669836 12323 raft_consensus.cc:493] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.670197 12323 raft_consensus.cc:3060] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.671604 12314 raft_consensus.cc:697] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Becoming Leader. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Running, Role: LEADER
I20260709 13:06:04.673108 12314 consensus_queue.cc:237] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.675339 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0833359f734033bfe7eab9cc74666f" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:04.676062 12197 raft_consensus.cc:3060] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.678825 12323 raft_consensus.cc:515] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.682564 12310 tablet_bootstrap.cc:492] T d6bd5ec5c50c48b78c88d403817fbdae P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:04.682768 12323 leader_election.cc:290] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:04.683203 12310 ts_tablet_manager.cc:1403] T d6bd5ec5c50c48b78c88d403817fbdae P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.111s	user 0.025s	sys 0.055s
I20260709 13:06:04.684408 12196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89ecf901f543718a4f562df52c12ff" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:04.685348 12196 raft_consensus.cc:3060] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.686882 12310 raft_consensus.cc:359] T d6bd5ec5c50c48b78c88d403817fbdae P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.687729 12310 raft_consensus.cc:385] T d6bd5ec5c50c48b78c88d403817fbdae P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.687882 12197 raft_consensus.cc:2468] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 1.
I20260709 13:06:04.688033 12310 raft_consensus.cc:740] T d6bd5ec5c50c48b78c88d403817fbdae P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.679936 12316 leader_election.cc:290] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:04.689459 12235 leader_election.cc:304] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 2780688fce76495ca9c1ff6e8e998046
I20260709 13:06:04.690033 12320 raft_consensus.cc:493] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.692077 12323 raft_consensus.cc:2804] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.691905 12320 raft_consensus.cc:515] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.689316 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89ecf901f543718a4f562df52c12ff" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:04.699257 12313 tablet_bootstrap.cc:492] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:04.699003 12310 consensus_queue.cc:260] T d6bd5ec5c50c48b78c88d403817fbdae P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.699805 12313 ts_tablet_manager.cc:1403] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.062s	user 0.037s	sys 0.023s
I20260709 13:06:04.703032 12310 ts_tablet_manager.cc:1434] T d6bd5ec5c50c48b78c88d403817fbdae P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:06:04.702872 12196 raft_consensus.cc:2468] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 1.
I20260709 13:06:04.703965 12320 leader_election.cc:290] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:04.704380 12313 raft_consensus.cc:359] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.705165 12313 raft_consensus.cc:385] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.705466 12313 raft_consensus.cc:740] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.705760 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63915eb22a6f42e3875c55499d3cafab" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:04.705879 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63915eb22a6f42e3875c55499d3cafab" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:04.706590 12197 raft_consensus.cc:2468] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 0.
I20260709 13:06:04.707441 12313 consensus_queue.cc:260] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.707981 12310 tablet_bootstrap.cc:492] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:04.708305 12086 leader_election.cc:304] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:04.706598 12270 raft_consensus.cc:2468] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 0.
I20260709 13:06:04.711445 12320 raft_consensus.cc:2804] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:04.711709 12313 ts_tablet_manager.cc:1434] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:06:04.711892 12320 raft_consensus.cc:493] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.712262 12320 raft_consensus.cc:3060] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.712860 12313 tablet_bootstrap.cc:492] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
W20260709 13:06:04.702782 12237 leader_election.cc:343] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.718199 12235 leader_election.cc:304] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 2780688fce76495ca9c1ff6e8e998046
I20260709 13:06:04.718518 12310 tablet_bootstrap.cc:654] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.723349 12316 raft_consensus.cc:2804] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.724099 12316 raft_consensus.cc:697] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Becoming Leader. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:04.725286 12316 consensus_queue.cc:237] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.731324 12349 raft_consensus.cc:493] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.732095 12349 raft_consensus.cc:515] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.731454 12323 raft_consensus.cc:697] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Becoming Leader. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:04.736114 12323 consensus_queue.cc:237] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.734433 12320 raft_consensus.cc:515] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.739503 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd09f5dac3864c3da2319df9b220cc3f" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:04.740953 12320 leader_election.cc:290] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Requested vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:04.741629 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63915eb22a6f42e3875c55499d3cafab" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:04.742317 12270 raft_consensus.cc:3060] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:04.742439 12163 leader_election.cc:343] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.743410 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63915eb22a6f42e3875c55499d3cafab" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:04.744014 12197 raft_consensus.cc:3060] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.731366 12313 tablet_bootstrap.cc:654] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.749051 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd09f5dac3864c3da2319df9b220cc3f" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:04.738749 12017 catalog_manager.cc:5697] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:04.749737 12271 raft_consensus.cc:2468] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 0.
I20260709 13:06:04.751127 12270 raft_consensus.cc:2468] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 1.
I20260709 13:06:04.751490 12163 leader_election.cc:304] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 2780688fce76495ca9c1ff6e8e998046
I20260709 13:06:04.752398 12088 leader_election.cc:304] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 
I20260709 13:06:04.752681 12349 leader_election.cc:290] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:04.754160 12320 raft_consensus.cc:2804] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.754310 12314 raft_consensus.cc:2804] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:04.755084 12314 raft_consensus.cc:493] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.755323 12320 raft_consensus.cc:697] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Becoming Leader. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:04.755810 12314 raft_consensus.cc:3060] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.757118 12320 consensus_queue.cc:237] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.760228 12197 raft_consensus.cc:2468] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 1.
I20260709 13:06:04.758632 12313 tablet_bootstrap.cc:492] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:04.769834 12313 ts_tablet_manager.cc:1403] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.057s	user 0.017s	sys 0.000s
I20260709 13:06:04.769344 12314 raft_consensus.cc:515] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.772940 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd09f5dac3864c3da2319df9b220cc3f" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:04.773746 12314 leader_election.cc:290] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:04.773594 12313 raft_consensus.cc:359] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.774639 12313 raft_consensus.cc:385] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:06:04.774644 12163 leader_election.cc:343] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:04.774818 12310 tablet_bootstrap.cc:492] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:04.775133 12313 raft_consensus.cc:740] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.775766 12310 ts_tablet_manager.cc:1403] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.068s	user 0.021s	sys 0.003s
I20260709 13:06:04.776245 12313 consensus_queue.cc:260] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.778229 12310 raft_consensus.cc:359] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } }
I20260709 13:06:04.778805 12310 raft_consensus.cc:385] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.779047 12313 ts_tablet_manager.cc:1434] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:06:04.779106 12310 raft_consensus.cc:740] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.779740 12323 raft_consensus.cc:493] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.780306 12313 tablet_bootstrap.cc:492] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:04.780267 12323 raft_consensus.cc:515] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.780529 12310 consensus_queue.cc:260] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } }
I20260709 13:06:04.782902 12310 ts_tablet_manager.cc:1434] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:06:04.783345 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd09f5dac3864c3da2319df9b220cc3f" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:04.784042 12270 raft_consensus.cc:3060] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.784200 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106eefe703c24ccc96acae1f610d7fdc" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:04.785205 12197 raft_consensus.cc:2468] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 0.
I20260709 13:06:04.785375 12310 tablet_bootstrap.cc:492] T f38db537da264dd4adf4363864b96250 P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:04.786376 12235 leader_election.cc:304] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:04.787406 12316 raft_consensus.cc:2804] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:04.788036 12316 raft_consensus.cc:493] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.777858 12020 catalog_manager.cc:5697] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 437a699518154aa580c1ba0868048c05 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "437a699518154aa580c1ba0868048c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:04.788421 12316 raft_consensus.cc:3060] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.782801 12323 leader_election.cc:290] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:04.788450 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106eefe703c24ccc96acae1f610d7fdc" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:04.795382 12122 raft_consensus.cc:2468] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 0.
I20260709 13:06:04.799419 12316 raft_consensus.cc:515] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.799746 12270 raft_consensus.cc:2468] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 1.
I20260709 13:06:04.789192 12313 tablet_bootstrap.cc:654] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.802467 12163 leader_election.cc:304] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 2780688fce76495ca9c1ff6e8e998046
I20260709 13:06:04.804764 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106eefe703c24ccc96acae1f610d7fdc" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:04.805100 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106eefe703c24ccc96acae1f610d7fdc" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:04.805573 12316 leader_election.cc:290] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:04.805523 12122 raft_consensus.cc:3060] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.805876 12197 raft_consensus.cc:3060] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.811311 12313 tablet_bootstrap.cc:492] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:04.812127 12313 ts_tablet_manager.cc:1403] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.002s
I20260709 13:06:04.813712 12122 raft_consensus.cc:2468] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 1.
I20260709 13:06:04.811272 12314 raft_consensus.cc:2804] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.814551 12314 raft_consensus.cc:697] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Becoming Leader. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Running, Role: LEADER
I20260709 13:06:04.815200 12237 leader_election.cc:304] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 
I20260709 13:06:04.815594 12313 raft_consensus.cc:359] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.816256 12316 raft_consensus.cc:2804] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.815630 12314 consensus_queue.cc:237] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.816555 12313 raft_consensus.cc:385] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.817049 12316 raft_consensus.cc:697] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Becoming Leader. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:04.817404 12313 raft_consensus.cc:740] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.818137 12316 consensus_queue.cc:237] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.818572 12349 raft_consensus.cc:493] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.819353 12349 raft_consensus.cc:515] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } }
I20260709 13:06:04.821645 12349 leader_election.cc:290] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769), 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857)
I20260709 13:06:04.824324 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4f009c4594491582878ae8b1f5f21b" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:04.825145 12270 raft_consensus.cc:2468] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 0.
I20260709 13:06:04.826853 12163 leader_election.cc:304] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:04.817776 12020 catalog_manager.cc:5697] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 437a699518154aa580c1ba0868048c05 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "437a699518154aa580c1ba0868048c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:04.828758 12349 raft_consensus.cc:2804] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:04.829238 12349 raft_consensus.cc:493] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.829614 12349 raft_consensus.cc:3060] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.828034 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4f009c4594491582878ae8b1f5f21b" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:04.835326 12122 raft_consensus.cc:2468] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 0.
I20260709 13:06:04.835701 12197 raft_consensus.cc:2468] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 1.
I20260709 13:06:04.837675 12313 consensus_queue.cc:260] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.840673 12313 ts_tablet_manager.cc:1434] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:06:04.841821 12313 tablet_bootstrap.cc:492] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:04.846684 12349 raft_consensus.cc:515] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } }
I20260709 13:06:04.847525 12310 tablet_bootstrap.cc:654] T f38db537da264dd4adf4363864b96250 P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.850204 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4f009c4594491582878ae8b1f5f21b" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:04.851018 12270 raft_consensus.cc:3060] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.853430 12017 catalog_manager.cc:5697] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "2780688fce76495ca9c1ff6e8e998046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:04.859894 12270 raft_consensus.cc:2468] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 1.
I20260709 13:06:04.861356 12163 leader_election.cc:304] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:04.862383 12314 raft_consensus.cc:2804] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:04.862142 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4f009c4594491582878ae8b1f5f21b" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:04.863240 12122 raft_consensus.cc:3060] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:04.861987 12020 catalog_manager.cc:5697] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:04.868589 12310 tablet_bootstrap.cc:492] T f38db537da264dd4adf4363864b96250 P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:04.869287 12310 ts_tablet_manager.cc:1403] T f38db537da264dd4adf4363864b96250 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.084s	user 0.012s	sys 0.004s
I20260709 13:06:04.872478 12310 raft_consensus.cc:359] T f38db537da264dd4adf4363864b96250 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.873433 12310 raft_consensus.cc:385] T f38db537da264dd4adf4363864b96250 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.873802 12310 raft_consensus.cc:740] T f38db537da264dd4adf4363864b96250 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.874953 12349 leader_election.cc:290] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 election: Requested vote from peers 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769), 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857)
I20260709 13:06:04.874785 12310 consensus_queue.cc:260] T f38db537da264dd4adf4363864b96250 P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.858343 12017 catalog_manager.cc:5697] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "2780688fce76495ca9c1ff6e8e998046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:04.882808 12310 ts_tablet_manager.cc:1434] T f38db537da264dd4adf4363864b96250 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:06:04.887303 12313 tablet_bootstrap.cc:654] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.889669 12122 raft_consensus.cc:2468] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 1.
I20260709 13:06:04.895308 12020 catalog_manager.cc:5697] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 437a699518154aa580c1ba0868048c05 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "437a699518154aa580c1ba0868048c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:04.900662 12313 tablet_bootstrap.cc:492] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:04.901484 12313 ts_tablet_manager.cc:1403] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.060s	user 0.016s	sys 0.004s
I20260709 13:06:04.903467 12314 raft_consensus.cc:697] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Becoming Leader. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Running, Role: LEADER
I20260709 13:06:04.904448 12314 consensus_queue.cc:237] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } }
I20260709 13:06:04.904736 12313 raft_consensus.cc:359] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.906580 12313 raft_consensus.cc:385] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.907065 12313 raft_consensus.cc:740] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.907934 12313 consensus_queue.cc:260] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.910188 12313 ts_tablet_manager.cc:1434] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:04.911289 12313 tablet_bootstrap.cc:492] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:04.915237 12017 catalog_manager.cc:5697] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:04.917827 12313 tablet_bootstrap.cc:654] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:04.922820 12313 tablet_bootstrap.cc:492] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:04.923354 12313 ts_tablet_manager.cc:1403] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.003s
I20260709 13:06:04.925645 12313 raft_consensus.cc:359] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.926429 12313 raft_consensus.cc:385] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:04.926708 12313 raft_consensus.cc:740] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:04.927294 12313 consensus_queue.cc:260] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:04.928891 12313 ts_tablet_manager.cc:1434] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:06:05.029285 12197 raft_consensus.cc:3060] T d6bd5ec5c50c48b78c88d403817fbdae P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.034173 12196 raft_consensus.cc:3060] T f38db537da264dd4adf4363864b96250 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.038004 12320 consensus_queue.cc:1048] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:05.042896 12320 consensus_queue.cc:1048] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.052024 12343 consensus_queue.cc:1048] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:05.068656 12337 consensus_queue.cc:1048] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:05.105052 12320 raft_consensus.cc:493] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.105568 12320 raft_consensus.cc:515] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:05.107532 12320 leader_election.cc:290] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:05.108553 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16e10739a0e45b4ae22a2366e1162c3" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:05.108434 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16e10739a0e45b4ae22a2366e1162c3" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:05.109465 12271 raft_consensus.cc:2393] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2780688fce76495ca9c1ff6e8e998046 in current term 1: Already voted for candidate 52d9dc2493034c6cbd8196d41f879ce8 in this term.
I20260709 13:06:05.110448 12088 leader_election.cc:304] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046; no voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:05.111232 12320 raft_consensus.cc:3060] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.116979 12320 raft_consensus.cc:2749] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:05.133571 12314 consensus_queue.cc:1048] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.135308 12314 consensus_queue.cc:1048] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.150693 12197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b4f009c4594491582878ae8b1f5f21b"
dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
mode: GRACEFUL
new_leader_uuid: "2780688fce76495ca9c1ff6e8e998046"
 from {username='slave'} at 127.0.0.1:33384
I20260709 13:06:05.151371 12197 raft_consensus.cc:606] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Received request to transfer leadership to TS 2780688fce76495ca9c1ff6e8e998046
I20260709 13:06:05.157591 12320 consensus_queue.cc:1048] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.173534 12337 consensus_queue.cc:1048] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:05.186709 12271 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b89ecf901f543718a4f562df52c12ff"
dest_uuid: "437a699518154aa580c1ba0868048c05"
mode: GRACEFUL
new_leader_uuid: "2780688fce76495ca9c1ff6e8e998046"
 from {username='slave'} at 127.0.0.1:36126
I20260709 13:06:05.187345 12271 raft_consensus.cc:606] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Received request to transfer leadership to TS 2780688fce76495ca9c1ff6e8e998046
I20260709 13:06:05.201730 12314 consensus_queue.cc:1048] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:05.202207 12337 raft_consensus.cc:493] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.203038 12337 raft_consensus.cc:515] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:05.205646 12337 leader_election.cc:290] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:05.206347 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0833359f734033bfe7eab9cc74666f" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:05.207969 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0833359f734033bfe7eab9cc74666f" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:05.208834 12197 raft_consensus.cc:2393] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2780688fce76495ca9c1ff6e8e998046 in current term 1: Already voted for candidate 437a699518154aa580c1ba0868048c05 in this term.
I20260709 13:06:05.213284 12086 leader_election.cc:304] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046; no voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:05.214155 12337 raft_consensus.cc:3060] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.218281 12122 raft_consensus.cc:3060] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.221132 12337 raft_consensus.cc:2749] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:05.222668 12121 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63915eb22a6f42e3875c55499d3cafab"
dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
mode: GRACEFUL
new_leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
 from {username='slave'} at 127.0.0.1:58942
I20260709 13:06:05.223428 12121 raft_consensus.cc:606] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Received request to transfer leadership to TS 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:05.227125 12365 consensus_queue.cc:1048] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:05.229413 12337 raft_consensus.cc:493] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.229470 12271 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "106eefe703c24ccc96acae1f610d7fdc"
dest_uuid: "437a699518154aa580c1ba0868048c05"
mode: GRACEFUL
new_leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
 from {username='slave'} at 127.0.0.1:36126
I20260709 13:06:05.230177 12337 raft_consensus.cc:515] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:05.230360 12271 raft_consensus.cc:606] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Received request to transfer leadership to TS 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:05.232443 12337 leader_election.cc:290] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:05.233713 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd09f5dac3864c3da2319df9b220cc3f" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:05.234041 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd09f5dac3864c3da2319df9b220cc3f" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:05.235769 12086 leader_election.cc:304] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046; no voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:05.236920 12382 raft_consensus.cc:2749] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:05.253950 12316 consensus_queue.cc:1048] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.265865 12389 consensus_queue.cc:1048] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:05.272241 12389 consensus_queue.cc:1048] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.274593 12382 raft_consensus.cc:493] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.275382 12316 raft_consensus.cc:993] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05: : Instructing follower 52d9dc2493034c6cbd8196d41f879ce8 to start an election
I20260709 13:06:05.275297 12382 raft_consensus.cc:515] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:05.275879 12316 raft_consensus.cc:1081] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Signalling peer 52d9dc2493034c6cbd8196d41f879ce8 to start an election
I20260709 13:06:05.278369 12196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89ecf901f543718a4f562df52c12ff" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:05.278889 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89ecf901f543718a4f562df52c12ff" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:05.280376 12195 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "106eefe703c24ccc96acae1f610d7fdc"
dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
 from {username='slave'} at 127.0.0.1:33368
I20260709 13:06:05.281298 12195 raft_consensus.cc:493] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:05.281458 12382 leader_election.cc:290] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:05.281750 12195 raft_consensus.cc:3060] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:05.282006 12088 leader_election.cc:304] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046; no voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:05.283596 12382 raft_consensus.cc:2749] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:05.286285 12122 raft_consensus.cc:3060] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:05.290620 12195 raft_consensus.cc:515] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:05.294106 12121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106eefe703c24ccc96acae1f610d7fdc" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:05.294834 12121 raft_consensus.cc:3060] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:05.295027 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106eefe703c24ccc96acae1f610d7fdc" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:05.295710 12271 raft_consensus.cc:3055] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:05.296036 12271 raft_consensus.cc:740] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:05.296695 12316 consensus_queue.cc:1048] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.296870 12271 consensus_queue.cc:260] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:05.298099 12271 raft_consensus.cc:3060] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:05.299253 12195 leader_election.cc:290] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 2 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:05.302953 12121 raft_consensus.cc:2468] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 2.
I20260709 13:06:05.304245 12163 leader_election.cc:304] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:05.305383 12365 raft_consensus.cc:2804] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:05.305963 12365 raft_consensus.cc:697] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 LEADER]: Becoming Leader. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Running, Role: LEADER
I20260709 13:06:05.309098 12389 raft_consensus.cc:993] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05: : Instructing follower 2780688fce76495ca9c1ff6e8e998046 to start an election
I20260709 13:06:05.309655 12389 raft_consensus.cc:1081] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Signalling peer 2780688fce76495ca9c1ff6e8e998046 to start an election
I20260709 13:06:05.311730 12122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b89ecf901f543718a4f562df52c12ff"
dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
 from {username='slave'} at 127.0.0.1:58926
I20260709 13:06:05.310801 12365 consensus_queue.cc:237] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:05.313014 12122 raft_consensus.cc:493] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:05.313441 12122 raft_consensus.cc:3060] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:05.315889 12316 consensus_queue.cc:1048] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.320362 12271 raft_consensus.cc:2468] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 2.
I20260709 13:06:05.321836 12122 raft_consensus.cc:515] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:05.321276 12017 catalog_manager.cc:5697] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 reported cstate change: term changed from 1 to 2, leader changed from 437a699518154aa580c1ba0868048c05 (127.0.107.195) to 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:05.325505 12365 consensus_queue.cc:1048] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.327708 12122 leader_election.cc:290] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 2 election: Requested vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:05.329821 12195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89ecf901f543718a4f562df52c12ff" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:05.330863 12195 raft_consensus.cc:3060] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:05.332304 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89ecf901f543718a4f562df52c12ff" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:05.332953 12270 raft_consensus.cc:3055] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:05.333300 12270 raft_consensus.cc:740] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:05.334112 12270 consensus_queue.cc:260] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:05.335374 12270 raft_consensus.cc:3060] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:05.342516 12270 raft_consensus.cc:2468] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 2.
I20260709 13:06:05.342581 12195 raft_consensus.cc:2468] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 2.
I20260709 13:06:05.343819 12088 leader_election.cc:304] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 
I20260709 13:06:05.346591 12382 raft_consensus.cc:2804] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:05.347230 12382 raft_consensus.cc:697] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Becoming Leader. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:05.348544 12382 consensus_queue.cc:237] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:05.352926 12323 consensus_queue.cc:1048] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:05.363374 12349 consensus_queue.cc:1048] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:05.375283 12017 catalog_manager.cc:5697] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 reported cstate change: term changed from 1 to 2, leader changed from 437a699518154aa580c1ba0868048c05 (127.0.107.195) to 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "2780688fce76495ca9c1ff6e8e998046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:05.379899 12365 raft_consensus.cc:993] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8: : Instructing follower 2780688fce76495ca9c1ff6e8e998046 to start an election
I20260709 13:06:05.380463 12349 raft_consensus.cc:1081] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Signalling peer 2780688fce76495ca9c1ff6e8e998046 to start an election
I20260709 13:06:05.382237 12122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b4f009c4594491582878ae8b1f5f21b"
dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
 from {username='slave'} at 127.0.0.1:58934
I20260709 13:06:05.382841 12122 raft_consensus.cc:493] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:05.383240 12122 raft_consensus.cc:3060] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:05.391628 12122 raft_consensus.cc:515] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } }
I20260709 13:06:05.393801 12122 leader_election.cc:290] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 2 election: Requested vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:05.394624 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4f009c4594491582878ae8b1f5f21b" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:05.395005 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b4f009c4594491582878ae8b1f5f21b" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:05.395500 12197 raft_consensus.cc:3055] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:05.395669 12271 raft_consensus.cc:3060] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:05.395906 12197 raft_consensus.cc:740] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Running, Role: LEADER
I20260709 13:06:05.396875 12197 consensus_queue.cc:260] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } }
I20260709 13:06:05.398180 12197 raft_consensus.cc:3060] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:05.403524 12271 raft_consensus.cc:2468] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 2.
I20260709 13:06:05.404912 12088 leader_election.cc:304] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 
I20260709 13:06:05.405721 12197 raft_consensus.cc:2468] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 2.
I20260709 13:06:05.405782 12382 raft_consensus.cc:2804] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:05.406574 12382 raft_consensus.cc:697] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Becoming Leader. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:05.407344 12382 consensus_queue.cc:237] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } }
I20260709 13:06:05.413606 12016 catalog_manager.cc:5697] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 reported cstate change: term changed from 1 to 2, leader changed from 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194) to 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "2780688fce76495ca9c1ff6e8e998046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:05.704787 12337 raft_consensus.cc:993] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046: : Instructing follower 52d9dc2493034c6cbd8196d41f879ce8 to start an election
I20260709 13:06:05.705289 12382 raft_consensus.cc:1081] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Signalling peer 52d9dc2493034c6cbd8196d41f879ce8 to start an election
I20260709 13:06:05.707224 12197 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "63915eb22a6f42e3875c55499d3cafab"
dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
 from {username='slave'} at 127.0.0.1:33378
I20260709 13:06:05.707775 12197 raft_consensus.cc:493] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:05.708138 12197 raft_consensus.cc:3060] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:05.714763 12197 raft_consensus.cc:515] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:05.716653 12197 leader_election.cc:290] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 2 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:05.717599 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63915eb22a6f42e3875c55499d3cafab" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:05.718075 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63915eb22a6f42e3875c55499d3cafab" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:05.718461 12122 raft_consensus.cc:3055] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:05.718794 12271 raft_consensus.cc:3060] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:05.718861 12122 raft_consensus.cc:740] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:05.719937 12122 consensus_queue.cc:260] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:05.721392 12122 raft_consensus.cc:3060] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:05.727437 12271 raft_consensus.cc:2468] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 2.
I20260709 13:06:05.729111 12163 leader_election.cc:304] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:05.729496 12122 raft_consensus.cc:2468] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 2.
I20260709 13:06:05.729907 12364 raft_consensus.cc:2804] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:05.731503 12364 raft_consensus.cc:697] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 LEADER]: Becoming Leader. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Running, Role: LEADER
I20260709 13:06:05.732441 12364 consensus_queue.cc:237] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:05.732949 12122 raft_consensus.cc:1275] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Refusing update from remote peer 52d9dc2493034c6cbd8196d41f879ce8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:05.733207 12271 raft_consensus.cc:1275] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Refusing update from remote peer 52d9dc2493034c6cbd8196d41f879ce8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:05.734834 12397 consensus_queue.cc:1048] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:05.735535 12349 consensus_queue.cc:1048] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:05.747524 12016 catalog_manager.cc:5697] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 reported cstate change: term changed from 1 to 2, leader changed from 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193) to 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:05.776667 12271 raft_consensus.cc:1275] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Refusing update from remote peer 2780688fce76495ca9c1ff6e8e998046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:05.778250 12382 consensus_queue.cc:1048] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:05.787019 12197 raft_consensus.cc:1275] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Refusing update from remote peer 2780688fce76495ca9c1ff6e8e998046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:05.789072 12320 consensus_queue.cc:1048] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:05.791266 12197 raft_consensus.cc:1275] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Refusing update from remote peer 2780688fce76495ca9c1ff6e8e998046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:05.795792 12398 consensus_queue.cc:1048] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:05.804833 12271 raft_consensus.cc:1275] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Refusing update from remote peer 2780688fce76495ca9c1ff6e8e998046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:05.806499 12337 consensus_queue.cc:1048] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:06.278302 12121 raft_consensus.cc:1275] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Refusing update from remote peer 52d9dc2493034c6cbd8196d41f879ce8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:06.279682 12401 consensus_queue.cc:1048] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:06.289459 12271 raft_consensus.cc:1275] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Refusing update from remote peer 52d9dc2493034c6cbd8196d41f879ce8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:06.291262 12401 consensus_queue.cc:1048] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:08.276894 12016 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52594:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:08.280401 12016 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:08.349823 12108 tablet_service.cc:1511] Processing CreateTablet for tablet 28e53bfdf42a4890830f03cd39081785 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:08.351209 12185 tablet_service.cc:1511] Processing CreateTablet for tablet 28e53bfdf42a4890830f03cd39081785 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:08.353183 12185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28e53bfdf42a4890830f03cd39081785. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.355659 12109 tablet_service.cc:1511] Processing CreateTablet for tablet a95812f5c040405ea3d02c1dc6de40a0 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:08.356302 12256 tablet_service.cc:1511] Processing CreateTablet for tablet 28e53bfdf42a4890830f03cd39081785 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:08.356608 12187 tablet_service.cc:1511] Processing CreateTablet for tablet a95812f5c040405ea3d02c1dc6de40a0 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:08.357420 12109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a95812f5c040405ea3d02c1dc6de40a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.357946 12256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28e53bfdf42a4890830f03cd39081785. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.358211 12187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a95812f5c040405ea3d02c1dc6de40a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.364945 12108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28e53bfdf42a4890830f03cd39081785. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.362543 12182 tablet_service.cc:1511] Processing CreateTablet for tablet d19809bed5a94e91bfaa191a19cc2074 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:08.365556 12254 tablet_service.cc:1511] Processing CreateTablet for tablet a95812f5c040405ea3d02c1dc6de40a0 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:08.365806 12253 tablet_service.cc:1511] Processing CreateTablet for tablet d19809bed5a94e91bfaa191a19cc2074 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:08.367147 12182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d19809bed5a94e91bfaa191a19cc2074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.367336 12254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a95812f5c040405ea3d02c1dc6de40a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.368353 12106 tablet_service.cc:1511] Processing CreateTablet for tablet d19809bed5a94e91bfaa191a19cc2074 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:08.370075 12106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d19809bed5a94e91bfaa191a19cc2074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.374980 12253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d19809bed5a94e91bfaa191a19cc2074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.382164 12105 tablet_service.cc:1511] Processing CreateTablet for tablet cd616c68dbc74070b03abb376f3f5151 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:08.384155 12105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd616c68dbc74070b03abb376f3f5151. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.394946 12184 tablet_service.cc:1511] Processing CreateTablet for tablet cd616c68dbc74070b03abb376f3f5151 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:08.396682 12184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd616c68dbc74070b03abb376f3f5151. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.399076 12107 tablet_service.cc:1511] Processing CreateTablet for tablet 6ef9bfc698934bd59128a2b6be47e79c (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:08.400565 12107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ef9bfc698934bd59128a2b6be47e79c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.401578 12186 tablet_service.cc:1511] Processing CreateTablet for tablet 6ef9bfc698934bd59128a2b6be47e79c (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:08.404022 12186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ef9bfc698934bd59128a2b6be47e79c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.404325 12258 tablet_service.cc:1511] Processing CreateTablet for tablet cd616c68dbc74070b03abb376f3f5151 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:08.404703 12259 tablet_service.cc:1511] Processing CreateTablet for tablet 6ef9bfc698934bd59128a2b6be47e79c (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:08.405851 12258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd616c68dbc74070b03abb376f3f5151. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.407116 12112 tablet_service.cc:1511] Processing CreateTablet for tablet 29ae0bd6628a4dd3857b9e898b25febe (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:08.408120 12183 tablet_service.cc:1511] Processing CreateTablet for tablet 29ae0bd6628a4dd3857b9e898b25febe (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:08.409198 12259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ef9bfc698934bd59128a2b6be47e79c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.409685 12183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29ae0bd6628a4dd3857b9e898b25febe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.408594 12112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29ae0bd6628a4dd3857b9e898b25febe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.416801 12255 tablet_service.cc:1511] Processing CreateTablet for tablet 29ae0bd6628a4dd3857b9e898b25febe (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:08.418499 12255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29ae0bd6628a4dd3857b9e898b25febe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.426860 12111 tablet_service.cc:1511] Processing CreateTablet for tablet 1731218f724f4b1da5dc847c9ff4a42e (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:08.428431 12111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1731218f724f4b1da5dc847c9ff4a42e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.429476 12180 tablet_service.cc:1511] Processing CreateTablet for tablet 1731218f724f4b1da5dc847c9ff4a42e (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:08.430969 12180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1731218f724f4b1da5dc847c9ff4a42e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.443221 12257 tablet_service.cc:1511] Processing CreateTablet for tablet 1731218f724f4b1da5dc847c9ff4a42e (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:08.444818 12257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1731218f724f4b1da5dc847c9ff4a42e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.463490 12104 tablet_service.cc:1511] Processing CreateTablet for tablet 26e1bbe181f94724b4f3375b6a418fb4 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:08.465011 12104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e1bbe181f94724b4f3375b6a418fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.470180 12181 tablet_service.cc:1511] Processing CreateTablet for tablet 26e1bbe181f94724b4f3375b6a418fb4 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:08.471894 12181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e1bbe181f94724b4f3375b6a418fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.494660 12256 tablet_service.cc:1511] Processing CreateTablet for tablet 26e1bbe181f94724b4f3375b6a418fb4 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
W20260709 13:06:08.495826 12068 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:08.496307 12256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e1bbe181f94724b4f3375b6a418fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.501032 12110 tablet_service.cc:1511] Processing CreateTablet for tablet e18d1cfcb6cb4d26b5c374b738541e08 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:08.502636 12110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e18d1cfcb6cb4d26b5c374b738541e08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.521183 12255 tablet_service.cc:1511] Processing CreateTablet for tablet e18d1cfcb6cb4d26b5c374b738541e08 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:08.522622 12255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e18d1cfcb6cb4d26b5c374b738541e08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.532172 12179 tablet_service.cc:1511] Processing CreateTablet for tablet e18d1cfcb6cb4d26b5c374b738541e08 (DEFAULT_TABLE table=soft_deleted_table [id=041ded360c494bfa8775544899cba37c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:08.533752 12179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e18d1cfcb6cb4d26b5c374b738541e08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:08.553601 12419 tablet_bootstrap.cc:492] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
W20260709 13:06:08.585186 12086 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:34543, user_credentials={real_user=slave}} blocked reactor thread for 60780.7us
I20260709 13:06:08.582847 12419 tablet_bootstrap.cc:654] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.595392 12421 tablet_bootstrap.cc:492] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:08.612036 12421 tablet_bootstrap.cc:654] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.613369 12422 tablet_bootstrap.cc:492] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:08.615978 12419 tablet_bootstrap.cc:492] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:08.616698 12419 ts_tablet_manager.cc:1403] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.064s	user 0.019s	sys 0.006s
I20260709 13:06:08.619917 12421 tablet_bootstrap.cc:492] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:08.620628 12421 ts_tablet_manager.cc:1403] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.000s
I20260709 13:06:08.620218 12419 raft_consensus.cc:359] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.621333 12422 tablet_bootstrap.cc:654] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.621239 12419 raft_consensus.cc:385] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.621793 12419 raft_consensus.cc:740] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.622715 12419 consensus_queue.cc:260] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.624140 12421 raft_consensus.cc:359] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.624984 12421 raft_consensus.cc:385] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.625314 12421 raft_consensus.cc:740] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.626521 12419 ts_tablet_manager.cc:1434] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:06:08.626212 12421 consensus_queue.cc:260] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.628043 12419 tablet_bootstrap.cc:492] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:08.629356 12421 ts_tablet_manager.cc:1434] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:06:08.630564 12421 tablet_bootstrap.cc:492] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:08.637754 12419 tablet_bootstrap.cc:654] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.655265 12400 raft_consensus.cc:493] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.655980 12400 raft_consensus.cc:515] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.659484 12422 tablet_bootstrap.cc:492] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:08.660264 12422 ts_tablet_manager.cc:1403] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.047s	user 0.017s	sys 0.020s
I20260709 13:06:08.660238 12195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95812f5c040405ea3d02c1dc6de40a0" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:08.660771 12421 tablet_bootstrap.cc:654] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:08.662261 12235 leader_election.cc:343] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:08.663295 12400 leader_election.cc:290] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:08.663869 12422 raft_consensus.cc:359] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.664822 12422 raft_consensus.cc:385] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.664676 12121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95812f5c040405ea3d02c1dc6de40a0" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:08.665251 12422 raft_consensus.cc:740] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.666492 12422 consensus_queue.cc:260] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
W20260709 13:06:08.667042 12237 leader_election.cc:343] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:08.667618 12237 leader_election.cc:304] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 437a699518154aa580c1ba0868048c05; no voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:08.669109 12400 raft_consensus.cc:2749] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:08.669914 12422 ts_tablet_manager.cc:1434] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:06:08.671120 12422 tablet_bootstrap.cc:492] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:08.672793 12421 tablet_bootstrap.cc:492] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:08.673394 12421 ts_tablet_manager.cc:1403] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.000s
I20260709 13:06:08.676681 12421 raft_consensus.cc:359] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.677242 12419 tablet_bootstrap.cc:492] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:08.678128 12422 tablet_bootstrap.cc:654] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.679557 12419 ts_tablet_manager.cc:1403] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.052s	user 0.017s	sys 0.013s
I20260709 13:06:08.677520 12421 raft_consensus.cc:385] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.681730 12421 raft_consensus.cc:740] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.682551 12421 consensus_queue.cc:260] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.684762 12421 ts_tablet_manager.cc:1434] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:06:08.685571 12422 tablet_bootstrap.cc:492] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:08.686137 12422 ts_tablet_manager.cc:1403] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.003s
I20260709 13:06:08.686161 12421 tablet_bootstrap.cc:492] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:08.683447 12419 raft_consensus.cc:359] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.687057 12419 raft_consensus.cc:385] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.687450 12419 raft_consensus.cc:740] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.688239 12419 consensus_queue.cc:260] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.695693 12422 raft_consensus.cc:359] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.696578 12422 raft_consensus.cc:385] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.696866 12422 raft_consensus.cc:740] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.698872 12422 consensus_queue.cc:260] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.700740 12419 ts_tablet_manager.cc:1434] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:06:08.701442 12422 ts_tablet_manager.cc:1434] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.015s	user 0.000s	sys 0.008s
I20260709 13:06:08.702548 12422 tablet_bootstrap.cc:492] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:08.703195 12421 tablet_bootstrap.cc:654] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.709957 12422 tablet_bootstrap.cc:654] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.709910 12419 tablet_bootstrap.cc:492] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:08.712244 12421 tablet_bootstrap.cc:492] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:08.712839 12421 ts_tablet_manager.cc:1403] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.004s
I20260709 13:06:08.716578 12421 raft_consensus.cc:359] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.717825 12421 raft_consensus.cc:385] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.718186 12421 raft_consensus.cc:740] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.719091 12421 consensus_queue.cc:260] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.720665 12419 tablet_bootstrap.cc:654] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.721807 12421 ts_tablet_manager.cc:1434] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:06:08.722826 12421 tablet_bootstrap.cc:492] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:08.726887 12422 tablet_bootstrap.cc:492] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:08.727496 12422 ts_tablet_manager.cc:1403] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.003s
I20260709 13:06:08.729950 12421 tablet_bootstrap.cc:654] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.730166 12422 raft_consensus.cc:359] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.730988 12422 raft_consensus.cc:385] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.731294 12422 raft_consensus.cc:740] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.732071 12422 consensus_queue.cc:260] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.734638 12422 ts_tablet_manager.cc:1434] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:06:08.735813 12422 tablet_bootstrap.cc:492] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:08.737622 12419 tablet_bootstrap.cc:492] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:08.738251 12419 ts_tablet_manager.cc:1403] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.003s
I20260709 13:06:08.742003 12419 raft_consensus.cc:359] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.742856 12419 raft_consensus.cc:385] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.743243 12419 raft_consensus.cc:740] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.743985 12422 tablet_bootstrap.cc:654] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.744087 12419 consensus_queue.cc:260] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.750067 12419 ts_tablet_manager.cc:1434] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 13:06:08.751441 12419 tablet_bootstrap.cc:492] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:08.759707 12419 tablet_bootstrap.cc:654] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.763805 12421 tablet_bootstrap.cc:492] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:08.764526 12421 ts_tablet_manager.cc:1403] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.014s
I20260709 13:06:08.772897 12400 raft_consensus.cc:493] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.773528 12400 raft_consensus.cc:515] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.771265 12421 raft_consensus.cc:359] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.777129 12421 raft_consensus.cc:385] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.777372 12195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28e53bfdf42a4890830f03cd39081785" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:08.777602 12421 raft_consensus.cc:740] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.781328 12400 leader_election.cc:290] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
W20260709 13:06:08.780946 12235 leader_election.cc:343] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:08.782104 12421 consensus_queue.cc:260] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.788558 12121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28e53bfdf42a4890830f03cd39081785" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:08.789484 12121 raft_consensus.cc:2468] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 0.
I20260709 13:06:08.792883 12421 ts_tablet_manager.cc:1434] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.003s
I20260709 13:06:08.793295 12422 tablet_bootstrap.cc:492] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:08.793418 12419 tablet_bootstrap.cc:492] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:08.794052 12421 tablet_bootstrap.cc:492] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:08.794118 12422 ts_tablet_manager.cc:1403] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.059s	user 0.018s	sys 0.024s
I20260709 13:06:08.794216 12419 ts_tablet_manager.cc:1403] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.043s	user 0.018s	sys 0.006s
I20260709 13:06:08.791254 12237 leader_election.cc:304] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:08.797855 12400 raft_consensus.cc:2804] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:08.798245 12400 raft_consensus.cc:493] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.798570 12400 raft_consensus.cc:3060] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:08.802901 12421 tablet_bootstrap.cc:654] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.803503 12419 raft_consensus.cc:359] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.804355 12419 raft_consensus.cc:385] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.804708 12419 raft_consensus.cc:740] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.805033 12422 raft_consensus.cc:359] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.805928 12422 raft_consensus.cc:385] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.805631 12419 consensus_queue.cc:260] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.806383 12422 raft_consensus.cc:740] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.808429 12419 ts_tablet_manager.cc:1434] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:06:08.807837 12422 consensus_queue.cc:260] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.809760 12419 tablet_bootstrap.cc:492] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:08.810931 12422 ts_tablet_manager.cc:1434] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:06:08.812201 12422 tablet_bootstrap.cc:492] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:08.817983 12419 tablet_bootstrap.cc:654] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.820742 12422 tablet_bootstrap.cc:654] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.821519 12421 tablet_bootstrap.cc:492] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:08.822173 12421 ts_tablet_manager.cc:1403] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.004s
I20260709 13:06:08.826395 12421 raft_consensus.cc:359] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.827863 12421 raft_consensus.cc:385] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.828325 12421 raft_consensus.cc:740] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.829576 12421 consensus_queue.cc:260] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.832170 12419 tablet_bootstrap.cc:492] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:08.832829 12419 ts_tablet_manager.cc:1403] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.003s
I20260709 13:06:08.836004 12419 raft_consensus.cc:359] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.836925 12419 raft_consensus.cc:385] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.837294 12419 raft_consensus.cc:740] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.838165 12419 consensus_queue.cc:260] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.838745 12421 ts_tablet_manager.cc:1434] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 13:06:08.840574 12421 tablet_bootstrap.cc:492] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:08.842000 12419 ts_tablet_manager.cc:1434] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:06:08.841864 12420 raft_consensus.cc:493] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.843398 12419 tablet_bootstrap.cc:492] T d19809bed5a94e91bfaa191a19cc2074 P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:08.842890 12420 raft_consensus.cc:515] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.848230 12400 raft_consensus.cc:515] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.848636 12195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d19809bed5a94e91bfaa191a19cc2074" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:08.849337 12195 raft_consensus.cc:2468] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 0.
I20260709 13:06:08.850888 12086 leader_election.cc:304] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:08.852205 12398 raft_consensus.cc:2804] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:08.852541 12398 raft_consensus.cc:493] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:08.852630 12237 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:35857, user_credentials={real_user=slave}} blocked reactor thread for 61458.8us
I20260709 13:06:08.852964 12398 raft_consensus.cc:3060] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:08.853521 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d19809bed5a94e91bfaa191a19cc2074" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:08.854365 12121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28e53bfdf42a4890830f03cd39081785" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:08.855089 12121 raft_consensus.cc:3060] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:08.856611 12088 leader_election.cc:343] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:08.857640 12420 leader_election.cc:290] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:08.863533 12398 raft_consensus.cc:515] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.866365 12422 tablet_bootstrap.cc:492] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:08.867100 12421 tablet_bootstrap.cc:654] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.867388 12422 ts_tablet_manager.cc:1403] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.055s	user 0.020s	sys 0.030s
I20260709 13:06:08.868034 12419 tablet_bootstrap.cc:654] T d19809bed5a94e91bfaa191a19cc2074 P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.868963 12121 raft_consensus.cc:2468] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 1.
I20260709 13:06:08.871582 12237 leader_election.cc:304] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 
I20260709 13:06:08.871455 12422 raft_consensus.cc:359] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.873034 12422 raft_consensus.cc:385] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.873667 12422 raft_consensus.cc:740] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.878402 12195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d19809bed5a94e91bfaa191a19cc2074" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:08.879081 12195 raft_consensus.cc:3060] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:08.879588 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d19809bed5a94e91bfaa191a19cc2074" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05"
W20260709 13:06:08.882089 12088 leader_election.cc:343] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:08.883000 12398 leader_election.cc:290] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Requested vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:08.884208 12422 consensus_queue.cc:260] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.888768 12195 raft_consensus.cc:2468] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 1.
I20260709 13:06:08.889230 12436 raft_consensus.cc:2804] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:08.889741 12419 tablet_bootstrap.cc:492] T d19809bed5a94e91bfaa191a19cc2074 P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:08.891983 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28e53bfdf42a4890830f03cd39081785" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:08.890437 12433 raft_consensus.cc:493] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.894840 12086 leader_election.cc:304] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 437a699518154aa580c1ba0868048c05
I20260709 13:06:08.894987 12433 raft_consensus.cc:515] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
W20260709 13:06:08.897430 12235 leader_election.cc:343] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:08.897670 12398 raft_consensus.cc:2804] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:08.897987 12421 tablet_bootstrap.cc:492] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:08.897475 12433 leader_election.cc:290] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:08.898725 12421 ts_tablet_manager.cc:1403] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.059s	user 0.015s	sys 0.022s
I20260709 13:06:08.898161 12398 raft_consensus.cc:697] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Becoming Leader. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:08.899113 12121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95812f5c040405ea3d02c1dc6de40a0" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:08.899909 12121 raft_consensus.cc:2468] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 0.
I20260709 13:06:08.901376 12163 leader_election.cc:304] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:08.894095 12419 ts_tablet_manager.cc:1403] T d19809bed5a94e91bfaa191a19cc2074 P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.000s
I20260709 13:06:08.902411 12421 raft_consensus.cc:359] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.899914 12398 consensus_queue.cc:237] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.903437 12421 raft_consensus.cc:385] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.903858 12421 raft_consensus.cc:740] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.903334 12401 raft_consensus.cc:2804] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:08.905658 12401 raft_consensus.cc:493] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.905158 12421 consensus_queue.cc:260] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.906028 12401 raft_consensus.cc:3060] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:08.908320 12421 ts_tablet_manager.cc:1434] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 13:06:08.909550 12421 tablet_bootstrap.cc:492] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:08.910962 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95812f5c040405ea3d02c1dc6de40a0" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:08.911792 12270 raft_consensus.cc:2468] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 0.
I20260709 13:06:08.892771 12422 ts_tablet_manager.cc:1434] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.025s	user 0.002s	sys 0.009s
I20260709 13:06:08.916582 12422 tablet_bootstrap.cc:492] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:08.890008 12400 leader_election.cc:290] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:08.905169 12419 raft_consensus.cc:359] T d19809bed5a94e91bfaa191a19cc2074 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.918684 12419 raft_consensus.cc:385] T d19809bed5a94e91bfaa191a19cc2074 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.919009 12419 raft_consensus.cc:740] T d19809bed5a94e91bfaa191a19cc2074 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.921936 12421 tablet_bootstrap.cc:654] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.914562 12401 raft_consensus.cc:515] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.925554 12401 leader_election.cc:290] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:08.921156 12419 consensus_queue.cc:260] T d19809bed5a94e91bfaa191a19cc2074 P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.926944 12121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95812f5c040405ea3d02c1dc6de40a0" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:08.927703 12121 raft_consensus.cc:3060] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:08.927950 12020 catalog_manager.cc:5697] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "2780688fce76495ca9c1ff6e8e998046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:08.928982 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95812f5c040405ea3d02c1dc6de40a0" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:08.929845 12270 raft_consensus.cc:3060] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:08.933255 12419 ts_tablet_manager.cc:1434] T d19809bed5a94e91bfaa191a19cc2074 P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:06:08.934535 12419 tablet_bootstrap.cc:492] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:08.937085 12121 raft_consensus.cc:2468] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 1.
I20260709 13:06:08.940373 12163 leader_election.cc:304] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:08.942016 12400 raft_consensus.cc:493] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.942613 12400 raft_consensus.cc:515] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.943363 12401 raft_consensus.cc:2804] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:08.944383 12401 raft_consensus.cc:697] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Becoming Leader. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Running, Role: LEADER
I20260709 13:06:08.945098 12419 tablet_bootstrap.cc:654] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.944932 12400 leader_election.cc:290] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:08.948563 12398 raft_consensus.cc:493] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.949100 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1731218f724f4b1da5dc847c9ff4a42e" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:08.949251 12398 raft_consensus.cc:515] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.950219 12197 raft_consensus.cc:2468] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 0.
I20260709 13:06:08.951762 12422 tablet_bootstrap.cc:654] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:08.945575 12401 consensus_queue.cc:237] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.953035 12421 tablet_bootstrap.cc:492] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:08.953564 12421 ts_tablet_manager.cc:1403] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.044s	user 0.010s	sys 0.024s
I20260709 13:06:08.954646 12420 raft_consensus.cc:493] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.957386 12436 raft_consensus.cc:697] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Becoming Leader. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:08.959159 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1731218f724f4b1da5dc847c9ff4a42e" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:08.960909 12197 raft_consensus.cc:2468] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 0.
I20260709 13:06:08.960477 12121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1731218f724f4b1da5dc847c9ff4a42e" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:08.975330 12086 leader_election.cc:304] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:08.961264 12436 consensus_queue.cc:237] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.976053 12121 raft_consensus.cc:2468] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 0.
I20260709 13:06:08.979349 12450 raft_consensus.cc:2804] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:08.979748 12450 raft_consensus.cc:493] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.980129 12450 raft_consensus.cc:3060] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:08.980495 12421 raft_consensus.cc:359] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.981349 12421 raft_consensus.cc:385] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:08.981678 12421 raft_consensus.cc:740] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:08.982439 12421 consensus_queue.cc:260] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:08.984475 12421 ts_tablet_manager.cc:1434] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.002s
I20260709 13:06:08.985497 12421 tablet_bootstrap.cc:492] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:08.964797 12398 leader_election.cc:290] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:08.958840 12235 leader_election.cc:304] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:08.959729 12420 raft_consensus.cc:515] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.000468 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1731218f724f4b1da5dc847c9ff4a42e" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:08.997575 12436 raft_consensus.cc:2804] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:09.002772 12271 raft_consensus.cc:2468] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 0.
I20260709 13:06:09.003216 12270 raft_consensus.cc:2468] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 1.
I20260709 13:06:09.008252 12436 raft_consensus.cc:493] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.008872 12436 raft_consensus.cc:3060] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.007936 12016 catalog_manager.cc:5697] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 437a699518154aa580c1ba0868048c05 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "437a699518154aa580c1ba0868048c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:09.013401 12420 leader_election.cc:290] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:09.019176 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9bfc698934bd59128a2b6be47e79c" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
W20260709 13:06:09.021000 12088 leader_election.cc:343] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:09.026325 12422 tablet_bootstrap.cc:492] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:09.027360 12422 ts_tablet_manager.cc:1403] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.111s	user 0.014s	sys 0.016s
I20260709 13:06:09.029243 12421 tablet_bootstrap.cc:654] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:09.031270 12419 tablet_bootstrap.cc:492] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:09.038300 12419 ts_tablet_manager.cc:1403] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.104s	user 0.019s	sys 0.008s
I20260709 13:06:09.039371 12450 raft_consensus.cc:515] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.044960 12450 leader_election.cc:290] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Requested vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
W20260709 13:06:09.044997 12086 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:34543, user_credentials={real_user=slave}} blocked reactor thread for 75427.8us
I20260709 13:06:09.045416 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1731218f724f4b1da5dc847c9ff4a42e" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:09.048187 12436 raft_consensus.cc:515] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.048378 12195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9bfc698934bd59128a2b6be47e79c" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:09.049340 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1731218f724f4b1da5dc847c9ff4a42e" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:09.050031 12197 raft_consensus.cc:3060] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.050334 12269 raft_consensus.cc:2393] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2780688fce76495ca9c1ff6e8e998046 in current term 1: Already voted for candidate 437a699518154aa580c1ba0868048c05 in this term.
I20260709 13:06:09.053128 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1731218f724f4b1da5dc847c9ff4a42e" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:09.053960 12122 raft_consensus.cc:2393] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 437a699518154aa580c1ba0868048c05 in current term 1: Already voted for candidate 2780688fce76495ca9c1ff6e8e998046 in this term.
W20260709 13:06:09.054503 12086 leader_election.cc:343] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:09.055048 12086 leader_election.cc:304] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046; no voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:09.056802 12453 raft_consensus.cc:2749] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:09.057314 12017 catalog_manager.cc:5697] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.060048 12419 raft_consensus.cc:359] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.060715 12419 raft_consensus.cc:385] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:09.061025 12419 raft_consensus.cc:740] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:09.061800 12419 consensus_queue.cc:260] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.064039 12419 ts_tablet_manager.cc:1434] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.024s	user 0.002s	sys 0.004s
I20260709 13:06:09.064677 12195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1731218f724f4b1da5dc847c9ff4a42e" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:09.065050 12419 tablet_bootstrap.cc:492] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:09.066165 12235 leader_election.cc:304] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 437a699518154aa580c1ba0868048c05; no voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:09.067282 12442 raft_consensus.cc:2749] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:09.069075 12422 raft_consensus.cc:359] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.070817 12422 raft_consensus.cc:385] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:09.071249 12422 raft_consensus.cc:740] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:09.072106 12422 consensus_queue.cc:260] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.072883 12419 tablet_bootstrap.cc:654] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:09.074638 12422 ts_tablet_manager.cc:1434] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.001s
I20260709 13:06:09.075857 12422 tablet_bootstrap.cc:492] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:09.078640 12419 tablet_bootstrap.cc:492] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:09.070319 12436 leader_election.cc:290] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:09.079246 12419 ts_tablet_manager.cc:1403] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.004s
I20260709 13:06:09.082351 12421 tablet_bootstrap.cc:492] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:09.082305 12419 raft_consensus.cc:359] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.083005 12421 ts_tablet_manager.cc:1403] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.098s	user 0.007s	sys 0.007s
I20260709 13:06:09.083153 12419 raft_consensus.cc:385] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:09.083509 12419 raft_consensus.cc:740] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:09.084157 12197 raft_consensus.cc:2468] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 1.
I20260709 13:06:09.084208 12419 consensus_queue.cc:260] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.085548 12086 leader_election.cc:304] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 437a699518154aa580c1ba0868048c05
I20260709 13:06:09.086575 12419 ts_tablet_manager.cc:1434] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:09.087430 12453 raft_consensus.cc:2804] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:09.087960 12453 raft_consensus.cc:697] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Becoming Leader. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:09.088869 12421 raft_consensus.cc:359] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.088790 12453 consensus_queue.cc:237] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.089740 12421 raft_consensus.cc:385] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:09.090101 12421 raft_consensus.cc:740] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:09.090903 12421 consensus_queue.cc:260] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.092272 12422 tablet_bootstrap.cc:654] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:09.093245 12421 ts_tablet_manager.cc:1434] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:06:09.094422 12421 tablet_bootstrap.cc:492] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046: Bootstrap starting.
I20260709 13:06:09.095690 12419 tablet_bootstrap.cc:492] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05: Bootstrap starting.
I20260709 13:06:09.100769 12421 tablet_bootstrap.cc:654] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:09.105548 12452 raft_consensus.cc:493] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.106171 12452 raft_consensus.cc:515] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.109122 12452 leader_election.cc:290] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:09.110258 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e1bbe181f94724b4f3375b6a418fb4" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:09.111039 12122 raft_consensus.cc:2468] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 0.
I20260709 13:06:09.112479 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e1bbe181f94724b4f3375b6a418fb4" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
W20260709 13:06:09.114450 12163 leader_election.cc:343] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:09.115209 12163 leader_election.cc:304] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 437a699518154aa580c1ba0868048c05
I20260709 13:06:09.116101 12452 raft_consensus.cc:2804] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:09.115752 12017 catalog_manager.cc:5697] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "2780688fce76495ca9c1ff6e8e998046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.116569 12452 raft_consensus.cc:493] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.117048 12452 raft_consensus.cc:3060] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.118862 12422 tablet_bootstrap.cc:492] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:09.119585 12422 ts_tablet_manager.cc:1403] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.007s
I20260709 13:06:09.120445 12421 tablet_bootstrap.cc:492] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046: No bootstrap required, opened a new log
I20260709 13:06:09.120981 12421 ts_tablet_manager.cc:1403] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.003s
I20260709 13:06:09.122833 12419 tablet_bootstrap.cc:654] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:09.122797 12422 raft_consensus.cc:359] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.123910 12422 raft_consensus.cc:385] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:09.124266 12422 raft_consensus.cc:740] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:09.123993 12421 raft_consensus.cc:359] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.124797 12421 raft_consensus.cc:385] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:09.125068 12421 raft_consensus.cc:740] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Initialized, Role: FOLLOWER
I20260709 13:06:09.125252 12422 consensus_queue.cc:260] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.128741 12452 raft_consensus.cc:515] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.130336 12421 consensus_queue.cc:260] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.132151 12422 ts_tablet_manager.cc:1434] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:06:09.132431 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e1bbe181f94724b4f3375b6a418fb4" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:09.133157 12122 raft_consensus.cc:3060] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.133333 12421 ts_tablet_manager.cc:1434] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 13:06:09.133503 12422 tablet_bootstrap.cc:492] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:09.134824 12452 leader_election.cc:290] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:09.136739 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e1bbe181f94724b4f3375b6a418fb4" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05"
W20260709 13:06:09.138245 12163 leader_election.cc:343] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:09.142318 12419 tablet_bootstrap.cc:492] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05: No bootstrap required, opened a new log
I20260709 13:06:09.142966 12419 ts_tablet_manager.cc:1403] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05: Time spent bootstrapping tablet: real 0.048s	user 0.016s	sys 0.000s
I20260709 13:06:09.143060 12422 tablet_bootstrap.cc:654] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:09.143004 12453 raft_consensus.cc:493] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.144335 12453 raft_consensus.cc:515] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.146348 12419 raft_consensus.cc:359] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.147177 12419 raft_consensus.cc:385] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:09.147467 12419 raft_consensus.cc:740] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Initialized, Role: FOLLOWER
I20260709 13:06:09.148263 12419 consensus_queue.cc:260] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.150697 12419 ts_tablet_manager.cc:1434] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:06:09.151890 12436 raft_consensus.cc:493] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.152206 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29ae0bd6628a4dd3857b9e898b25febe" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:09.152499 12436 raft_consensus.cc:515] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.153036 12453 leader_election.cc:290] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:09.154717 12436 leader_election.cc:290] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:09.155520 12121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd616c68dbc74070b03abb376f3f5151" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:09.156394 12121 raft_consensus.cc:2468] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 0.
W20260709 13:06:09.157774 12086 leader_election.cc:343] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:09.157838 12237 leader_election.cc:304] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 
I20260709 13:06:09.158108 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29ae0bd6628a4dd3857b9e898b25febe" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:09.158798 12269 raft_consensus.cc:2468] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 0.
I20260709 13:06:09.159842 12122 raft_consensus.cc:2468] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 1.
I20260709 13:06:09.160128 12088 leader_election.cc:304] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:09.160755 12442 raft_consensus.cc:2804] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:09.161162 12442 raft_consensus.cc:493] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.161499 12442 raft_consensus.cc:3060] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.161715 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd616c68dbc74070b03abb376f3f5151" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:09.161873 12163 leader_election.cc:304] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 437a699518154aa580c1ba0868048c05
I20260709 13:06:09.162485 12197 raft_consensus.cc:2468] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 0.
I20260709 13:06:09.163275 12452 raft_consensus.cc:2804] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:09.163832 12452 raft_consensus.cc:697] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Becoming Leader. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Running, Role: LEADER
I20260709 13:06:09.164731 12452 consensus_queue.cc:237] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.166954 12450 raft_consensus.cc:2804] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:09.167423 12450 raft_consensus.cc:493] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.167850 12450 raft_consensus.cc:3060] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.168182 12422 tablet_bootstrap.cc:492] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:09.168725 12422 ts_tablet_manager.cc:1403] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.002s
I20260709 13:06:09.175777 12422 raft_consensus.cc:359] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.176430 12422 raft_consensus.cc:385] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:09.176013 12442 raft_consensus.cc:515] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.176692 12422 raft_consensus.cc:740] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:09.178674 12422 consensus_queue.cc:260] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.180999 12422 ts_tablet_manager.cc:1434] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 13:06:09.180624 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd616c68dbc74070b03abb376f3f5151" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:09.182662 12122 raft_consensus.cc:3060] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.183157 12422 tablet_bootstrap.cc:492] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8: Bootstrap starting.
I20260709 13:06:09.183570 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd616c68dbc74070b03abb376f3f5151" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:09.184479 12197 raft_consensus.cc:3060] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.181479 12442 leader_election.cc:290] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:09.178776 12450 raft_consensus.cc:515] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.189610 12122 raft_consensus.cc:2468] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 1.
I20260709 13:06:09.190953 12237 leader_election.cc:304] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 
I20260709 13:06:09.192274 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29ae0bd6628a4dd3857b9e898b25febe" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:09.192966 12269 raft_consensus.cc:3060] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.192982 12195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29ae0bd6628a4dd3857b9e898b25febe" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:09.193812 12195 raft_consensus.cc:3060] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.194435 12450 leader_election.cc:290] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Requested vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:09.194857 12422 tablet_bootstrap.cc:654] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:09.197526 12442 raft_consensus.cc:2804] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:09.198165 12442 raft_consensus.cc:697] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Becoming Leader. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:09.199067 12442 consensus_queue.cc:237] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.202090 12269 raft_consensus.cc:2468] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 1.
I20260709 13:06:09.202762 12436 raft_consensus.cc:493] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.203574 12436 raft_consensus.cc:515] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.207186 12436 leader_election.cc:290] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:09.207612 12088 leader_election.cc:304] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 
I20260709 13:06:09.208302 12422 tablet_bootstrap.cc:492] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8: No bootstrap required, opened a new log
I20260709 13:06:09.208467 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e1bbe181f94724b4f3375b6a418fb4" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:09.208847 12422 ts_tablet_manager.cc:1403] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.011s
I20260709 13:06:09.209390 12122 raft_consensus.cc:2393] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 437a699518154aa580c1ba0868048c05 in current term 1: Already voted for candidate 52d9dc2493034c6cbd8196d41f879ce8 in this term.
I20260709 13:06:09.208624 12450 raft_consensus.cc:2804] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:09.211191 12450 raft_consensus.cc:697] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Becoming Leader. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:09.210964 12020 catalog_manager.cc:5697] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 437a699518154aa580c1ba0868048c05 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "437a699518154aa580c1ba0868048c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:09.211988 12422 raft_consensus.cc:359] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.212740 12422 raft_consensus.cc:385] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:09.212303 12450 consensus_queue.cc:237] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.213016 12422 raft_consensus.cc:740] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Initialized, Role: FOLLOWER
I20260709 13:06:09.213806 12422 consensus_queue.cc:260] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.215952 12422 ts_tablet_manager.cc:1434] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:09.216243 12195 raft_consensus.cc:2468] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 1.
I20260709 13:06:09.206661 12017 catalog_manager.cc:5697] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.219130 12197 raft_consensus.cc:2468] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 1.
I20260709 13:06:09.224774 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e1bbe181f94724b4f3375b6a418fb4" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:09.226163 12235 leader_election.cc:304] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 437a699518154aa580c1ba0868048c05; no voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:09.227106 12436 raft_consensus.cc:3060] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.234689 12017 catalog_manager.cc:5697] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "2780688fce76495ca9c1ff6e8e998046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.238440 12436 raft_consensus.cc:2749] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:09.283318 12452 raft_consensus.cc:493] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.283922 12452 raft_consensus.cc:515] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.285725 12452 leader_election.cc:290] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:09.286480 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28e53bfdf42a4890830f03cd39081785" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:09.286787 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28e53bfdf42a4890830f03cd39081785" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:09.287411 12122 raft_consensus.cc:2393] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52d9dc2493034c6cbd8196d41f879ce8 in current term 1: Already voted for candidate 437a699518154aa580c1ba0868048c05 in this term.
I20260709 13:06:09.288583 12163 leader_election.cc:304] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52d9dc2493034c6cbd8196d41f879ce8; no voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05
I20260709 13:06:09.289295 12452 raft_consensus.cc:3060] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.290028 12269 raft_consensus.cc:3060] T d19809bed5a94e91bfaa191a19cc2074 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.295272 12452 raft_consensus.cc:2749] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:09.297102 12450 consensus_queue.cc:1048] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:09.308765 12450 consensus_queue.cc:1048] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:09.353235 12436 raft_consensus.cc:493] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.353818 12436 raft_consensus.cc:515] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.356068 12436 leader_election.cc:290] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:09.356782 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9bfc698934bd59128a2b6be47e79c" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046" is_pre_election: true
I20260709 13:06:09.357124 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9bfc698934bd59128a2b6be47e79c" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:09.357435 12122 raft_consensus.cc:2468] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 0.
I20260709 13:06:09.357800 12197 raft_consensus.cc:2468] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 0.
I20260709 13:06:09.358490 12237 leader_election.cc:304] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 
I20260709 13:06:09.359301 12436 raft_consensus.cc:2804] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:09.359651 12436 raft_consensus.cc:493] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.359897 12436 raft_consensus.cc:3060] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.360314 12450 raft_consensus.cc:493] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.360806 12450 raft_consensus.cc:515] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.362803 12450 leader_election.cc:290] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:09.363831 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d1cfcb6cb4d26b5c374b738541e08" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05" is_pre_election: true
I20260709 13:06:09.363876 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d1cfcb6cb4d26b5c374b738541e08" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8" is_pre_election: true
I20260709 13:06:09.364741 12269 raft_consensus.cc:2468] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 0.
I20260709 13:06:09.364753 12197 raft_consensus.cc:2468] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 0.
I20260709 13:06:09.365998 12088 leader_election.cc:304] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 
I20260709 13:06:09.366638 12436 raft_consensus.cc:515] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.367148 12450 raft_consensus.cc:2804] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:09.367637 12450 raft_consensus.cc:493] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.368002 12450 raft_consensus.cc:3060] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.369011 12436 leader_election.cc:290] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:09.369747 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9bfc698934bd59128a2b6be47e79c" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:09.370391 12122 raft_consensus.cc:3060] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.370764 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9bfc698934bd59128a2b6be47e79c" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:09.371451 12197 raft_consensus.cc:3060] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.374598 12450 raft_consensus.cc:515] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.376602 12436 consensus_queue.cc:1048] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:09.377825 12450 leader_election.cc:290] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Requested vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:09.377805 12433 consensus_queue.cc:1048] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:09.379376 12195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d1cfcb6cb4d26b5c374b738541e08" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:09.380052 12195 raft_consensus.cc:3060] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.380348 12197 raft_consensus.cc:2468] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 1.
I20260709 13:06:09.378506 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d1cfcb6cb4d26b5c374b738541e08" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:09.381428 12269 raft_consensus.cc:3060] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:09.381453 12235 leader_election.cc:304] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:09.382781 12442 raft_consensus.cc:2804] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:09.383317 12442 raft_consensus.cc:697] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Becoming Leader. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:09.384205 12442 consensus_queue.cc:237] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.386809 12122 raft_consensus.cc:2468] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 1.
I20260709 13:06:09.388185 12195 raft_consensus.cc:2468] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 1.
I20260709 13:06:09.389667 12086 leader_election.cc:304] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:09.391497 12269 raft_consensus.cc:2468] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 1.
I20260709 13:06:09.401885 12436 consensus_queue.cc:1048] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:09.404651 12420 raft_consensus.cc:2804] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:09.405238 12420 raft_consensus.cc:697] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Becoming Leader. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:09.406083 12420 consensus_queue.cc:237] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.414029 12452 consensus_queue.cc:1048] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:09.423075 12020 catalog_manager.cc:5697] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 437a699518154aa580c1ba0868048c05 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "437a699518154aa580c1ba0868048c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:09.433916 12016 catalog_manager.cc:5697] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "2780688fce76495ca9c1ff6e8e998046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.506191 12450 consensus_queue.cc:1048] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:09.525121 12453 consensus_queue.cc:1048] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:09.644510 12452 consensus_queue.cc:1048] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:09.653573 12271 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28e53bfdf42a4890830f03cd39081785"
dest_uuid: "437a699518154aa580c1ba0868048c05"
mode: GRACEFUL
new_leader_uuid: "2780688fce76495ca9c1ff6e8e998046"
 from {username='slave'} at 127.0.0.1:36126
I20260709 13:06:09.655424 12271 raft_consensus.cc:606] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Received request to transfer leadership to TS 2780688fce76495ca9c1ff6e8e998046
I20260709 13:06:09.656756 12401 consensus_queue.cc:1048] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:09.662457 12197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a95812f5c040405ea3d02c1dc6de40a0"
dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
mode: GRACEFUL
new_leader_uuid: "2780688fce76495ca9c1ff6e8e998046"
 from {username='slave'} at 127.0.0.1:33384
I20260709 13:06:09.663053 12197 raft_consensus.cc:606] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Received request to transfer leadership to TS 2780688fce76495ca9c1ff6e8e998046
I20260709 13:06:09.674321 12271 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd616c68dbc74070b03abb376f3f5151"
dest_uuid: "437a699518154aa580c1ba0868048c05"
mode: GRACEFUL
new_leader_uuid: "2780688fce76495ca9c1ff6e8e998046"
 from {username='slave'} at 127.0.0.1:36126
I20260709 13:06:09.674913 12271 raft_consensus.cc:606] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Received request to transfer leadership to TS 2780688fce76495ca9c1ff6e8e998046
I20260709 13:06:09.681068 12271 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ef9bfc698934bd59128a2b6be47e79c"
dest_uuid: "437a699518154aa580c1ba0868048c05"
mode: GRACEFUL
new_leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
 from {username='slave'} at 127.0.0.1:36126
I20260709 13:06:09.681857 12271 raft_consensus.cc:606] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Received request to transfer leadership to TS 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:09.688853 12122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29ae0bd6628a4dd3857b9e898b25febe"
dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
mode: GRACEFUL
new_leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
 from {username='slave'} at 127.0.0.1:58942
I20260709 13:06:09.689394 12122 raft_consensus.cc:606] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Received request to transfer leadership to TS 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:09.695340 12122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1731218f724f4b1da5dc847c9ff4a42e"
dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
mode: GRACEFUL
new_leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
 from {username='slave'} at 127.0.0.1:58942
I20260709 13:06:09.695870 12122 raft_consensus.cc:606] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Received request to transfer leadership to TS 52d9dc2493034c6cbd8196d41f879ce8
I20260709 13:06:09.704141 12122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e18d1cfcb6cb4d26b5c374b738541e08"
dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
mode: GRACEFUL
new_leader_uuid: "437a699518154aa580c1ba0868048c05"
 from {username='slave'} at 127.0.0.1:58942
I20260709 13:06:09.704679 12122 raft_consensus.cc:606] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Received request to transfer leadership to TS 437a699518154aa580c1ba0868048c05
I20260709 13:06:09.730301 12442 consensus_queue.cc:1048] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:09.743541 12442 consensus_queue.cc:1048] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:09.752580 12483 raft_consensus.cc:993] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05: : Instructing follower 2780688fce76495ca9c1ff6e8e998046 to start an election
I20260709 13:06:09.753234 12483 raft_consensus.cc:1081] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Signalling peer 2780688fce76495ca9c1ff6e8e998046 to start an election
I20260709 13:06:09.754887 12122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cd616c68dbc74070b03abb376f3f5151"
dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
 from {username='slave'} at 127.0.0.1:58926
I20260709 13:06:09.755566 12122 raft_consensus.cc:493] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:09.755968 12122 raft_consensus.cc:3060] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.762037 12122 raft_consensus.cc:515] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.764088 12122 leader_election.cc:290] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 2 election: Requested vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:09.765386 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd616c68dbc74070b03abb376f3f5151" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:09.765475 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd616c68dbc74070b03abb376f3f5151" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:09.766271 12271 raft_consensus.cc:3055] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:09.766384 12197 raft_consensus.cc:3060] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.766752 12271 raft_consensus.cc:740] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:09.767767 12271 consensus_queue.cc:260] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.769243 12271 raft_consensus.cc:3060] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.774724 12197 raft_consensus.cc:2468] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 2.
I20260709 13:06:09.776144 12086 leader_election.cc:304] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:09.776970 12271 raft_consensus.cc:2468] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 2.
I20260709 13:06:09.777163 12450 raft_consensus.cc:2804] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:09.777809 12450 raft_consensus.cc:697] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Becoming Leader. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:09.778644 12450 consensus_queue.cc:237] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.785243 12450 consensus_queue.cc:1048] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:09.786980 12020 catalog_manager.cc:5697] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 reported cstate change: term changed from 1 to 2, leader changed from 437a699518154aa580c1ba0868048c05 (127.0.107.195) to 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "2780688fce76495ca9c1ff6e8e998046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.791047 12450 consensus_queue.cc:1048] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:09.794950 12450 raft_consensus.cc:993] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046: : Instructing follower 52d9dc2493034c6cbd8196d41f879ce8 to start an election
I20260709 13:06:09.795447 12450 raft_consensus.cc:1081] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Signalling peer 52d9dc2493034c6cbd8196d41f879ce8 to start an election
I20260709 13:06:09.799734 12195 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29ae0bd6628a4dd3857b9e898b25febe"
dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
 from {username='slave'} at 127.0.0.1:33378
I20260709 13:06:09.800565 12195 raft_consensus.cc:493] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:09.800998 12195 raft_consensus.cc:3060] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.805809 12442 consensus_queue.cc:1048] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:09.808797 12195 raft_consensus.cc:515] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.813691 12197 raft_consensus.cc:1240] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Rejecting Update request from peer 2780688fce76495ca9c1ff6e8e998046 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:09.816293 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29ae0bd6628a4dd3857b9e898b25febe" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:09.816996 12122 raft_consensus.cc:3055] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:09.817080 12436 raft_consensus.cc:993] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05: : Instructing follower 52d9dc2493034c6cbd8196d41f879ce8 to start an election
I20260709 13:06:09.817349 12122 raft_consensus.cc:740] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:09.817756 12436 raft_consensus.cc:1081] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Signalling peer 52d9dc2493034c6cbd8196d41f879ce8 to start an election
I20260709 13:06:09.818296 12122 consensus_queue.cc:260] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.819200 12122 raft_consensus.cc:3060] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.827147 12400 consensus_queue.cc:1048] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:09.832118 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29ae0bd6628a4dd3857b9e898b25febe" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:09.833026 12271 raft_consensus.cc:3060] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.833101 12197 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ef9bfc698934bd59128a2b6be47e79c"
dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
 from {username='slave'} at 127.0.0.1:33368
I20260709 13:06:09.835460 12197 raft_consensus.cc:493] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:09.835873 12197 raft_consensus.cc:3060] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.842360 12398 consensus_queue.cc:1048] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:09.849308 12197 raft_consensus.cc:515] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.850323 12195 leader_election.cc:290] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 2 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:09.852315 12197 leader_election.cc:290] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 2 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:09.853757 12270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9bfc698934bd59128a2b6be47e79c" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:09.854259 12121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9bfc698934bd59128a2b6be47e79c" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:09.854516 12270 raft_consensus.cc:3055] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:09.854945 12270 raft_consensus.cc:740] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:09.855001 12121 raft_consensus.cc:3060] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.855952 12270 consensus_queue.cc:260] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.857172 12270 raft_consensus.cc:3060] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.864151 12420 raft_consensus.cc:993] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046: : Instructing follower 437a699518154aa580c1ba0868048c05 to start an election
I20260709 13:06:09.864732 12453 raft_consensus.cc:1081] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Signalling peer 437a699518154aa580c1ba0868048c05 to start an election
I20260709 13:06:09.864864 12121 raft_consensus.cc:2468] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 2.
I20260709 13:06:09.866820 12271 raft_consensus.cc:2468] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 2.
I20260709 13:06:09.867970 12122 raft_consensus.cc:2468] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 2.
I20260709 13:06:09.868144 12163 leader_election.cc:304] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:09.869154 12452 raft_consensus.cc:2804] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:09.869634 12452 raft_consensus.cc:697] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 LEADER]: Becoming Leader. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Running, Role: LEADER
I20260709 13:06:09.870471 12452 consensus_queue.cc:237] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.872810 12163 leader_election.cc:304] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:09.873904 12401 raft_consensus.cc:2804] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:09.874058 12269 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e18d1cfcb6cb4d26b5c374b738541e08"
dest_uuid: "437a699518154aa580c1ba0868048c05"
 from {username='slave'} at 127.0.0.1:36108
I20260709 13:06:09.874404 12401 raft_consensus.cc:697] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 LEADER]: Becoming Leader. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Running, Role: LEADER
I20260709 13:06:09.874058 12450 consensus_queue.cc:1048] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:09.875348 12401 consensus_queue.cc:237] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.875939 12269 raft_consensus.cc:493] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:09.876413 12269 raft_consensus.cc:3060] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.882908 12270 raft_consensus.cc:2468] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 2.
I20260709 13:06:09.889722 12269 raft_consensus.cc:515] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.896979 12269 leader_election.cc:290] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 2 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:09.897830 12121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d1cfcb6cb4d26b5c374b738541e08" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:09.898160 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e18d1cfcb6cb4d26b5c374b738541e08" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:09.898607 12121 raft_consensus.cc:3055] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:09.898681 12197 raft_consensus.cc:3060] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.899051 12121 raft_consensus.cc:740] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:09.899264 12020 catalog_manager.cc:5697] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 reported cstate change: term changed from 1 to 2, leader changed from 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193) to 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.899880 12121 consensus_queue.cc:260] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.901250 12121 raft_consensus.cc:3060] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.900873 12020 catalog_manager.cc:5697] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 reported cstate change: term changed from 1 to 2, leader changed from 437a699518154aa580c1ba0868048c05 (127.0.107.195) to 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:09.906211 12197 raft_consensus.cc:2468] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 2.
I20260709 13:06:09.907572 12235 leader_election.cc:304] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:09.908440 12483 raft_consensus.cc:2804] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:09.909247 12483 raft_consensus.cc:697] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [term 2 LEADER]: Becoming Leader. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:09.910264 12483 consensus_queue.cc:237] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.923786 12017 catalog_manager.cc:5697] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 reported cstate change: term changed from 1 to 2, leader changed from 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193) to 437a699518154aa580c1ba0868048c05 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "437a699518154aa580c1ba0868048c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:09.925280 12121 raft_consensus.cc:2468] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 2.
I20260709 13:06:09.955124 12452 raft_consensus.cc:993] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8: : Instructing follower 2780688fce76495ca9c1ff6e8e998046 to start an election
I20260709 13:06:09.955643 12452 raft_consensus.cc:1081] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Signalling peer 2780688fce76495ca9c1ff6e8e998046 to start an election
I20260709 13:06:09.957288 12121 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a95812f5c040405ea3d02c1dc6de40a0"
dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
 from {username='slave'} at 127.0.0.1:58934
I20260709 13:06:09.957880 12121 raft_consensus.cc:493] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:09.958261 12121 raft_consensus.cc:3060] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.963510 12121 raft_consensus.cc:515] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.965289 12121 leader_election.cc:290] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 2 election: Requested vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:09.966387 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95812f5c040405ea3d02c1dc6de40a0" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:09.966240 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95812f5c040405ea3d02c1dc6de40a0" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:09.967123 12269 raft_consensus.cc:3060] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.967243 12197 raft_consensus.cc:3055] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:09.967707 12197 raft_consensus.cc:740] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Running, Role: LEADER
I20260709 13:06:09.968334 12197 consensus_queue.cc:260] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.969195 12197 raft_consensus.cc:3060] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:09.972576 12269 raft_consensus.cc:2468] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 2.
I20260709 13:06:09.974198 12088 leader_election.cc:304] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 
I20260709 13:06:09.974567 12197 raft_consensus.cc:2468] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 2.
I20260709 13:06:09.974984 12398 raft_consensus.cc:2804] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:09.975734 12398 raft_consensus.cc:697] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Becoming Leader. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:09.976588 12398 consensus_queue.cc:237] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:09.984301 12017 catalog_manager.cc:5697] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 reported cstate change: term changed from 1 to 2, leader changed from 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194) to 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "2780688fce76495ca9c1ff6e8e998046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:10.027046 12488 raft_consensus.cc:993] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05: : Instructing follower 2780688fce76495ca9c1ff6e8e998046 to start an election
I20260709 13:06:10.027689 12483 raft_consensus.cc:1081] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Signalling peer 2780688fce76495ca9c1ff6e8e998046 to start an election
I20260709 13:06:10.029450 12121 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28e53bfdf42a4890830f03cd39081785"
dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
 from {username='slave'} at 127.0.0.1:58926
I20260709 13:06:10.029979 12121 raft_consensus.cc:493] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:10.030349 12121 raft_consensus.cc:3060] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:10.038630 12121 raft_consensus.cc:515] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:10.041343 12121 leader_election.cc:290] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 2 election: Requested vote from peers 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:10.041901 12269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28e53bfdf42a4890830f03cd39081785" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:10.042773 12269 raft_consensus.cc:3055] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:10.043233 12398 raft_consensus.cc:993] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046: : Instructing follower 52d9dc2493034c6cbd8196d41f879ce8 to start an election
I20260709 13:06:10.043161 12269 raft_consensus.cc:740] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:10.043797 12450 raft_consensus.cc:1081] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Signalling peer 52d9dc2493034c6cbd8196d41f879ce8 to start an election
I20260709 13:06:10.044791 12197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28e53bfdf42a4890830f03cd39081785" candidate_uuid: "2780688fce76495ca9c1ff6e8e998046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:10.044590 12269 consensus_queue.cc:260] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:10.045641 12195 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1731218f724f4b1da5dc847c9ff4a42e"
dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
 from {username='slave'} at 127.0.0.1:33378
I20260709 13:06:10.045991 12269 raft_consensus.cc:3060] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:10.046316 12195 raft_consensus.cc:493] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:10.046737 12195 raft_consensus.cc:3060] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:10.045380 12197 raft_consensus.cc:3060] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:10.053762 12269 raft_consensus.cc:2468] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 2.
I20260709 13:06:10.053970 12197 raft_consensus.cc:2468] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2780688fce76495ca9c1ff6e8e998046 in term 2.
I20260709 13:06:10.053794 12195 raft_consensus.cc:515] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:10.055557 12086 leader_election.cc:304] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:10.056541 12195 leader_election.cc:290] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 2 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 437a699518154aa580c1ba0868048c05 (127.0.107.195:32769)
I20260709 13:06:10.057257 12450 raft_consensus.cc:2804] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:10.057843 12271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1731218f724f4b1da5dc847c9ff4a42e" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "437a699518154aa580c1ba0868048c05"
I20260709 13:06:10.057940 12450 raft_consensus.cc:697] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Becoming Leader. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:10.058079 12121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1731218f724f4b1da5dc847c9ff4a42e" candidate_uuid: "52d9dc2493034c6cbd8196d41f879ce8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:10.058530 12271 raft_consensus.cc:3060] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:10.058789 12121 raft_consensus.cc:3055] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:10.059206 12121 raft_consensus.cc:740] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:10.059788 12450 consensus_queue.cc:237] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:10.060123 12121 consensus_queue.cc:260] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:10.061439 12121 raft_consensus.cc:3060] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:10.064692 12271 raft_consensus.cc:2468] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 2.
I20260709 13:06:10.065887 12163 leader_election.cc:304] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:10.066792 12452 raft_consensus.cc:2804] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:10.067306 12452 raft_consensus.cc:697] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 LEADER]: Becoming Leader. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Running, Role: LEADER
I20260709 13:06:10.068140 12452 consensus_queue.cc:237] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:10.068800 12121 raft_consensus.cc:2468] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52d9dc2493034c6cbd8196d41f879ce8 in term 2.
I20260709 13:06:10.069248 12017 catalog_manager.cc:5697] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 reported cstate change: term changed from 1 to 2, leader changed from 437a699518154aa580c1ba0868048c05 (127.0.107.195) to 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "2780688fce76495ca9c1ff6e8e998046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:10.077040 12020 catalog_manager.cc:5697] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 reported cstate change: term changed from 1 to 2, leader changed from 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193) to 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "52d9dc2493034c6cbd8196d41f879ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:10.162123 12195 raft_consensus.cc:1275] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Refusing update from remote peer 2780688fce76495ca9c1ff6e8e998046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.163471 12398 consensus_queue.cc:1048] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:10.171727 12269 raft_consensus.cc:1275] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Refusing update from remote peer 2780688fce76495ca9c1ff6e8e998046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.173195 12398 consensus_queue.cc:1048] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:10.324726 12195 raft_consensus.cc:1275] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Refusing update from remote peer 437a699518154aa580c1ba0868048c05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.326505 12483 consensus_queue.cc:1048] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:10.335822 12121 raft_consensus.cc:1275] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Refusing update from remote peer 437a699518154aa580c1ba0868048c05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.337431 12488 consensus_queue.cc:1048] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:10.380897 12271 raft_consensus.cc:1275] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Refusing update from remote peer 52d9dc2493034c6cbd8196d41f879ce8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.382309 12452 consensus_queue.cc:1048] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:10.390811 12121 raft_consensus.cc:1275] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Refusing update from remote peer 52d9dc2493034c6cbd8196d41f879ce8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.392328 12493 consensus_queue.cc:1048] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:10.403066 12121 raft_consensus.cc:1275] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Refusing update from remote peer 52d9dc2493034c6cbd8196d41f879ce8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.404728 12401 consensus_queue.cc:1048] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:10.418124 12271 raft_consensus.cc:1275] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Refusing update from remote peer 52d9dc2493034c6cbd8196d41f879ce8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.419648 12401 consensus_queue.cc:1048] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:10.514155 12121 raft_consensus.cc:1275] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Refusing update from remote peer 52d9dc2493034c6cbd8196d41f879ce8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.515802 12401 consensus_queue.cc:1048] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:10.525915 12271 raft_consensus.cc:1275] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Refusing update from remote peer 52d9dc2493034c6cbd8196d41f879ce8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.528239 12401 consensus_queue.cc:1048] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:10.533211 12269 raft_consensus.cc:1275] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Refusing update from remote peer 2780688fce76495ca9c1ff6e8e998046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.536371 12398 consensus_queue.cc:1048] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:10.552248 12195 raft_consensus.cc:1275] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Refusing update from remote peer 2780688fce76495ca9c1ff6e8e998046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.554603 12453 consensus_queue.cc:1048] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:10.565294 12271 raft_consensus.cc:1275] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Refusing update from remote peer 2780688fce76495ca9c1ff6e8e998046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.567210 12498 consensus_queue.cc:1048] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:10.582370 12195 raft_consensus.cc:1275] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Refusing update from remote peer 2780688fce76495ca9c1ff6e8e998046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:10.589308 12450 consensus_queue.cc:1048] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:12.708420 12020 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52594:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:06:12.728420 12020 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52594:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:06:12.783248   431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:06:12.952781 12547 tablet.cc:1724] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:12.956851 12548 tablet.cc:1724] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:12.965651 12549 tablet.cc:1724] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:12.984571 12550 tablet.cc:1724] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.000727 12557 tablet.cc:1724] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.002859 12559 tablet.cc:1724] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.038955 12568 tablet.cc:1724] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:13.049857 12163 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:32769, user_credentials={real_user=slave}} blocked reactor thread for 60065.4us
I20260709 13:06:13.053104 12548 tablet.cc:1724] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.068181 12550 tablet.cc:1724] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:13.056293 12088 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:32769, user_credentials={real_user=slave}} blocked reactor thread for 57722.4us
I20260709 13:06:13.082547 12565 tablet.cc:1724] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.099746 12571 tablet.cc:1724] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.091138 12559 tablet.cc:1724] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.111086 12568 tablet.cc:1724] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.123950 12569 tablet.cc:1724] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.126122 12194 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26e1bbe181f94724b4f3375b6a418fb4"
dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
mode: GRACEFUL
new_leader_uuid: "437a699518154aa580c1ba0868048c05"
 from {username='slave'} at 127.0.0.1:36168
I20260709 13:06:13.127127 12194 raft_consensus.cc:606] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Received request to transfer leadership to TS 437a699518154aa580c1ba0868048c05
I20260709 13:06:13.140105 12572 tablet.cc:1724] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.157613 12568 tablet.cc:1724] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.162762 12549 tablet.cc:1724] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.175724 12568 tablet.cc:1724] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.201627 12550 tablet.cc:1724] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.219789 12571 tablet.cc:1724] T d19809bed5a94e91bfaa191a19cc2074 P 437a699518154aa580c1ba0868048c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.224072 12572 tablet.cc:1724] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.225128 12569 tablet.cc:1724] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.187877 12547 tablet.cc:1724] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.225593 12550 tablet.cc:1724] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.209465 12573 tablet.cc:1724] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.226835 12559 tablet.cc:1724] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.248446 12549 tablet.cc:1724] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:13.271816 12120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd616c68dbc74070b03abb376f3f5151"
dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
mode: GRACEFUL
new_leader_uuid: "437a699518154aa580c1ba0868048c05"
 from {username='slave'} at 127.0.0.1:35088
I20260709 13:06:13.272568 12120 raft_consensus.cc:606] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Received request to transfer leadership to TS 437a699518154aa580c1ba0868048c05
I20260709 13:06:13.274317   431 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:06:13.275049   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:13.286584 12120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b89ecf901f543718a4f562df52c12ff"
dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
mode: GRACEFUL
new_leader_uuid: "437a699518154aa580c1ba0868048c05"
 from {username='slave'} at 127.0.0.1:35088
I20260709 13:06:13.287446 12120 raft_consensus.cc:606] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Received request to transfer leadership to TS 437a699518154aa580c1ba0868048c05
I20260709 13:06:13.291095 12196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd09f5dac3864c3da2319df9b220cc3f"
dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
mode: GRACEFUL
new_leader_uuid: "437a699518154aa580c1ba0868048c05"
 from {username='slave'} at 127.0.0.1:36168
I20260709 13:06:13.291872 12196 raft_consensus.cc:606] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Received request to transfer leadership to TS 437a699518154aa580c1ba0868048c05
I20260709 13:06:13.293697   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:13.294502   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:13.291814 12016 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c: soft_deleted_table [id=041ded360c494bfa8775544899cba37c] alter complete (version 1)
I20260709 13:06:13.555045 12493 raft_consensus.cc:993] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8: : Instructing follower 437a699518154aa580c1ba0868048c05 to start an election
I20260709 13:06:13.555774 12504 raft_consensus.cc:1081] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Signalling peer 437a699518154aa580c1ba0868048c05 to start an election
I20260709 13:06:13.557472 12270 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd09f5dac3864c3da2319df9b220cc3f"
dest_uuid: "437a699518154aa580c1ba0868048c05"
 from {username='slave'} at 127.0.0.1:36116
I20260709 13:06:13.558032 12270 raft_consensus.cc:493] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:13.558305 12270 raft_consensus.cc:3060] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.563678 12270 raft_consensus.cc:515] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:13.565457 12270 leader_election.cc:290] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 2 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:13.566666 12120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd09f5dac3864c3da2319df9b220cc3f" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:13.566875 12196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd09f5dac3864c3da2319df9b220cc3f" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:13.567619 12120 raft_consensus.cc:3060] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.567662 12196 raft_consensus.cc:3055] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:13.568202 12196 raft_consensus.cc:740] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Running, Role: LEADER
I20260709 13:06:13.569023 12196 consensus_queue.cc:260] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:13.570092 12196 raft_consensus.cc:3060] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.573611 12120 raft_consensus.cc:2468] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 2.
I20260709 13:06:13.575103 12237 leader_election.cc:304] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2780688fce76495ca9c1ff6e8e998046, 437a699518154aa580c1ba0868048c05; no voters: 
I20260709 13:06:13.576113 12552 raft_consensus.cc:2804] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:13.576736 12552 raft_consensus.cc:697] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 2 LEADER]: Becoming Leader. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:13.576879 12196 raft_consensus.cc:2468] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 2.
I20260709 13:06:13.577757 12552 consensus_queue.cc:237] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:13.587168 12017 catalog_manager.cc:5697] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 reported cstate change: term changed from 1 to 2, leader changed from 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194) to 437a699518154aa580c1ba0868048c05 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "437a699518154aa580c1ba0868048c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:13.618937 12507 raft_consensus.cc:993] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046: : Instructing follower 437a699518154aa580c1ba0868048c05 to start an election
I20260709 13:06:13.619447 12507 raft_consensus.cc:1081] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Signalling peer 437a699518154aa580c1ba0868048c05 to start an election
I20260709 13:06:13.621177 12270 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b89ecf901f543718a4f562df52c12ff"
dest_uuid: "437a699518154aa580c1ba0868048c05"
 from {username='slave'} at 127.0.0.1:36108
I20260709 13:06:13.621716 12270 raft_consensus.cc:493] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:13.622049 12270 raft_consensus.cc:3060] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:13.629284 12270 raft_consensus.cc:515] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:13.631589 12270 leader_election.cc:290] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 3 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:13.632679 12120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89ecf901f543718a4f562df52c12ff" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:13.633378 12120 raft_consensus.cc:3055] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:13.633584 12196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89ecf901f543718a4f562df52c12ff" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:13.633795 12120 raft_consensus.cc:740] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:13.634514 12196 raft_consensus.cc:3060] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:13.635063 12120 consensus_queue.cc:260] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:13.636610 12120 raft_consensus.cc:3060] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:13.637537 12556 raft_consensus.cc:993] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046: : Instructing follower 437a699518154aa580c1ba0868048c05 to start an election
I20260709 13:06:13.638057 12507 raft_consensus.cc:1081] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Signalling peer 437a699518154aa580c1ba0868048c05 to start an election
I20260709 13:06:13.640544 12270 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cd616c68dbc74070b03abb376f3f5151"
dest_uuid: "437a699518154aa580c1ba0868048c05"
 from {username='slave'} at 127.0.0.1:36108
I20260709 13:06:13.641129 12270 raft_consensus.cc:493] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:13.641461 12270 raft_consensus.cc:3060] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:13.643895 12196 raft_consensus.cc:2468] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 3.
I20260709 13:06:13.645215 12235 leader_election.cc:304] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:13.646173 12552 raft_consensus.cc:2804] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:13.646688 12552 raft_consensus.cc:697] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 3 LEADER]: Becoming Leader. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:13.647570 12552 consensus_queue.cc:237] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:13.648052 12120 raft_consensus.cc:2468] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 3.
I20260709 13:06:13.648104 12270 raft_consensus.cc:515] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:13.651140 12120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd616c68dbc74070b03abb376f3f5151" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:13.651252 12270 leader_election.cc:290] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 3 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:13.652307 12196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd616c68dbc74070b03abb376f3f5151" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:13.651856 12120 raft_consensus.cc:3055] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:13.652909 12196 raft_consensus.cc:3060] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:13.653034 12120 raft_consensus.cc:740] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2780688fce76495ca9c1ff6e8e998046, State: Running, Role: LEADER
I20260709 13:06:13.655817 12493 raft_consensus.cc:993] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8: : Instructing follower 437a699518154aa580c1ba0868048c05 to start an election
I20260709 13:06:13.656415 12504 raft_consensus.cc:1081] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Signalling peer 437a699518154aa580c1ba0868048c05 to start an election
I20260709 13:06:13.656132 12120 consensus_queue.cc:260] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:13.659051 12270 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26e1bbe181f94724b4f3375b6a418fb4"
dest_uuid: "437a699518154aa580c1ba0868048c05"
 from {username='slave'} at 127.0.0.1:36116
I20260709 13:06:13.659790 12270 raft_consensus.cc:493] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:13.659083 12120 raft_consensus.cc:3060] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:13.660152 12017 catalog_manager.cc:5697] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 reported cstate change: term changed from 2 to 3, leader changed from 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193) to 437a699518154aa580c1ba0868048c05 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "437a699518154aa580c1ba0868048c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:13.661656 12196 raft_consensus.cc:2468] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 3.
I20260709 13:06:13.662704 12235 leader_election.cc:304] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:13.660185 12270 raft_consensus.cc:3060] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.665868 12552 raft_consensus.cc:2804] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:13.666465 12552 raft_consensus.cc:697] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 3 LEADER]: Becoming Leader. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:13.668246 12120 raft_consensus.cc:2468] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 3.
I20260709 13:06:13.668052 12552 consensus_queue.cc:237] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:13.673053 12270 raft_consensus.cc:515] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:13.676187 12270 leader_election.cc:290] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 2 election: Requested vote from peers 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857), 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543)
I20260709 13:06:13.676635 12196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e1bbe181f94724b4f3375b6a418fb4" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "52d9dc2493034c6cbd8196d41f879ce8"
I20260709 13:06:13.677446 12196 raft_consensus.cc:3055] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:13.677886 12196 raft_consensus.cc:740] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52d9dc2493034c6cbd8196d41f879ce8, State: Running, Role: LEADER
I20260709 13:06:13.678877 12196 consensus_queue.cc:260] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:13.679489 12017 catalog_manager.cc:5697] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 reported cstate change: term changed from 2 to 3, leader changed from 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193) to 437a699518154aa580c1ba0868048c05 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "437a699518154aa580c1ba0868048c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:13.680428 12196 raft_consensus.cc:3060] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.682550 12120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e1bbe181f94724b4f3375b6a418fb4" candidate_uuid: "437a699518154aa580c1ba0868048c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "2780688fce76495ca9c1ff6e8e998046"
I20260709 13:06:13.683285 12120 raft_consensus.cc:3060] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:13.688920 12196 raft_consensus.cc:2468] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 2.
I20260709 13:06:13.689960 12120 raft_consensus.cc:2468] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437a699518154aa580c1ba0868048c05 in term 2.
I20260709 13:06:13.690225 12235 leader_election.cc:304] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437a699518154aa580c1ba0868048c05, 52d9dc2493034c6cbd8196d41f879ce8; no voters: 
I20260709 13:06:13.691217 12552 raft_consensus.cc:2804] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:13.691728 12552 raft_consensus.cc:697] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [term 2 LEADER]: Becoming Leader. State: Replica: 437a699518154aa580c1ba0868048c05, State: Running, Role: LEADER
I20260709 13:06:13.692380 12552 consensus_queue.cc:237] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } }
I20260709 13:06:13.699364 12017 catalog_manager.cc:5697] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 reported cstate change: term changed from 1 to 2, leader changed from 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194) to 437a699518154aa580c1ba0868048c05 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "437a699518154aa580c1ba0868048c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437a699518154aa580c1ba0868048c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 32769 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:14.102766 12196 raft_consensus.cc:1275] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Refusing update from remote peer 437a699518154aa580c1ba0868048c05: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:06:14.104212 12552 consensus_queue.cc:1048] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:14.112737 12120 raft_consensus.cc:1275] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Refusing update from remote peer 437a699518154aa580c1ba0868048c05: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:06:14.115392 12583 consensus_queue.cc:1048] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:14.131089 12120 raft_consensus.cc:1275] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Refusing update from remote peer 437a699518154aa580c1ba0868048c05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.132400 12583 consensus_queue.cc:1048] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:14.140322 12196 raft_consensus.cc:1275] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [term 3 FOLLOWER]: Refusing update from remote peer 437a699518154aa580c1ba0868048c05: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:14.142107 12583 consensus_queue.cc:1048] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:14.156584 12120 raft_consensus.cc:1275] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 3 FOLLOWER]: Refusing update from remote peer 437a699518154aa580c1ba0868048c05: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:14.159637 12194 raft_consensus.cc:1275] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Refusing update from remote peer 437a699518154aa580c1ba0868048c05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.162284 12503 consensus_queue.cc:1048] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:14.166716 12503 consensus_queue.cc:1048] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:14.222190 12120 raft_consensus.cc:1275] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 3 FOLLOWER]: Refusing update from remote peer 437a699518154aa580c1ba0868048c05: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:14.223866 12503 consensus_queue.cc:1048] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2780688fce76495ca9c1ff6e8e998046" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:14.235885 12194 raft_consensus.cc:1275] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [term 3 FOLLOWER]: Refusing update from remote peer 437a699518154aa580c1ba0868048c05: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:14.237427 12552 consensus_queue.cc:1048] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52d9dc2493034c6cbd8196d41f879ce8" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:14.366371   431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:06:14.372210   431 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:06:14.372749   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:14.378090   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:06:14.378607   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:15.413163   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:06:15.461510   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:15.462672   431 tablet_replica.cc:333] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.463797   431 raft_consensus.cc:2243] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.464561   431 raft_consensus.cc:2272] T e18d1cfcb6cb4d26b5c374b738541e08 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.466773   431 tablet_replica.cc:333] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.467554   431 raft_consensus.cc:2243] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.468003   431 raft_consensus.cc:2272] T 26e1bbe181f94724b4f3375b6a418fb4 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.470085   431 tablet_replica.cc:333] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.470885   431 raft_consensus.cc:2243] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.471449   431 raft_consensus.cc:2272] T cd616c68dbc74070b03abb376f3f5151 P 2780688fce76495ca9c1ff6e8e998046 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.473712   431 tablet_replica.cc:333] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.474301   431 raft_consensus.cc:2243] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.475164   431 raft_consensus.cc:2272] T d19809bed5a94e91bfaa191a19cc2074 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.477036   431 tablet_replica.cc:333] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.477666   431 raft_consensus.cc:2243] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.478286   431 raft_consensus.cc:2272] T 29ae0bd6628a4dd3857b9e898b25febe P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.480602   431 tablet_replica.cc:333] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.481161   431 raft_consensus.cc:2243] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.481909   431 raft_consensus.cc:2272] T 28e53bfdf42a4890830f03cd39081785 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.484045   431 tablet_replica.cc:333] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.484745   431 raft_consensus.cc:2243] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.485285   431 raft_consensus.cc:2272] T 1731218f724f4b1da5dc847c9ff4a42e P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.487514   431 tablet_replica.cc:333] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.488106   431 raft_consensus.cc:2243] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.488544   431 raft_consensus.cc:2272] T 3d0833359f734033bfe7eab9cc74666f P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.490726   431 tablet_replica.cc:333] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.491475   431 raft_consensus.cc:2243] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.492323   431 raft_consensus.cc:2272] T a95812f5c040405ea3d02c1dc6de40a0 P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.494210   431 tablet_replica.cc:333] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.494827   431 raft_consensus.cc:2243] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.495357   431 raft_consensus.cc:2272] T fd09f5dac3864c3da2319df9b220cc3f P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.497378   431 tablet_replica.cc:333] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.497984   431 raft_consensus.cc:2243] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.498426   431 raft_consensus.cc:2272] T 8b89ecf901f543718a4f562df52c12ff P 2780688fce76495ca9c1ff6e8e998046 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.500509   431 tablet_replica.cc:333] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.501073   431 raft_consensus.cc:2243] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.501547   431 raft_consensus.cc:2272] T c16e10739a0e45b4ae22a2366e1162c3 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.503630   431 tablet_replica.cc:333] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.504191   431 raft_consensus.cc:2243] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.504638   431 raft_consensus.cc:2272] T 106eefe703c24ccc96acae1f610d7fdc P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.506748   431 tablet_replica.cc:333] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.507368   431 raft_consensus.cc:2243] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.507813   431 raft_consensus.cc:2272] T 6ef9bfc698934bd59128a2b6be47e79c P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.509603   431 tablet_replica.cc:333] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.510196   431 raft_consensus.cc:2243] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.510970   431 raft_consensus.cc:2272] T 9b4f009c4594491582878ae8b1f5f21b P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.512915   431 tablet_replica.cc:333] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.513378   431 raft_consensus.cc:2243] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.514133   431 raft_consensus.cc:2272] T d6bd5ec5c50c48b78c88d403817fbdae P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.516281   431 tablet_replica.cc:333] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.516774   431 raft_consensus.cc:2243] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.517493   431 raft_consensus.cc:2272] T f38db537da264dd4adf4363864b96250 P 2780688fce76495ca9c1ff6e8e998046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.519444   431 tablet_replica.cc:333] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046: stopping tablet replica
I20260709 13:06:15.519883   431 raft_consensus.cc:2243] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.520292   431 raft_consensus.cc:2272] T 63915eb22a6f42e3875c55499d3cafab P 2780688fce76495ca9c1ff6e8e998046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.562327   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:06:15.597851   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:06:15.622987 12163 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:35857: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20260709 13:06:15.628507 12163 consensus_peers.cc:597] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 -> Peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Couldn't send request to peer 2780688fce76495ca9c1ff6e8e998046. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:35857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:15.633095   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:15.633937   431 tablet_replica.cc:333] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.634581   431 raft_consensus.cc:2243] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.635141   431 raft_consensus.cc:2272] T e18d1cfcb6cb4d26b5c374b738541e08 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.637204   431 tablet_replica.cc:333] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.637811   431 raft_consensus.cc:2243] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.638584   431 raft_consensus.cc:2272] T 29ae0bd6628a4dd3857b9e898b25febe P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.640488   431 tablet_replica.cc:333] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.641065   431 raft_consensus.cc:2243] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.641510   431 raft_consensus.cc:2272] T 28e53bfdf42a4890830f03cd39081785 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.643313   431 tablet_replica.cc:333] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.643886   431 raft_consensus.cc:2243] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.644821   431 raft_consensus.cc:2272] T 6ef9bfc698934bd59128a2b6be47e79c P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.647195   431 tablet_replica.cc:333] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.647926   431 raft_consensus.cc:2243] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.648561   431 raft_consensus.cc:2272] T cd616c68dbc74070b03abb376f3f5151 P 52d9dc2493034c6cbd8196d41f879ce8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.650856   431 tablet_replica.cc:333] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.651609   431 raft_consensus.cc:2243] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.652122   431 raft_consensus.cc:2272] T d19809bed5a94e91bfaa191a19cc2074 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.654539   431 tablet_replica.cc:333] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.655313   431 raft_consensus.cc:2243] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.656366   431 raft_consensus.cc:2272] T 1731218f724f4b1da5dc847c9ff4a42e P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.658813   431 tablet_replica.cc:333] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.659669   431 raft_consensus.cc:2243] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.660205   431 raft_consensus.cc:2272] T 3d0833359f734033bfe7eab9cc74666f P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.663283   431 tablet_replica.cc:333] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.664112   431 raft_consensus.cc:2243] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.664721   431 raft_consensus.cc:2272] T a95812f5c040405ea3d02c1dc6de40a0 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.667151   431 tablet_replica.cc:333] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
W20260709 13:06:15.667827 12237 consensus_peers.cc:597] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 -> Peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Couldn't send request to peer 2780688fce76495ca9c1ff6e8e998046. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:35857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:15.668097   431 raft_consensus.cc:2243] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.668792   431 raft_consensus.cc:2272] T 9b4f009c4594491582878ae8b1f5f21b P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.671420   431 tablet_replica.cc:333] T d6bd5ec5c50c48b78c88d403817fbdae P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.672066   431 raft_consensus.cc:2243] T d6bd5ec5c50c48b78c88d403817fbdae P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.672537   431 raft_consensus.cc:2272] T d6bd5ec5c50c48b78c88d403817fbdae P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.675635   431 tablet_replica.cc:333] T f38db537da264dd4adf4363864b96250 P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.676319   431 raft_consensus.cc:2243] T f38db537da264dd4adf4363864b96250 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.676836   431 raft_consensus.cc:2272] T f38db537da264dd4adf4363864b96250 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:15.678314 12163 consensus_peers.cc:597] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 -> Peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Couldn't send request to peer 2780688fce76495ca9c1ff6e8e998046. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:35857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:15.679416   431 tablet_replica.cc:333] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.680117   431 raft_consensus.cc:2243] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.680938   431 raft_consensus.cc:2272] T 63915eb22a6f42e3875c55499d3cafab P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.683597   431 tablet_replica.cc:333] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.684267   431 raft_consensus.cc:2243] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.685070   431 raft_consensus.cc:2272] T 8b89ecf901f543718a4f562df52c12ff P 52d9dc2493034c6cbd8196d41f879ce8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.688007   431 tablet_replica.cc:333] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.688839   431 raft_consensus.cc:2243] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:15.688953 12237 consensus_peers.cc:597] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 -> Peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Couldn't send request to peer 2780688fce76495ca9c1ff6e8e998046. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:35857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:15.689422   431 raft_consensus.cc:2272] T fd09f5dac3864c3da2319df9b220cc3f P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.691812   431 tablet_replica.cc:333] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.692483   431 raft_consensus.cc:2243] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.693094   431 raft_consensus.cc:2272] T 26e1bbe181f94724b4f3375b6a418fb4 P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.697116   431 tablet_replica.cc:333] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.698060   431 raft_consensus.cc:2243] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:06:15.698671 12237 consensus_peers.cc:597] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 -> Peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Couldn't send request to peer 2780688fce76495ca9c1ff6e8e998046. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:35857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:15.699208   431 raft_consensus.cc:2272] T 106eefe703c24ccc96acae1f610d7fdc P 52d9dc2493034c6cbd8196d41f879ce8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.701673   431 tablet_replica.cc:333] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8: stopping tablet replica
I20260709 13:06:15.702257   431 raft_consensus.cc:2243] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.703197   431 raft_consensus.cc:2272] T c16e10739a0e45b4ae22a2366e1162c3 P 52d9dc2493034c6cbd8196d41f879ce8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.737964   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:06:15.766376 12235 consensus_peers.cc:597] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 -> Peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Couldn't send request to peer 52d9dc2493034c6cbd8196d41f879ce8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:34543: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:15.775213   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:06:15.800915 12235 consensus_peers.cc:597] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 -> Peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Couldn't send request to peer 52d9dc2493034c6cbd8196d41f879ce8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:34543: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:15.805861 12235 consensus_peers.cc:597] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 -> Peer 52d9dc2493034c6cbd8196d41f879ce8 (127.0.107.194:34543): Couldn't send request to peer 52d9dc2493034c6cbd8196d41f879ce8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:34543: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:15.811088   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:15.812053   431 tablet_replica.cc:333] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.813088   431 raft_consensus.cc:2243] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.814211   431 raft_consensus.cc:2272] T 26e1bbe181f94724b4f3375b6a418fb4 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.816263   431 tablet_replica.cc:333] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.816857   431 raft_consensus.cc:2243] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.817308   431 raft_consensus.cc:2272] T 6ef9bfc698934bd59128a2b6be47e79c P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.819223   431 tablet_replica.cc:333] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.819761   431 raft_consensus.cc:2243] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.820544   431 raft_consensus.cc:2272] T e18d1cfcb6cb4d26b5c374b738541e08 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.822479   431 tablet_replica.cc:333] T d19809bed5a94e91bfaa191a19cc2074 P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.823179   431 raft_consensus.cc:2243] T d19809bed5a94e91bfaa191a19cc2074 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.823678   431 raft_consensus.cc:2272] T d19809bed5a94e91bfaa191a19cc2074 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.825599   431 tablet_replica.cc:333] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.826210   431 raft_consensus.cc:2243] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.827126   431 raft_consensus.cc:2272] T cd616c68dbc74070b03abb376f3f5151 P 437a699518154aa580c1ba0868048c05 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.829002   431 tablet_replica.cc:333] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.829638   431 raft_consensus.cc:2243] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.830112   431 raft_consensus.cc:2272] T 1731218f724f4b1da5dc847c9ff4a42e P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.832199   431 tablet_replica.cc:333] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.832862   431 raft_consensus.cc:2243] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.833722   431 raft_consensus.cc:2272] T 3d0833359f734033bfe7eab9cc74666f P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.836428   431 tablet_replica.cc:333] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.837153   431 raft_consensus.cc:2243] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.837740   431 raft_consensus.cc:2272] T a95812f5c040405ea3d02c1dc6de40a0 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.839934   431 tablet_replica.cc:333] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.840684   431 raft_consensus.cc:2243] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.841229   431 raft_consensus.cc:2272] T 29ae0bd6628a4dd3857b9e898b25febe P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.844030   431 tablet_replica.cc:333] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.844749   431 raft_consensus.cc:2243] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.845281   431 raft_consensus.cc:2272] T 28e53bfdf42a4890830f03cd39081785 P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:15.846838 12237 consensus_peers.cc:597] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 -> Peer 2780688fce76495ca9c1ff6e8e998046 (127.0.107.193:35857): Couldn't send request to peer 2780688fce76495ca9c1ff6e8e998046. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:35857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:15.847693   431 tablet_replica.cc:333] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.848395   431 raft_consensus.cc:2243] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.848956   431 raft_consensus.cc:2272] T f38db537da264dd4adf4363864b96250 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.851390   431 tablet_replica.cc:333] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.852025   431 raft_consensus.cc:2243] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.852502   431 raft_consensus.cc:2272] T 63915eb22a6f42e3875c55499d3cafab P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.855232   431 tablet_replica.cc:333] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.855901   431 raft_consensus.cc:2243] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.856438   431 raft_consensus.cc:2272] T c16e10739a0e45b4ae22a2366e1162c3 P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.858619   431 tablet_replica.cc:333] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.859387   431 raft_consensus.cc:2243] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.860271   431 raft_consensus.cc:2272] T 8b89ecf901f543718a4f562df52c12ff P 437a699518154aa580c1ba0868048c05 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.862509   431 tablet_replica.cc:333] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.863256   431 raft_consensus.cc:2243] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.863754   431 raft_consensus.cc:2272] T d6bd5ec5c50c48b78c88d403817fbdae P 437a699518154aa580c1ba0868048c05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.866123   431 tablet_replica.cc:333] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.866720   431 raft_consensus.cc:2243] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.867316   431 raft_consensus.cc:2272] T 9b4f009c4594491582878ae8b1f5f21b P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.869535   431 tablet_replica.cc:333] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.870119   431 raft_consensus.cc:2243] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:15.870649   431 raft_consensus.cc:2272] T 106eefe703c24ccc96acae1f610d7fdc P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.872835   431 tablet_replica.cc:333] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05: stopping tablet replica
I20260709 13:06:15.873374   431 raft_consensus.cc:2243] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.874202   431 raft_consensus.cc:2272] T fd09f5dac3864c3da2319df9b220cc3f P 437a699518154aa580c1ba0868048c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.916666   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:06:15.949774   431 master.cc:562] Master@127.0.107.254:42411 shutting down...
I20260709 13:06:15.976173   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:15.976835   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:15.977219   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 634b2b70487747899d3404a63bf5c87c: stopping tablet replica
I20260709 13:06:15.999085   431 master.cc:584] Master@127.0.107.254:42411 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (12757 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:06:16.053725   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:35909
I20260709 13:06:16.054858   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:16.062338 12597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:16.062640   431 server_base.cc:1061] running on GCE node
W20260709 13:06:16.062361 12598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:16.063138 12600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:16.065594   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:16.065794   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:16.066020   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602376066003 us; error 0 us; skew 500 ppm
I20260709 13:06:16.066484   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:16.069420   431 webserver.cc:533] Webserver started at http://127.0.107.254:36899/ using document root <none> and password file <none>
I20260709 13:06:16.069929   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:16.070101   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:16.070333   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:16.071687   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/master-0-root/instance:
uuid: "ec8e2d1e62004163b57ec94997e000a0"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-cq4t"
I20260709 13:06:16.076529   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:06:16.080555 12605 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.081578   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:06:16.081966   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/master-0-root
uuid: "ec8e2d1e62004163b57ec94997e000a0"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-cq4t"
I20260709 13:06:16.082254   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:16.109679   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:16.111122   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:16.155407   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:35909
I20260709 13:06:16.155597 12656 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:35909 every 8 connection(s)
I20260709 13:06:16.159595 12657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.170773 12657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0: Bootstrap starting.
I20260709 13:06:16.175963 12657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:16.180606 12657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0: No bootstrap required, opened a new log
I20260709 13:06:16.183198 12657 raft_consensus.cc:359] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8e2d1e62004163b57ec94997e000a0" member_type: VOTER }
I20260709 13:06:16.183663 12657 raft_consensus.cc:385] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:16.184008 12657 raft_consensus.cc:740] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8e2d1e62004163b57ec94997e000a0, State: Initialized, Role: FOLLOWER
I20260709 13:06:16.184703 12657 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8e2d1e62004163b57ec94997e000a0" member_type: VOTER }
I20260709 13:06:16.185181 12657 raft_consensus.cc:399] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:16.185391 12657 raft_consensus.cc:493] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:16.185667 12657 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:16.191167 12657 raft_consensus.cc:515] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8e2d1e62004163b57ec94997e000a0" member_type: VOTER }
I20260709 13:06:16.191733 12657 leader_election.cc:304] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec8e2d1e62004163b57ec94997e000a0; no voters: 
I20260709 13:06:16.192912 12657 leader_election.cc:290] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:16.193323 12660 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:16.194713 12660 raft_consensus.cc:697] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [term 1 LEADER]: Becoming Leader. State: Replica: ec8e2d1e62004163b57ec94997e000a0, State: Running, Role: LEADER
I20260709 13:06:16.195449 12660 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8e2d1e62004163b57ec94997e000a0" member_type: VOTER }
I20260709 13:06:16.196367 12657 sys_catalog.cc:565] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:16.200038 12661 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec8e2d1e62004163b57ec94997e000a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8e2d1e62004163b57ec94997e000a0" member_type: VOTER } }
I20260709 13:06:16.200127 12662 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec8e2d1e62004163b57ec94997e000a0. Latest consensus state: current_term: 1 leader_uuid: "ec8e2d1e62004163b57ec94997e000a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8e2d1e62004163b57ec94997e000a0" member_type: VOTER } }
I20260709 13:06:16.201014 12661 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:16.201201 12662 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:16.204100 12666 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:16.209261 12666 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:16.216514   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:16.220428 12666 catalog_manager.cc:1383] Generated new cluster ID: da43a750a436498389467caeb082a728
I20260709 13:06:16.220798 12666 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:16.246498 12666 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:16.247915 12666 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:16.268675 12666 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0: Generated new TSK 0
I20260709 13:06:16.269336 12666 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:16.286780   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:16.293890 12678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:16.297215 12679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:16.301354 12681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:16.301515   431 server_base.cc:1061] running on GCE node
I20260709 13:06:16.304207   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:16.304471   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:16.304768   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602376304745 us; error 0 us; skew 500 ppm
I20260709 13:06:16.305385   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:16.308349   431 webserver.cc:533] Webserver started at http://127.0.107.193:45933/ using document root <none> and password file <none>
I20260709 13:06:16.308878   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:16.309091   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:16.309412   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:16.310519   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-0-root/instance:
uuid: "3a9fb085add94dd99d22ee2b1129932c"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-cq4t"
I20260709 13:06:16.315228   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:06:16.319026 12686 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.320087   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:16.320431   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-0-root
uuid: "3a9fb085add94dd99d22ee2b1129932c"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-cq4t"
I20260709 13:06:16.320739   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:16.342725   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:16.344012   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:16.345561   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:16.347941   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:16.348176   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.348438   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:16.348646   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.394531   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:42057
I20260709 13:06:16.394712 12748 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:42057 every 8 connection(s)
I20260709 13:06:16.400141   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:16.407337 12753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:16.409467 12754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:16.413163 12756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:16.413756   431 server_base.cc:1061] running on GCE node
I20260709 13:06:16.414083 12749 heartbeater.cc:344] Connected to a master server at 127.0.107.254:35909
I20260709 13:06:16.414565 12749 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:16.415421 12749 heartbeater.cc:507] Master 127.0.107.254:35909 requested a full tablet report, sending...
I20260709 13:06:16.415467   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:16.415879   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:16.416083   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602376416064 us; error 0 us; skew 500 ppm
I20260709 13:06:16.416754   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:16.418180 12622 ts_manager.cc:194] Registered new tserver with Master: 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057)
I20260709 13:06:16.420316   431 webserver.cc:533] Webserver started at http://127.0.107.194:44789/ using document root <none> and password file <none>
I20260709 13:06:16.421105   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:16.421249 12622 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43190
I20260709 13:06:16.421381   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:16.422863   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:16.424247   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-1-root/instance:
uuid: "fc1fcc33b1c04369ab276792291f3331"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-cq4t"
I20260709 13:06:16.429677   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:06:16.433559 12761 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.434518   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:06:16.435185   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-1-root
uuid: "fc1fcc33b1c04369ab276792291f3331"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-cq4t"
I20260709 13:06:16.435693   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:16.448527   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:16.449793   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:16.451720   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:16.454629   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:16.454901   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.455284   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:16.455479   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.502422   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:36543
I20260709 13:06:16.502591 12823 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:36543 every 8 connection(s)
I20260709 13:06:16.507498   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:16.515617 12827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:16.519016 12828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:16.524561 12824 heartbeater.cc:344] Connected to a master server at 127.0.107.254:35909
I20260709 13:06:16.525152 12824 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:16.525967   431 server_base.cc:1061] running on GCE node
I20260709 13:06:16.526233 12824 heartbeater.cc:507] Master 127.0.107.254:35909 requested a full tablet report, sending...
W20260709 13:06:16.526865 12830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:16.528069   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:16.528331   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:16.528606   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602376528523 us; error 0 us; skew 500 ppm
I20260709 13:06:16.528793 12622 ts_manager.cc:194] Registered new tserver with Master: fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:16.529263   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:16.530534 12622 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43206
I20260709 13:06:16.532321   431 webserver.cc:533] Webserver started at http://127.0.107.195:46213/ using document root <none> and password file <none>
I20260709 13:06:16.533110   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:16.533377   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:16.533694   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:16.534905   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-2-root/instance:
uuid: "4a2c916532db4ec1b36070c705d7e07f"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-cq4t"
I20260709 13:06:16.540201   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:06:16.544049 12835 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.545167   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:06:16.545465   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-2-root
uuid: "4a2c916532db4ec1b36070c705d7e07f"
format_stamp: "Formatted at 2026-07-09 13:06:16 on dist-test-slave-cq4t"
I20260709 13:06:16.545756   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskXSiRsn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602014632082-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:16.562016   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:16.563243   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:16.564750   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:16.567297   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:16.567492   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.567834   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:16.568011   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:16.617264   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:36913
I20260709 13:06:16.617394 12897 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:36913 every 8 connection(s)
I20260709 13:06:16.633469 12898 heartbeater.cc:344] Connected to a master server at 127.0.107.254:35909
I20260709 13:06:16.634050 12898 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:16.634845 12898 heartbeater.cc:507] Master 127.0.107.254:35909 requested a full tablet report, sending...
I20260709 13:06:16.637262 12622 ts_manager.cc:194] Registered new tserver with Master: 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:16.637926   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017170708s
I20260709 13:06:16.639149 12622 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43216
I20260709 13:06:16.667411 12622 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43218:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:16.670095 12622 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:16.752532 12714 tablet_service.cc:1511] Processing CreateTablet for tablet 56647efebe9c40b0bc6363de5264c359 (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:16.754123 12714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56647efebe9c40b0bc6363de5264c359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.760876 12713 tablet_service.cc:1511] Processing CreateTablet for tablet a98c8d37fff242259cc617625b0f83b0 (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:16.761634 12712 tablet_service.cc:1511] Processing CreateTablet for tablet 51bbf038592a410da9f6958348a1934b (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:16.762643 12713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a98c8d37fff242259cc617625b0f83b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.763516 12789 tablet_service.cc:1511] Processing CreateTablet for tablet 56647efebe9c40b0bc6363de5264c359 (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:16.764261 12863 tablet_service.cc:1511] Processing CreateTablet for tablet 56647efebe9c40b0bc6363de5264c359 (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:16.765162 12789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56647efebe9c40b0bc6363de5264c359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.765810 12863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56647efebe9c40b0bc6363de5264c359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.765264 12711 tablet_service.cc:1511] Processing CreateTablet for tablet bcf73d1d6c504892aca0e7c5abf13da7 (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:16.768187 12788 tablet_service.cc:1511] Processing CreateTablet for tablet a98c8d37fff242259cc617625b0f83b0 (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:16.769758 12788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a98c8d37fff242259cc617625b0f83b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.770671 12862 tablet_service.cc:1511] Processing CreateTablet for tablet a98c8d37fff242259cc617625b0f83b0 (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:16.772385 12862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a98c8d37fff242259cc617625b0f83b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.772154 12787 tablet_service.cc:1511] Processing CreateTablet for tablet 51bbf038592a410da9f6958348a1934b (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:16.773769 12787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51bbf038592a410da9f6958348a1934b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.776117 12786 tablet_service.cc:1511] Processing CreateTablet for tablet bcf73d1d6c504892aca0e7c5abf13da7 (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:16.777661 12786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf73d1d6c504892aca0e7c5abf13da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.763574 12712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51bbf038592a410da9f6958348a1934b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.779917 12711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf73d1d6c504892aca0e7c5abf13da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.800979 12861 tablet_service.cc:1511] Processing CreateTablet for tablet 51bbf038592a410da9f6958348a1934b (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:16.803617 12861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51bbf038592a410da9f6958348a1934b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.817884 12860 tablet_service.cc:1511] Processing CreateTablet for tablet bcf73d1d6c504892aca0e7c5abf13da7 (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:16.821038 12860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf73d1d6c504892aca0e7c5abf13da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.831370 12710 tablet_service.cc:1511] Processing CreateTablet for tablet d38859ad5d584abab1d3d1c93729a5f4 (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:16.833714 12710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d38859ad5d584abab1d3d1c93729a5f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.839987 12785 tablet_service.cc:1511] Processing CreateTablet for tablet d38859ad5d584abab1d3d1c93729a5f4 (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:16.842387 12785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d38859ad5d584abab1d3d1c93729a5f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.867862 12859 tablet_service.cc:1511] Processing CreateTablet for tablet d38859ad5d584abab1d3d1c93729a5f4 (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:16.870573 12859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d38859ad5d584abab1d3d1c93729a5f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.893023 12784 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb3192718d94c2e850c178a9ad03e1d (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:16.895287 12784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb3192718d94c2e850c178a9ad03e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.904331 12709 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb3192718d94c2e850c178a9ad03e1d (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:16.906810 12709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb3192718d94c2e850c178a9ad03e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.923437 12858 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb3192718d94c2e850c178a9ad03e1d (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:16.925863 12858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb3192718d94c2e850c178a9ad03e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:16.973502 12708 tablet_service.cc:1511] Processing CreateTablet for tablet 86c4c48405d84950bc27d4972be8a04c (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:16.996402 12913 tablet_bootstrap.cc:492] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:16.996173 12783 tablet_service.cc:1511] Processing CreateTablet for tablet 86c4c48405d84950bc27d4972be8a04c (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:16.997989 12783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c4c48405d84950bc27d4972be8a04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.007417 12708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c4c48405d84950bc27d4972be8a04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.010205 12857 tablet_service.cc:1511] Processing CreateTablet for tablet 86c4c48405d84950bc27d4972be8a04c (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:17.013023 12857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c4c48405d84950bc27d4972be8a04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.026047 12914 tablet_bootstrap.cc:492] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:17.057581 12915 tablet_bootstrap.cc:492] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:17.061874 12914 tablet_bootstrap.cc:654] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.076649 12711 tablet_service.cc:1511] Processing CreateTablet for tablet 17fa856455174351b44f0fa9466fbf4d (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:17.078156 12711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17fa856455174351b44f0fa9466fbf4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.080653 12789 tablet_service.cc:1511] Processing CreateTablet for tablet 17fa856455174351b44f0fa9466fbf4d (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:17.082749 12913 tablet_bootstrap.cc:654] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.083292 12915 tablet_bootstrap.cc:654] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.087457 12860 tablet_service.cc:1511] Processing CreateTablet for tablet 17fa856455174351b44f0fa9466fbf4d (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:17.089301 12860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17fa856455174351b44f0fa9466fbf4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.092447 12712 tablet_service.cc:1511] Processing CreateTablet for tablet f4fe0354684a4ea08b937a39cec88abb (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:17.093887 12712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4fe0354684a4ea08b937a39cec88abb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.096355 12789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17fa856455174351b44f0fa9466fbf4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.105406 12914 tablet_bootstrap.cc:492] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:17.106365 12914 ts_tablet_manager.cc:1403] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.081s	user 0.011s	sys 0.006s
I20260709 13:06:17.107767 12784 tablet_service.cc:1511] Processing CreateTablet for tablet f4fe0354684a4ea08b937a39cec88abb (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:17.109174 12784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4fe0354684a4ea08b937a39cec88abb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.109449 12914 raft_consensus.cc:359] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } }
I20260709 13:06:17.111263 12914 raft_consensus.cc:385] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.111568 12914 raft_consensus.cc:740] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.112608 12914 consensus_queue.cc:260] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } }
I20260709 13:06:17.115353 12915 tablet_bootstrap.cc:492] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:17.123504 12858 tablet_service.cc:1511] Processing CreateTablet for tablet f4fe0354684a4ea08b937a39cec88abb (DEFAULT_TABLE table=test-workload [id=1241909cab5a4689afb528ec56a0e172]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:17.126715 12915 ts_tablet_manager.cc:1403] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.010s
I20260709 13:06:17.125824 12913 tablet_bootstrap.cc:492] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:17.127175 12858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4fe0354684a4ea08b937a39cec88abb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:17.127964 12913 ts_tablet_manager.cc:1403] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.132s	user 0.015s	sys 0.000s
I20260709 13:06:17.130147 12915 raft_consensus.cc:359] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.131062 12915 raft_consensus.cc:385] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.131418 12915 raft_consensus.cc:740] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.132310 12915 consensus_queue.cc:260] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.141796 12824 heartbeater.cc:499] Master 127.0.107.254:35909 was elected leader, sending a full tablet report...
I20260709 13:06:17.141429 12915 ts_tablet_manager.cc:1434] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.009s
I20260709 13:06:17.143770 12915 tablet_bootstrap.cc:492] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:17.147732 12913 raft_consensus.cc:359] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } }
I20260709 13:06:17.148734 12913 raft_consensus.cc:385] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.149106 12913 raft_consensus.cc:740] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.152621 12898 heartbeater.cc:499] Master 127.0.107.254:35909 was elected leader, sending a full tablet report...
I20260709 13:06:17.152688 12915 tablet_bootstrap.cc:654] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.159984 12914 ts_tablet_manager.cc:1434] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.053s	user 0.029s	sys 0.008s
I20260709 13:06:17.151973 12913 consensus_queue.cc:260] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } }
I20260709 13:06:17.161283 12914 tablet_bootstrap.cc:492] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:17.167603 12914 tablet_bootstrap.cc:654] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.177881 12915 tablet_bootstrap.cc:492] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:17.178501 12915 ts_tablet_manager.cc:1403] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.014s
I20260709 13:06:17.181840 12915 raft_consensus.cc:359] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.182693 12915 raft_consensus.cc:385] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.183095 12915 raft_consensus.cc:740] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.184229 12915 consensus_queue.cc:260] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.186674 12915 ts_tablet_manager.cc:1434] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:06:17.187758 12915 tablet_bootstrap.cc:492] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:17.194571 12915 tablet_bootstrap.cc:654] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.206888 12915 tablet_bootstrap.cc:492] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:17.207518 12915 ts_tablet_manager.cc:1403] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.020s	user 0.005s	sys 0.014s
W20260709 13:06:17.210706 12670 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:17.210752 12915 raft_consensus.cc:359] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } }
I20260709 13:06:17.212030 12915 raft_consensus.cc:385] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.212415 12915 raft_consensus.cc:740] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.213404 12915 consensus_queue.cc:260] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } }
I20260709 13:06:17.214742 12749 heartbeater.cc:499] Master 127.0.107.254:35909 was elected leader, sending a full tablet report...
I20260709 13:06:17.215929 12913 ts_tablet_manager.cc:1434] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.088s	user 0.027s	sys 0.022s
I20260709 13:06:17.217541 12913 tablet_bootstrap.cc:492] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:17.219501 12915 ts_tablet_manager.cc:1434] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.003s
I20260709 13:06:17.220742 12915 tablet_bootstrap.cc:492] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:17.227413 12915 tablet_bootstrap.cc:654] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.228560 12913 tablet_bootstrap.cc:654] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.236339 12914 tablet_bootstrap.cc:492] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:17.237098 12914 ts_tablet_manager.cc:1403] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.076s	user 0.026s	sys 0.043s
I20260709 13:06:17.241024 12914 raft_consensus.cc:359] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.242251 12914 raft_consensus.cc:385] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.242683 12914 raft_consensus.cc:740] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.246344 12913 tablet_bootstrap.cc:492] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:17.247010 12913 ts_tablet_manager.cc:1403] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.030s	user 0.007s	sys 0.016s
I20260709 13:06:17.247176 12914 consensus_queue.cc:260] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.250512 12914 ts_tablet_manager.cc:1434] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:06:17.251618 12914 tablet_bootstrap.cc:492] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:17.259119 12914 tablet_bootstrap.cc:654] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.258590 12913 raft_consensus.cc:359] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.259719 12913 raft_consensus.cc:385] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.260185 12913 raft_consensus.cc:740] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.261111 12913 consensus_queue.cc:260] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.263512 12913 ts_tablet_manager.cc:1434] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 13:06:17.264760 12913 tablet_bootstrap.cc:492] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:17.271230 12922 raft_consensus.cc:493] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.272228 12922 raft_consensus.cc:515] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } }
I20260709 13:06:17.273223 12913 tablet_bootstrap.cc:654] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.275869 12914 tablet_bootstrap.cc:492] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:17.276587 12914 ts_tablet_manager.cc:1403] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.006s
I20260709 13:06:17.280591 12914 raft_consensus.cc:359] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.281762 12914 raft_consensus.cc:385] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.282347 12915 tablet_bootstrap.cc:492] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:17.282245 12914 raft_consensus.cc:740] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.283011 12915 ts_tablet_manager.cc:1403] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.062s	user 0.030s	sys 0.018s
I20260709 13:06:17.288820 12922 leader_election.cc:290] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:17.283653 12914 consensus_queue.cc:260] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.291791 12914 ts_tablet_manager.cc:1434] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:06:17.286463 12915 raft_consensus.cc:359] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.292898 12915 raft_consensus.cc:385] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.293315 12914 tablet_bootstrap.cc:492] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:17.293335 12915 raft_consensus.cc:740] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.296394 12915 consensus_queue.cc:260] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.302707 12914 tablet_bootstrap.cc:654] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.304111 12915 ts_tablet_manager.cc:1434] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:06:17.305286 12915 tablet_bootstrap.cc:492] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:17.321744 12914 tablet_bootstrap.cc:492] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:17.323091 12914 ts_tablet_manager.cc:1403] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.004s
I20260709 13:06:17.323619 12915 tablet_bootstrap.cc:654] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.324666 12913 tablet_bootstrap.cc:492] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:17.325313 12913 ts_tablet_manager.cc:1403] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.061s	user 0.011s	sys 0.000s
I20260709 13:06:17.327219 12919 raft_consensus.cc:493] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.327862 12919 raft_consensus.cc:515] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } }
I20260709 13:06:17.330997 12919 leader_election.cc:290] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:17.340018 12913 raft_consensus.cc:359] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.340991 12913 raft_consensus.cc:385] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.341365 12913 raft_consensus.cc:740] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.346349 12914 raft_consensus.cc:359] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.347546 12914 raft_consensus.cc:385] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.347878 12914 raft_consensus.cc:740] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.348767 12914 consensus_queue.cc:260] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.351507 12913 consensus_queue.cc:260] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.353325 12915 tablet_bootstrap.cc:492] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:17.353957 12915 ts_tablet_manager.cc:1403] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.049s	user 0.011s	sys 0.000s
I20260709 13:06:17.354149 12913 ts_tablet_manager.cc:1434] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:06:17.357435 12914 ts_tablet_manager.cc:1434] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:06:17.358757 12914 tablet_bootstrap.cc:492] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:17.364554 12913 tablet_bootstrap.cc:492] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:17.370054 12919 raft_consensus.cc:493] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.371208 12919 raft_consensus.cc:515] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.373772 12919 leader_election.cc:290] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:17.374693 12873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56647efebe9c40b0bc6363de5264c359" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:17.378619 12873 raft_consensus.cc:2468] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 0.
I20260709 13:06:17.376075 12915 raft_consensus.cc:359] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.380235 12915 raft_consensus.cc:385] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.380231 12690 leader_election.cc:304] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:17.380960 12915 raft_consensus.cc:740] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.381474 12922 raft_consensus.cc:2804] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:17.381930 12922 raft_consensus.cc:493] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.382372 12922 raft_consensus.cc:3060] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.382071 12915 consensus_queue.cc:260] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.385284 12915 ts_tablet_manager.cc:1434] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.031s	user 0.010s	sys 0.001s
I20260709 13:06:17.386637 12915 tablet_bootstrap.cc:492] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:17.388679 12799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56647efebe9c40b0bc6363de5264c359" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:17.377700 12914 tablet_bootstrap.cc:654] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.389585 12799 raft_consensus.cc:2468] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 0.
I20260709 13:06:17.393743 12913 tablet_bootstrap.cc:654] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.397436 12915 tablet_bootstrap.cc:654] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.398527 12799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56647efebe9c40b0bc6363de5264c359" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:17.399355 12799 raft_consensus.cc:2468] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 0.
I20260709 13:06:17.396469 12922 raft_consensus.cc:515] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } }
I20260709 13:06:17.400449 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38859ad5d584abab1d3d1c93729a5f4" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:17.401249 12798 raft_consensus.cc:2468] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 0.
I20260709 13:06:17.403834 12873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56647efebe9c40b0bc6363de5264c359" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:17.404464 12837 leader_election.cc:304] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:17.404724 12873 raft_consensus.cc:3060] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.405627 12837 leader_election.cc:304] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:17.407397 12942 raft_consensus.cc:2804] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:17.407985 12942 raft_consensus.cc:493] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.408427 12942 raft_consensus.cc:3060] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.416528 12922 leader_election.cc:290] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Requested vote from peers 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:17.416453 12942 raft_consensus.cc:515] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.417227 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56647efebe9c40b0bc6363de5264c359" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:17.417938 12798 raft_consensus.cc:3060] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.419989 12942 leader_election.cc:290] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:17.421206 12799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38859ad5d584abab1d3d1c93729a5f4" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:17.421913 12799 raft_consensus.cc:3060] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.423694 12873 raft_consensus.cc:2468] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 1.
I20260709 13:06:17.425405 12690 leader_election.cc:304] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:17.427042 12919 raft_consensus.cc:2764] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:06:17.428443 12913 tablet_bootstrap.cc:492] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:17.428742 12798 raft_consensus.cc:2468] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 1.
I20260709 13:06:17.429235 12913 ts_tablet_manager.cc:1403] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.065s	user 0.004s	sys 0.011s
I20260709 13:06:17.432750 12799 raft_consensus.cc:2468] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 1.
I20260709 13:06:17.433588 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38859ad5d584abab1d3d1c93729a5f4" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
I20260709 13:06:17.433981 12724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56647efebe9c40b0bc6363de5264c359" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
I20260709 13:06:17.434286 12837 leader_election.cc:304] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:17.434906 12724 raft_consensus.cc:2393] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4a2c916532db4ec1b36070c705d7e07f in current term 1: Already voted for candidate 3a9fb085add94dd99d22ee2b1129932c in this term.
I20260709 13:06:17.434707 12723 raft_consensus.cc:2468] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 0.
I20260709 13:06:17.435302 12919 raft_consensus.cc:2804] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:17.435838 12919 raft_consensus.cc:697] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Becoming Leader. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Running, Role: LEADER
I20260709 13:06:17.436558 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38859ad5d584abab1d3d1c93729a5f4" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:17.436733 12919 consensus_queue.cc:237] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.437523 12722 raft_consensus.cc:3060] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.443902 12922 raft_consensus.cc:2804] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:17.449672 12914 tablet_bootstrap.cc:492] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:17.450541 12914 ts_tablet_manager.cc:1403] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.092s	user 0.014s	sys 0.002s
I20260709 13:06:17.457343 12722 raft_consensus.cc:2468] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 1.
I20260709 13:06:17.454838 12914 raft_consensus.cc:359] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.463893 12915 tablet_bootstrap.cc:492] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:17.463845 12913 raft_consensus.cc:359] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.465384 12915 ts_tablet_manager.cc:1403] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.079s	user 0.021s	sys 0.054s
I20260709 13:06:17.468202 12913 raft_consensus.cc:385] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.468562 12913 raft_consensus.cc:740] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.469394 12913 consensus_queue.cc:260] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.471177 12915 raft_consensus.cc:359] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.472016 12915 raft_consensus.cc:385] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.472242 12913 ts_tablet_manager.cc:1434] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.004s
I20260709 13:06:17.472421 12915 raft_consensus.cc:740] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.467623 12922 raft_consensus.cc:697] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Becoming Leader. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:17.464970 12914 raft_consensus.cc:385] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.508147 12914 raft_consensus.cc:740] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.506111 12920 raft_consensus.cc:493] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.507525 12915 consensus_queue.cc:260] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.509850 12920 raft_consensus.cc:515] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.510584 12620 catalog_manager.cc:5697] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195). New cstate: current_term: 1 leader_uuid: "4a2c916532db4ec1b36070c705d7e07f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:17.520757 12920 leader_election.cc:290] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.511935 12914 consensus_queue.cc:260] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.527788 12914 ts_tablet_manager.cc:1434] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.077s	user 0.003s	sys 0.009s
I20260709 13:06:17.537742 12914 tablet_bootstrap.cc:492] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:17.542047 12915 ts_tablet_manager.cc:1434] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.074s	user 0.005s	sys 0.006s
I20260709 13:06:17.544322 12915 tablet_bootstrap.cc:492] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:17.529565 12922 consensus_queue.cc:237] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } }
I20260709 13:06:17.548002 12948 raft_consensus.cc:493] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.532042 12913 tablet_bootstrap.cc:492] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:17.549039 12948 raft_consensus.cc:515] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.552706 12948 leader_election.cc:290] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.554747 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a98c8d37fff242259cc617625b0f83b0" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
I20260709 13:06:17.552548 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf73d1d6c504892aca0e7c5abf13da7" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
I20260709 13:06:17.557215 12722 raft_consensus.cc:2468] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 0.
W20260709 13:06:17.559027 12765 leader_election.cc:343] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:17.560196 12914 tablet_bootstrap.cc:654] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.560204 12915 tablet_bootstrap.cc:654] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.560120 12765 leader_election.cc:304] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:17.564910 12948 raft_consensus.cc:2804] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:17.565404 12948 raft_consensus.cc:493] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.565850 12948 raft_consensus.cc:3060] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.581748 12873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf73d1d6c504892aca0e7c5abf13da7" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:17.582836 12873 raft_consensus.cc:2468] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 0.
I20260709 13:06:17.586179 12873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a98c8d37fff242259cc617625b0f83b0" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:17.587397 12873 raft_consensus.cc:2468] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 0.
I20260709 13:06:17.589200 12915 tablet_bootstrap.cc:492] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:17.589538 12765 leader_election.cc:304] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 3a9fb085add94dd99d22ee2b1129932c
I20260709 13:06:17.586424 12913 tablet_bootstrap.cc:654] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.592607 12915 ts_tablet_manager.cc:1403] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.021s
I20260709 13:06:17.593704 12920 raft_consensus.cc:2804] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:17.594319 12920 raft_consensus.cc:493] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.594972 12920 raft_consensus.cc:3060] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.597714 12915 raft_consensus.cc:359] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.598672 12915 raft_consensus.cc:385] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.599015 12915 raft_consensus.cc:740] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.599857 12915 consensus_queue.cc:260] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.600023 12948 raft_consensus.cc:515] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.604502 12914 tablet_bootstrap.cc:492] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:17.609113 12914 ts_tablet_manager.cc:1403] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.072s	user 0.018s	sys 0.006s
I20260709 13:06:17.610057 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf73d1d6c504892aca0e7c5abf13da7" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:17.610888 12722 raft_consensus.cc:3060] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.610345 12621 catalog_manager.cc:5697] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3a9fb085add94dd99d22ee2b1129932c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.614632 12914 raft_consensus.cc:359] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.615897 12914 raft_consensus.cc:385] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.615795 12920 raft_consensus.cc:515] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.616528 12914 raft_consensus.cc:740] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.617878 12914 consensus_queue.cc:260] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.619922 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a98c8d37fff242259cc617625b0f83b0" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:17.621317 12914 ts_tablet_manager.cc:1434] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:06:17.621357 12915 ts_tablet_manager.cc:1434] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.016s
I20260709 13:06:17.623247 12915 tablet_bootstrap.cc:492] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:17.623309 12914 tablet_bootstrap.cc:492] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:17.626865 12722 raft_consensus.cc:2468] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 1.
I20260709 13:06:17.643633 12915 tablet_bootstrap.cc:654] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.644183 12914 tablet_bootstrap.cc:654] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:17.635248 12765 leader_election.cc:343] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:17.649431 12765 leader_election.cc:304] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:17.656342 12920 leader_election.cc:290] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.657475 12920 raft_consensus.cc:2804] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:17.658092 12920 raft_consensus.cc:697] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Becoming Leader. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:17.661942 12920 consensus_queue.cc:237] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.664847 12948 leader_election.cc:290] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.664718 12873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a98c8d37fff242259cc617625b0f83b0" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:17.665911 12873 raft_consensus.cc:3060] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.667212 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf73d1d6c504892aca0e7c5abf13da7" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:17.668159 12872 raft_consensus.cc:3060] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.662607 12913 tablet_bootstrap.cc:492] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:17.677883 12913 ts_tablet_manager.cc:1403] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.147s	user 0.013s	sys 0.016s
I20260709 13:06:17.684085 12913 raft_consensus.cc:359] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.685446 12913 raft_consensus.cc:385] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.685968 12913 raft_consensus.cc:740] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.687552 12913 consensus_queue.cc:260] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.704919 12873 raft_consensus.cc:2468] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 1.
I20260709 13:06:17.706781 12765 leader_election.cc:304] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 3a9fb085add94dd99d22ee2b1129932c
I20260709 13:06:17.707305 12913 ts_tablet_manager.cc:1434] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.008s
I20260709 13:06:17.708307 12948 raft_consensus.cc:2804] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:17.709039 12948 raft_consensus.cc:697] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Becoming Leader. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:17.709276 12915 tablet_bootstrap.cc:492] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:17.709795 12915 ts_tablet_manager.cc:1403] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.087s	user 0.016s	sys 0.063s
I20260709 13:06:17.710072 12914 tablet_bootstrap.cc:492] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:17.710736 12914 ts_tablet_manager.cc:1403] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.088s	user 0.024s	sys 0.020s
I20260709 13:06:17.710314 12948 consensus_queue.cc:237] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.708976 12913 tablet_bootstrap.cc:492] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:17.714784 12914 raft_consensus.cc:359] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.715770 12914 raft_consensus.cc:385] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.716195 12914 raft_consensus.cc:740] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.717284 12914 consensus_queue.cc:260] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.719878 12872 raft_consensus.cc:2468] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 1.
I20260709 13:06:17.722491 12913 tablet_bootstrap.cc:654] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.724623 12620 catalog_manager.cc:5697] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc1fcc33b1c04369ab276792291f3331 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "fc1fcc33b1c04369ab276792291f3331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.728477 12914 ts_tablet_manager.cc:1434] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
I20260709 13:06:17.730337 12914 tablet_bootstrap.cc:492] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:17.727324 12915 raft_consensus.cc:359] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.733868 12915 raft_consensus.cc:385] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.734198 12915 raft_consensus.cc:740] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.737682 12915 consensus_queue.cc:260] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.740062 12915 ts_tablet_manager.cc:1434] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 13:06:17.740393 12914 tablet_bootstrap.cc:654] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.741688 12915 tablet_bootstrap.cc:492] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:17.746861 12914 tablet_bootstrap.cc:492] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:17.747638 12914 ts_tablet_manager.cc:1403] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 13:06:17.748194 12913 tablet_bootstrap.cc:492] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:17.748766 12915 tablet_bootstrap.cc:654] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.749315 12948 raft_consensus.cc:493] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.750203 12948 raft_consensus.cc:515] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.750659 12914 raft_consensus.cc:359] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.751490 12914 raft_consensus.cc:385] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.751883 12914 raft_consensus.cc:740] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.752727 12948 leader_election.cc:290] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.748852 12913 ts_tablet_manager.cc:1403] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.004s
I20260709 13:06:17.752964 12914 consensus_queue.cc:260] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.755571 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51bbf038592a410da9f6958348a1934b" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:17.755865 12914 ts_tablet_manager.cc:1434] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:17.756374 12872 raft_consensus.cc:2468] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 0.
I20260709 13:06:17.757194 12914 tablet_bootstrap.cc:492] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:17.753548 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51bbf038592a410da9f6958348a1934b" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
I20260709 13:06:17.757246 12913 raft_consensus.cc:359] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.758306 12913 raft_consensus.cc:385] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.758479 12765 leader_election.cc:304] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:17.758703 12913 raft_consensus.cc:740] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.759569 12948 raft_consensus.cc:2804] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:17.760015 12948 raft_consensus.cc:493] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.759830 12913 consensus_queue.cc:260] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.760437 12948 raft_consensus.cc:3060] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.764552 12914 tablet_bootstrap.cc:654] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.763370 12618 catalog_manager.cc:5697] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc1fcc33b1c04369ab276792291f3331 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "fc1fcc33b1c04369ab276792291f3331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.775916 12914 tablet_bootstrap.cc:492] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:17.776427 12914 ts_tablet_manager.cc:1403] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.001s
I20260709 13:06:17.778553 12915 tablet_bootstrap.cc:492] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:17.779093 12915 ts_tablet_manager.cc:1403] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.001s
I20260709 13:06:17.773449 12948 raft_consensus.cc:515] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.779263 12914 raft_consensus.cc:359] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.780071 12914 raft_consensus.cc:385] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.779582 12722 raft_consensus.cc:2468] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 0.
I20260709 13:06:17.780387 12914 raft_consensus.cc:740] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.781388 12914 consensus_queue.cc:260] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.782486 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51bbf038592a410da9f6958348a1934b" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:17.783785 12914 ts_tablet_manager.cc:1434] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:06:17.782011 12915 raft_consensus.cc:359] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.785970 12915 raft_consensus.cc:385] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.786234 12915 raft_consensus.cc:740] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.786474 12913 ts_tablet_manager.cc:1434] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:06:17.787022 12915 consensus_queue.cc:260] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.788065 12913 tablet_bootstrap.cc:492] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:17.791606 12915 ts_tablet_manager.cc:1434] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:06:17.791534 12948 leader_election.cc:290] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.795898 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51bbf038592a410da9f6958348a1934b" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:17.796684 12913 tablet_bootstrap.cc:654] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.796666 12872 raft_consensus.cc:3060] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.805536 12872 raft_consensus.cc:2468] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 1.
I20260709 13:06:17.805683 12913 tablet_bootstrap.cc:492] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:17.806414 12913 ts_tablet_manager.cc:1403] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.008s
I20260709 13:06:17.806798 12765 leader_election.cc:304] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 3a9fb085add94dd99d22ee2b1129932c
I20260709 13:06:17.808118 12948 raft_consensus.cc:2804] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:17.808565 12948 raft_consensus.cc:697] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Becoming Leader. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:17.809527 12948 consensus_queue.cc:237] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.809711 12913 raft_consensus.cc:359] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.810621 12913 raft_consensus.cc:385] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.810978 12913 raft_consensus.cc:740] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.812039 12913 consensus_queue.cc:260] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.815186 12913 ts_tablet_manager.cc:1434] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:06:17.816689 12913 tablet_bootstrap.cc:492] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:17.819284 12620 catalog_manager.cc:5697] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc1fcc33b1c04369ab276792291f3331 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "fc1fcc33b1c04369ab276792291f3331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.824800 12913 tablet_bootstrap.cc:654] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.828306 12922 raft_consensus.cc:493] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.828990 12922 raft_consensus.cc:515] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.831678 12922 leader_election.cc:290] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.831671 12913 tablet_bootstrap.cc:492] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:17.832850 12913 ts_tablet_manager.cc:1403] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 13:06:17.833266 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fa856455174351b44f0fa9466fbf4d" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:17.833345 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fa856455174351b44f0fa9466fbf4d" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:17.834168 12872 raft_consensus.cc:2468] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 0.
I20260709 13:06:17.834260 12798 raft_consensus.cc:2468] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 0.
I20260709 13:06:17.835901 12690 leader_election.cc:304] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:17.836066 12913 raft_consensus.cc:359] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.836984 12913 raft_consensus.cc:385] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.837317 12913 raft_consensus.cc:740] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.837299 12922 raft_consensus.cc:2804] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:17.838024 12922 raft_consensus.cc:493] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.838416 12922 raft_consensus.cc:3060] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.838487 12913 consensus_queue.cc:260] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.841204 12913 ts_tablet_manager.cc:1434] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:17.842711 12913 tablet_bootstrap.cc:492] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:17.849711 12913 tablet_bootstrap.cc:654] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:17.849296 12922 raft_consensus.cc:515] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.852916 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fa856455174351b44f0fa9466fbf4d" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:17.853559 12798 raft_consensus.cc:3060] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.853910 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fa856455174351b44f0fa9466fbf4d" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:17.854854 12872 raft_consensus.cc:3060] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.853207 12922 leader_election.cc:290] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Requested vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.860107 12798 raft_consensus.cc:2468] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 1.
I20260709 13:06:17.861439 12688 leader_election.cc:304] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:17.861694 12913 tablet_bootstrap.cc:492] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:17.862270 12922 raft_consensus.cc:2804] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:17.862325 12913 ts_tablet_manager.cc:1403] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.002s
I20260709 13:06:17.862812 12922 raft_consensus.cc:697] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Becoming Leader. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:17.863739 12922 consensus_queue.cc:237] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.864759 12872 raft_consensus.cc:2468] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 1.
I20260709 13:06:17.865605 12913 raft_consensus.cc:359] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.866420 12913 raft_consensus.cc:385] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:17.866720 12913 raft_consensus.cc:740] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:17.867738 12913 consensus_queue.cc:260] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.870167 12913 ts_tablet_manager.cc:1434] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:17.872993 12620 catalog_manager.cc:5697] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3a9fb085add94dd99d22ee2b1129932c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.887068 12922 raft_consensus.cc:493] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.887750 12922 raft_consensus.cc:515] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.889771 12922 leader_election.cc:290] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.890676 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb3192718d94c2e850c178a9ad03e1d" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:17.891165 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb3192718d94c2e850c178a9ad03e1d" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:17.891417 12798 raft_consensus.cc:2468] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 0.
I20260709 13:06:17.892043 12872 raft_consensus.cc:2468] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 0.
I20260709 13:06:17.892709 12688 leader_election.cc:304] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:17.893792 12922 raft_consensus.cc:2804] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:17.894088 12922 raft_consensus.cc:493] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.894330 12922 raft_consensus.cc:3060] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.901078 12922 raft_consensus.cc:515] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.903048 12922 leader_election.cc:290] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Requested vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.903916 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb3192718d94c2e850c178a9ad03e1d" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:17.904139 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb3192718d94c2e850c178a9ad03e1d" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:17.904508 12798 raft_consensus.cc:3060] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.904765 12872 raft_consensus.cc:3060] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.912174 12872 raft_consensus.cc:2468] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 1.
I20260709 13:06:17.913672 12690 leader_election.cc:304] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:17.914062 12798 raft_consensus.cc:2468] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 1.
I20260709 13:06:17.914769 12922 raft_consensus.cc:2804] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:17.915386 12922 raft_consensus.cc:697] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Becoming Leader. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:17.916574 12922 consensus_queue.cc:237] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.923978 12620 catalog_manager.cc:5697] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3a9fb085add94dd99d22ee2b1129932c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.938724 12922 raft_consensus.cc:493] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.939265 12922 raft_consensus.cc:515] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.941236 12922 leader_election.cc:290] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.941937 12948 raft_consensus.cc:493] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.942612 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4fe0354684a4ea08b937a39cec88abb" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:17.942620 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4fe0354684a4ea08b937a39cec88abb" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:17.942687 12948 raft_consensus.cc:515] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.943624 12798 raft_consensus.cc:2468] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 0.
I20260709 13:06:17.943805 12872 raft_consensus.cc:2468] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 0.
I20260709 13:06:17.944955 12688 leader_election.cc:304] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:17.945456 12948 leader_election.cc:290] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.946087 12922 raft_consensus.cc:2804] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:17.946540 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c4c48405d84950bc27d4972be8a04c" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
I20260709 13:06:17.946559 12922 raft_consensus.cc:493] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.946692 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c4c48405d84950bc27d4972be8a04c" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:17.947309 12723 raft_consensus.cc:2468] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 0.
I20260709 13:06:17.947382 12922 raft_consensus.cc:3060] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.947567 12872 raft_consensus.cc:2468] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 0.
I20260709 13:06:17.950718 12765 leader_election.cc:304] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:17.951639 12948 raft_consensus.cc:2804] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:17.952035 12948 raft_consensus.cc:493] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.952415 12948 raft_consensus.cc:3060] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.958074 12922 raft_consensus.cc:515] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.960564 12922 leader_election.cc:290] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Requested vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.960367 12948 raft_consensus.cc:515] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.961488 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4fe0354684a4ea08b937a39cec88abb" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:17.962108 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4fe0354684a4ea08b937a39cec88abb" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:17.962301 12798 raft_consensus.cc:3060] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.962790 12948 leader_election.cc:290] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.963905 12872 raft_consensus.cc:3060] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.964018 12873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c4c48405d84950bc27d4972be8a04c" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:17.964655 12873 raft_consensus.cc:3060] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.965690 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c4c48405d84950bc27d4972be8a04c" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:17.966410 12723 raft_consensus.cc:3060] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:17.970414 12919 consensus_queue.cc:1048] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:17.971554 12873 raft_consensus.cc:2468] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 1.
I20260709 13:06:17.972962 12765 leader_election.cc:304] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:17.973801 12948 raft_consensus.cc:2804] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:17.974257 12798 raft_consensus.cc:2468] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 1.
I20260709 13:06:17.975224 12948 raft_consensus.cc:697] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Becoming Leader. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:17.976198 12948 consensus_queue.cc:237] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.976559 12945 consensus_queue.cc:1048] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:17.977967 12945 raft_consensus.cc:493] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.978408 12945 raft_consensus.cc:515] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:17.980758 12945 leader_election.cc:290] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:17.985368 12723 raft_consensus.cc:2468] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 1.
I20260709 13:06:17.987437 12688 leader_election.cc:304] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:17.989737 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a98c8d37fff242259cc617625b0f83b0" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:17.991066 12870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a98c8d37fff242259cc617625b0f83b0" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:17.991914 12870 raft_consensus.cc:2393] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a9fb085add94dd99d22ee2b1129932c in current term 1: Already voted for candidate fc1fcc33b1c04369ab276792291f3331 in this term.
I20260709 13:06:17.992540 12973 raft_consensus.cc:2804] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:17.993244 12973 raft_consensus.cc:697] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Becoming Leader. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:18.001667 12872 raft_consensus.cc:2468] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 1.
I20260709 13:06:17.995553 12973 consensus_queue.cc:237] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.007182 12919 consensus_queue.cc:1048] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:18.007910 12688 leader_election.cc:304] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c; no voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331
I20260709 13:06:18.009480 12620 catalog_manager.cc:5697] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc1fcc33b1c04369ab276792291f3331 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "fc1fcc33b1c04369ab276792291f3331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.010875 12973 raft_consensus.cc:3060] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:18.019011 12922 consensus_queue.cc:1048] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:18.021714 12973 raft_consensus.cc:2749] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:18.024328 12619 catalog_manager.cc:5697] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3a9fb085add94dd99d22ee2b1129932c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.085177 12948 consensus_queue.cc:1048] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.099191 12920 consensus_queue.cc:1048] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.110502 12920 consensus_queue.cc:1048] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:18.135670 12948 consensus_queue.cc:1048] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.138013 12799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a98c8d37fff242259cc617625b0f83b0"
dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
mode: GRACEFUL
new_leader_uuid: "3a9fb085add94dd99d22ee2b1129932c"
 from {username='slave'} at 127.0.0.1:39408
I20260709 13:06:18.139005 12799 raft_consensus.cc:606] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Received request to transfer leadership to TS 3a9fb085add94dd99d22ee2b1129932c
I20260709 13:06:18.147042 12948 raft_consensus.cc:993] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331: : Instructing follower 3a9fb085add94dd99d22ee2b1129932c to start an election
I20260709 13:06:18.147542 12948 raft_consensus.cc:1081] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Signalling peer 3a9fb085add94dd99d22ee2b1129932c to start an election
I20260709 13:06:18.147756 12799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51bbf038592a410da9f6958348a1934b"
dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
mode: GRACEFUL
new_leader_uuid: "3a9fb085add94dd99d22ee2b1129932c"
 from {username='slave'} at 127.0.0.1:39408
I20260709 13:06:18.148684 12799 raft_consensus.cc:606] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Received request to transfer leadership to TS 3a9fb085add94dd99d22ee2b1129932c
I20260709 13:06:18.150029 12723 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a98c8d37fff242259cc617625b0f83b0"
dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
 from {username='slave'} at 127.0.0.1:37706
I20260709 13:06:18.150900 12723 raft_consensus.cc:493] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.151351 12723 raft_consensus.cc:3060] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.156008 12799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcf73d1d6c504892aca0e7c5abf13da7"
dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
mode: GRACEFUL
new_leader_uuid: "3a9fb085add94dd99d22ee2b1129932c"
 from {username='slave'} at 127.0.0.1:39408
I20260709 13:06:18.156875 12799 raft_consensus.cc:606] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Received request to transfer leadership to TS 3a9fb085add94dd99d22ee2b1129932c
I20260709 13:06:18.159507 12723 raft_consensus.cc:515] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.163162 12799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a98c8d37fff242259cc617625b0f83b0" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:18.164042 12799 raft_consensus.cc:3055] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:18.164561 12799 raft_consensus.cc:740] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:18.165218 12723 leader_election.cc:290] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 2 election: Requested vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:18.165550 12799 consensus_queue.cc:260] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.166872 12871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a98c8d37fff242259cc617625b0f83b0" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:18.167199 12799 raft_consensus.cc:3060] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.167559 12871 raft_consensus.cc:3060] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.177554 12799 raft_consensus.cc:2468] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 2.
I20260709 13:06:18.178752 12688 leader_election.cc:304] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:18.179057 12871 raft_consensus.cc:2468] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 2.
I20260709 13:06:18.179632 12973 raft_consensus.cc:2804] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:18.180058 12973 raft_consensus.cc:697] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [term 2 LEADER]: Becoming Leader. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:18.180884 12973 consensus_queue.cc:237] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.186158 12871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d38859ad5d584abab1d3d1c93729a5f4"
dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
mode: GRACEFUL
new_leader_uuid: "fc1fcc33b1c04369ab276792291f3331"
 from {username='slave'} at 127.0.0.1:38446
I20260709 13:06:18.186851 12871 raft_consensus.cc:606] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Received request to transfer leadership to TS fc1fcc33b1c04369ab276792291f3331
I20260709 13:06:18.189462 12618 catalog_manager.cc:5697] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c reported cstate change: term changed from 1 to 2, leader changed from fc1fcc33b1c04369ab276792291f3331 (127.0.107.194) to 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3a9fb085add94dd99d22ee2b1129932c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.203979 12723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fb3192718d94c2e850c178a9ad03e1d"
dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
mode: GRACEFUL
new_leader_uuid: "fc1fcc33b1c04369ab276792291f3331"
 from {username='slave'} at 127.0.0.1:37708
I20260709 13:06:18.204533 12723 raft_consensus.cc:606] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Received request to transfer leadership to TS fc1fcc33b1c04369ab276792291f3331
I20260709 13:06:18.208376 12723 raft_consensus.cc:3060] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:18.214003 12722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17fa856455174351b44f0fa9466fbf4d"
dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
mode: GRACEFUL
new_leader_uuid: "fc1fcc33b1c04369ab276792291f3331"
 from {username='slave'} at 127.0.0.1:37708
I20260709 13:06:18.214679 12722 raft_consensus.cc:606] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Received request to transfer leadership to TS fc1fcc33b1c04369ab276792291f3331
I20260709 13:06:18.217536 12948 consensus_queue.cc:1048] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.221678 12722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4fe0354684a4ea08b937a39cec88abb"
dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
mode: GRACEFUL
new_leader_uuid: "4a2c916532db4ec1b36070c705d7e07f"
 from {username='slave'} at 127.0.0.1:37708
I20260709 13:06:18.222312 12722 raft_consensus.cc:606] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Received request to transfer leadership to TS 4a2c916532db4ec1b36070c705d7e07f
I20260709 13:06:18.225411 12989 raft_consensus.cc:993] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331: : Instructing follower 3a9fb085add94dd99d22ee2b1129932c to start an election
I20260709 13:06:18.225895 12989 raft_consensus.cc:1081] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Signalling peer 3a9fb085add94dd99d22ee2b1129932c to start an election
I20260709 13:06:18.228679 12722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51bbf038592a410da9f6958348a1934b"
dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
 from {username='slave'} at 127.0.0.1:37706
I20260709 13:06:18.229367 12722 raft_consensus.cc:493] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.229727 12722 raft_consensus.cc:3060] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.232755 12989 consensus_queue.cc:1048] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:18.242489 12722 raft_consensus.cc:515] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.243698 12723 raft_consensus.cc:1240] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Rejecting Update request from peer fc1fcc33b1c04369ab276792291f3331 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:18.245368 12722 leader_election.cc:290] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 2 election: Requested vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:18.245617 12799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51bbf038592a410da9f6958348a1934b" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:18.246058 12871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51bbf038592a410da9f6958348a1934b" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:18.246374 12799 raft_consensus.cc:3055] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:18.247200 12799 raft_consensus.cc:740] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:18.247246 12871 raft_consensus.cc:3060] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.248193 12799 consensus_queue.cc:260] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.249490 12799 raft_consensus.cc:3060] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.253362 12871 raft_consensus.cc:2468] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 2.
I20260709 13:06:18.254658 12690 leader_election.cc:304] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:18.254856 12799 raft_consensus.cc:2468] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 2.
I20260709 13:06:18.255641 12973 raft_consensus.cc:2804] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:18.256145 12973 raft_consensus.cc:697] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [term 2 LEADER]: Becoming Leader. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:18.257009 12973 consensus_queue.cc:237] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.264273 12619 catalog_manager.cc:5697] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c reported cstate change: term changed from 1 to 2, leader changed from fc1fcc33b1c04369ab276792291f3331 (127.0.107.194) to 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3a9fb085add94dd99d22ee2b1129932c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.279435 12973 consensus_queue.cc:1048] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.287850 12945 raft_consensus.cc:993] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c: : Instructing follower fc1fcc33b1c04369ab276792291f3331 to start an election
I20260709 13:06:18.288414 12922 raft_consensus.cc:1081] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Signalling peer fc1fcc33b1c04369ab276792291f3331 to start an election
I20260709 13:06:18.291790 12798 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17fa856455174351b44f0fa9466fbf4d"
dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
 from {username='slave'} at 127.0.0.1:39388
I20260709 13:06:18.292428 12798 raft_consensus.cc:493] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.292891 12798 raft_consensus.cc:3060] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.298116 12973 consensus_queue.cc:1048] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.302134 12798 raft_consensus.cc:515] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.305585 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fa856455174351b44f0fa9466fbf4d" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:18.306149 12723 raft_consensus.cc:3055] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:18.306432 12723 raft_consensus.cc:740] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:18.307214 12723 consensus_queue.cc:260] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.308327 12723 raft_consensus.cc:3060] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.310220 12798 leader_election.cc:290] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 2 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:18.311622 12871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fa856455174351b44f0fa9466fbf4d" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:18.312395 12871 raft_consensus.cc:3060] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.314450 12723 raft_consensus.cc:2468] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 2.
I20260709 13:06:18.315668 12765 leader_election.cc:304] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:18.316745 12989 raft_consensus.cc:2804] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:18.317281 12989 raft_consensus.cc:697] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Becoming Leader. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:18.317955 12989 consensus_queue.cc:237] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.319010 12871 raft_consensus.cc:2468] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 2.
I20260709 13:06:18.325819 12619 catalog_manager.cc:5697] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 reported cstate change: term changed from 1 to 2, leader changed from 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193) to fc1fcc33b1c04369ab276792291f3331 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "fc1fcc33b1c04369ab276792291f3331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.404529 12989 consensus_queue.cc:1048] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.419473 12989 consensus_queue.cc:1048] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.473017 12973 consensus_queue.cc:1048] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.485133 12945 consensus_queue.cc:1048] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.493973 12973 raft_consensus.cc:993] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c: : Instructing follower fc1fcc33b1c04369ab276792291f3331 to start an election
I20260709 13:06:18.494531 12945 raft_consensus.cc:1081] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Signalling peer fc1fcc33b1c04369ab276792291f3331 to start an election
I20260709 13:06:18.496377 12798 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fb3192718d94c2e850c178a9ad03e1d"
dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
 from {username='slave'} at 127.0.0.1:39388
I20260709 13:06:18.497061 12798 raft_consensus.cc:493] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.497526 12798 raft_consensus.cc:3060] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.502674 12989 raft_consensus.cc:993] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331: : Instructing follower 3a9fb085add94dd99d22ee2b1129932c to start an election
I20260709 13:06:18.503355 12948 raft_consensus.cc:1081] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Signalling peer 3a9fb085add94dd99d22ee2b1129932c to start an election
I20260709 13:06:18.505237 12722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bcf73d1d6c504892aca0e7c5abf13da7"
dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
 from {username='slave'} at 127.0.0.1:37706
I20260709 13:06:18.505764 12722 raft_consensus.cc:493] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.505762 12798 raft_consensus.cc:515] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.506651 12722 raft_consensus.cc:3060] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.508441 12798 leader_election.cc:290] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 2 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:18.509786 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb3192718d94c2e850c178a9ad03e1d" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:18.509716 12871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb3192718d94c2e850c178a9ad03e1d" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:18.510653 12871 raft_consensus.cc:3060] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.510722 12723 raft_consensus.cc:3055] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:18.511245 12723 raft_consensus.cc:740] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:18.512157 12723 consensus_queue.cc:260] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.513597 12723 raft_consensus.cc:3060] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.516754 12722 raft_consensus.cc:515] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.519995 12722 leader_election.cc:290] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 2 election: Requested vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:18.519853 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf73d1d6c504892aca0e7c5abf13da7" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:18.520242 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf73d1d6c504892aca0e7c5abf13da7" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:18.520627 12798 raft_consensus.cc:3055] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:18.520677 12871 raft_consensus.cc:2468] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 2.
I20260709 13:06:18.520989 12872 raft_consensus.cc:3060] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.521152 12798 raft_consensus.cc:740] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:18.522529 12765 leader_election.cc:304] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:18.522647 12798 consensus_queue.cc:260] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.524228 12798 raft_consensus.cc:3060] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.524379 12948 raft_consensus.cc:2804] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:18.524991 12948 raft_consensus.cc:697] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Becoming Leader. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:18.525938 12948 consensus_queue.cc:237] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.527662 12872 raft_consensus.cc:2468] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 2.
I20260709 13:06:18.529505 12690 leader_election.cc:304] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:18.530035 12723 raft_consensus.cc:2468] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 2.
I20260709 13:06:18.530608 12945 raft_consensus.cc:2804] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:18.532106 12945 raft_consensus.cc:697] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [term 2 LEADER]: Becoming Leader. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:18.532826 12798 raft_consensus.cc:2468] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 2.
I20260709 13:06:18.533129 12945 consensus_queue.cc:237] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.536437 12973 consensus_queue.cc:1048] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.538187 12618 catalog_manager.cc:5697] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 reported cstate change: term changed from 1 to 2, leader changed from 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193) to fc1fcc33b1c04369ab276792291f3331 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "fc1fcc33b1c04369ab276792291f3331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.544154 12619 catalog_manager.cc:5697] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c reported cstate change: term changed from 1 to 2, leader changed from fc1fcc33b1c04369ab276792291f3331 (127.0.107.194) to 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3a9fb085add94dd99d22ee2b1129932c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.546828 12973 raft_consensus.cc:993] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c: : Instructing follower 4a2c916532db4ec1b36070c705d7e07f to start an election
I20260709 13:06:18.547477 12973 raft_consensus.cc:1081] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Signalling peer 4a2c916532db4ec1b36070c705d7e07f to start an election
I20260709 13:06:18.551270 12872 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4fe0354684a4ea08b937a39cec88abb"
dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
 from {username='slave'} at 127.0.0.1:38416
I20260709 13:06:18.551927 12872 raft_consensus.cc:493] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.552325 12872 raft_consensus.cc:3060] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.559928 12945 consensus_queue.cc:1048] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:18.571132 12872 raft_consensus.cc:515] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.575743 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4fe0354684a4ea08b937a39cec88abb" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:18.576726 12723 raft_consensus.cc:3055] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:18.577194 12723 raft_consensus.cc:740] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:18.578398 12723 consensus_queue.cc:260] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.580399 12723 raft_consensus.cc:3060] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.584801 12871 raft_consensus.cc:1240] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Rejecting Update request from peer 3a9fb085add94dd99d22ee2b1129932c for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:18.586679 12872 leader_election.cc:290] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 2 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:18.587841 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4fe0354684a4ea08b937a39cec88abb" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:18.588053 13003 raft_consensus.cc:993] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f: : Instructing follower fc1fcc33b1c04369ab276792291f3331 to start an election
I20260709 13:06:18.588577 12798 raft_consensus.cc:3060] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.588958 13003 raft_consensus.cc:1081] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Signalling peer fc1fcc33b1c04369ab276792291f3331 to start an election
I20260709 13:06:18.590971 12799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d38859ad5d584abab1d3d1c93729a5f4"
dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
 from {username='slave'} at 127.0.0.1:39402
I20260709 13:06:18.590902 12723 raft_consensus.cc:2468] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 2.
I20260709 13:06:18.591696 12799 raft_consensus.cc:493] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.592105 12799 raft_consensus.cc:3060] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.592346 12839 leader_election.cc:304] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:18.593186 13003 raft_consensus.cc:2804] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:18.593693 13003 raft_consensus.cc:697] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [term 2 LEADER]: Becoming Leader. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Running, Role: LEADER
I20260709 13:06:18.594751 13003 consensus_queue.cc:237] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.595798 12798 raft_consensus.cc:2468] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 2.
I20260709 13:06:18.600072 12799 raft_consensus.cc:515] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.602288 12799 leader_election.cc:290] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 2 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:18.603081 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38859ad5d584abab1d3d1c93729a5f4" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:18.603685 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38859ad5d584abab1d3d1c93729a5f4" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:18.603951 12723 raft_consensus.cc:3060] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.603188 12618 catalog_manager.cc:5697] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f reported cstate change: term changed from 1 to 2, leader changed from 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193) to 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195). New cstate: current_term: 2 leader_uuid: "4a2c916532db4ec1b36070c705d7e07f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:18.604358 12872 raft_consensus.cc:3055] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:18.604727 12872 raft_consensus.cc:740] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Running, Role: LEADER
I20260709 13:06:18.605589 12872 consensus_queue.cc:260] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.606851 12872 raft_consensus.cc:3060] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:18.611207 12723 raft_consensus.cc:2468] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 2.
I20260709 13:06:18.612830 12765 leader_election.cc:304] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:18.612960 12872 raft_consensus.cc:2468] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 2.
I20260709 13:06:18.613564 12948 raft_consensus.cc:2804] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:18.614049 12948 raft_consensus.cc:697] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Becoming Leader. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:18.615005 12948 consensus_queue.cc:237] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:18.621733 12618 catalog_manager.cc:5697] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 reported cstate change: term changed from 1 to 2, leader changed from 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195) to fc1fcc33b1c04369ab276792291f3331 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "fc1fcc33b1c04369ab276792291f3331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.641976 12872 raft_consensus.cc:1275] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Refusing update from remote peer 3a9fb085add94dd99d22ee2b1129932c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:18.643342 12945 consensus_queue.cc:1048] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:18.652426 12799 raft_consensus.cc:1275] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Refusing update from remote peer 3a9fb085add94dd99d22ee2b1129932c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:18.653885 13005 consensus_queue.cc:1048] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:18.707764 12723 raft_consensus.cc:1275] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Refusing update from remote peer fc1fcc33b1c04369ab276792291f3331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:18.709518 12948 consensus_queue.cc:1048] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:18.718860 12872 raft_consensus.cc:1275] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Refusing update from remote peer fc1fcc33b1c04369ab276792291f3331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:18.721215 12989 consensus_queue.cc:1048] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:18.756055 12799 raft_consensus.cc:1275] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Refusing update from remote peer 3a9fb085add94dd99d22ee2b1129932c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:18.757767 13005 consensus_queue.cc:1048] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:18.768697 12872 raft_consensus.cc:1275] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Refusing update from remote peer 3a9fb085add94dd99d22ee2b1129932c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:18.771512 12945 consensus_queue.cc:1048] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:18.948356 12723 raft_consensus.cc:1275] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Refusing update from remote peer fc1fcc33b1c04369ab276792291f3331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:18.950503 12989 consensus_queue.cc:1048] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:19.011484 12872 raft_consensus.cc:1275] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Refusing update from remote peer fc1fcc33b1c04369ab276792291f3331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:19.013837 12948 consensus_queue.cc:1048] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:19.015805 12722 raft_consensus.cc:1275] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Refusing update from remote peer fc1fcc33b1c04369ab276792291f3331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:19.018144 12948 consensus_queue.cc:1048] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:19.029971 12798 raft_consensus.cc:1275] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Refusing update from remote peer 3a9fb085add94dd99d22ee2b1129932c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:19.032173 12945 consensus_queue.cc:1048] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:19.036777 12872 raft_consensus.cc:1275] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Refusing update from remote peer fc1fcc33b1c04369ab276792291f3331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:19.039345 12948 consensus_queue.cc:1048] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:19.062220 12872 raft_consensus.cc:1275] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Refusing update from remote peer 3a9fb085add94dd99d22ee2b1129932c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:19.063865 12922 consensus_queue.cc:1048] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:19.128492 12722 raft_consensus.cc:1275] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Refusing update from remote peer 4a2c916532db4ec1b36070c705d7e07f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:19.129900 13004 consensus_queue.cc:1048] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:19.139544 12799 raft_consensus.cc:1275] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Refusing update from remote peer 4a2c916532db4ec1b36070c705d7e07f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:19.141283 13003 consensus_queue.cc:1048] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:19.971122 12745 debug-util.cc:398] Leaking SignalData structure 0x7b08001ef2c0 after lost signal to thread 434
W20260709 13:06:19.972321 12745 debug-util.cc:398] Leaking SignalData structure 0x7b0800044720 after lost signal to thread 12656
W20260709 13:06:19.973501 12745 debug-util.cc:398] Leaking SignalData structure 0x7b08001c0e80 after lost signal to thread 12748
W20260709 13:06:19.974537 12745 debug-util.cc:398] Leaking SignalData structure 0x7b0800356980 after lost signal to thread 12823
W20260709 13:06:19.975605 12745 debug-util.cc:398] Leaking SignalData structure 0x7b0800211660 after lost signal to thread 12897
I20260709 13:06:21.271471 12620 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43224:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:21.275393 12620 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:21.337344 12712 tablet_service.cc:1511] Processing CreateTablet for tablet b7cca9fc63574e26a1b1aac317816794 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:21.339453 12712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cca9fc63574e26a1b1aac317816794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.343243 12789 tablet_service.cc:1511] Processing CreateTablet for tablet b7cca9fc63574e26a1b1aac317816794 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:21.344944 12789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cca9fc63574e26a1b1aac317816794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.353879 12858 tablet_service.cc:1511] Processing CreateTablet for tablet b7cca9fc63574e26a1b1aac317816794 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:21.355463 12858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cca9fc63574e26a1b1aac317816794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.356477 12708 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee37120e5a4496bbea79d807e5ed39a (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:21.358006 12708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee37120e5a4496bbea79d807e5ed39a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.370682 12784 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee37120e5a4496bbea79d807e5ed39a (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:21.372268 12784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee37120e5a4496bbea79d807e5ed39a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.389309 12860 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee37120e5a4496bbea79d807e5ed39a (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:21.389663 12711 tablet_service.cc:1511] Processing CreateTablet for tablet 25c0c0ccabef4e72bf35ad3c1dc83f17 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:21.391000 12711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25c0c0ccabef4e72bf35ad3c1dc83f17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.391147 12860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee37120e5a4496bbea79d807e5ed39a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.402401 12785 tablet_service.cc:1511] Processing CreateTablet for tablet 25c0c0ccabef4e72bf35ad3c1dc83f17 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:21.403903 12785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25c0c0ccabef4e72bf35ad3c1dc83f17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.412153 13023 tablet_bootstrap.cc:492] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:21.419752 12859 tablet_service.cc:1511] Processing CreateTablet for tablet 25c0c0ccabef4e72bf35ad3c1dc83f17 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:21.422499 12859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25c0c0ccabef4e72bf35ad3c1dc83f17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.451220 12713 tablet_service.cc:1511] Processing CreateTablet for tablet f4318a4373b54622a1aa813bc53cfb44 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:21.452739 12713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4318a4373b54622a1aa813bc53cfb44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.464555 13022 tablet_bootstrap.cc:492] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:21.470319 13023 tablet_bootstrap.cc:654] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:21.476490 13024 tablet_bootstrap.cc:492] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:21.483019 13023 tablet_bootstrap.cc:492] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:21.485329 13023 ts_tablet_manager.cc:1403] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.073s	user 0.014s	sys 0.009s
I20260709 13:06:21.488142 13023 raft_consensus.cc:359] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.489182 13023 raft_consensus.cc:385] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:21.489523 13023 raft_consensus.cc:740] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:21.490393 13023 consensus_queue.cc:260] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.492848 13023 ts_tablet_manager.cc:1434] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:06:21.495658 13023 tablet_bootstrap.cc:492] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:21.496146 12789 tablet_service.cc:1511] Processing CreateTablet for tablet 5a8751836b2c4b6a8102c3dfa18d8c6b (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:21.497573 12789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a8751836b2c4b6a8102c3dfa18d8c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.502729 13023 tablet_bootstrap.cc:654] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:21.508064 12708 tablet_service.cc:1511] Processing CreateTablet for tablet 5a8751836b2c4b6a8102c3dfa18d8c6b (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:21.510000 12708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a8751836b2c4b6a8102c3dfa18d8c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.513300 12857 tablet_service.cc:1511] Processing CreateTablet for tablet 5a8751836b2c4b6a8102c3dfa18d8c6b (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:21.514755 12857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a8751836b2c4b6a8102c3dfa18d8c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.515726 12712 tablet_service.cc:1511] Processing CreateTablet for tablet 271e42be0a714018a752ea327df357ef (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:21.517169 12712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271e42be0a714018a752ea327df357ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.530910 12858 tablet_service.cc:1511] Processing CreateTablet for tablet f4318a4373b54622a1aa813bc53cfb44 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:21.531498 13023 tablet_bootstrap.cc:492] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:21.532219 13023 ts_tablet_manager.cc:1403] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.005s
I20260709 13:06:21.532706 12858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4318a4373b54622a1aa813bc53cfb44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.535478 12948 raft_consensus.cc:493] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.536062 13023 raft_consensus.cc:359] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.536952 13023 raft_consensus.cc:385] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:21.536183 12948 raft_consensus.cc:515] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.537935 12784 tablet_service.cc:1511] Processing CreateTablet for tablet f4318a4373b54622a1aa813bc53cfb44 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:21.539224 12948 leader_election.cc:290] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:21.539418 12784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4318a4373b54622a1aa813bc53cfb44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.541702 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cca9fc63574e26a1b1aac317816794" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
I20260709 13:06:21.543202 12871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cca9fc63574e26a1b1aac317816794" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:21.540097 12861 tablet_service.cc:1511] Processing CreateTablet for tablet 271e42be0a714018a752ea327df357ef (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:21.545584 12861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271e42be0a714018a752ea327df357ef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:21.546115 12765 leader_election.cc:343] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:21.546873 12765 leader_election.cc:343] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:21.547343 12765 leader_election.cc:304] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc1fcc33b1c04369ab276792291f3331; no voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f
I20260709 13:06:21.549141 13021 raft_consensus.cc:2749] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:21.557008 13024 tablet_bootstrap.cc:654] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:21.562875 12711 tablet_service.cc:1511] Processing CreateTablet for tablet 3111979238fe417aa43ba481ab84bfa0 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:21.564685 12711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3111979238fe417aa43ba481ab84bfa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.575584 12785 tablet_service.cc:1511] Processing CreateTablet for tablet 271e42be0a714018a752ea327df357ef (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:21.577602 12785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271e42be0a714018a752ea327df357ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.580359 13022 tablet_bootstrap.cc:654] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:21.537262 13023 raft_consensus.cc:740] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
W20260709 13:06:21.602283 12825 tablet.cc:2406] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:06:21.625967 13023 consensus_queue.cc:260] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.629356 13023 ts_tablet_manager.cc:1434] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.097s	user 0.010s	sys 0.000s
I20260709 13:06:21.629837 12783 tablet_service.cc:1511] Processing CreateTablet for tablet 3111979238fe417aa43ba481ab84bfa0 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:21.626807 12860 tablet_service.cc:1511] Processing CreateTablet for tablet 3111979238fe417aa43ba481ab84bfa0 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:21.641152 13024 tablet_bootstrap.cc:492] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:21.644804 12712 tablet_service.cc:1511] Processing CreateTablet for tablet e25d10382c5343c78b863083e4b4aa31 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:21.652680 12783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3111979238fe417aa43ba481ab84bfa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.653345 13024 ts_tablet_manager.cc:1403] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.177s	user 0.006s	sys 0.015s
I20260709 13:06:21.653708 12712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e25d10382c5343c78b863083e4b4aa31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.652678 12860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3111979238fe417aa43ba481ab84bfa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.658454 13023 tablet_bootstrap.cc:492] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:21.668279 12859 tablet_service.cc:1511] Processing CreateTablet for tablet e25d10382c5343c78b863083e4b4aa31 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:21.669718 12859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e25d10382c5343c78b863083e4b4aa31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.670397 13022 tablet_bootstrap.cc:492] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:21.671136 13022 ts_tablet_manager.cc:1403] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.207s	user 0.013s	sys 0.004s
I20260709 13:06:21.671999 12785 tablet_service.cc:1511] Processing CreateTablet for tablet e25d10382c5343c78b863083e4b4aa31 (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:21.673419 12785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e25d10382c5343c78b863083e4b4aa31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.661217 13024 raft_consensus.cc:359] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.685099 13024 raft_consensus.cc:385] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:21.686223 13024 raft_consensus.cc:740] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:21.688477 13024 consensus_queue.cc:260] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.687042 13022 raft_consensus.cc:359] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.692210 13022 raft_consensus.cc:385] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:21.692695 13022 raft_consensus.cc:740] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:21.698199 12708 tablet_service.cc:1511] Processing CreateTablet for tablet 121cd579cbe1405a81ad6f0a8693538b (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:21.699604 12708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 121cd579cbe1405a81ad6f0a8693538b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.697897 13022 consensus_queue.cc:260] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.694906 13024 ts_tablet_manager.cc:1434] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.041s	user 0.000s	sys 0.007s
I20260709 13:06:21.713467 13024 tablet_bootstrap.cc:492] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:21.701006 13023 tablet_bootstrap.cc:654] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:21.727083 13022 ts_tablet_manager.cc:1434] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.001s
I20260709 13:06:21.731642 13022 tablet_bootstrap.cc:492] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:21.741317 12860 tablet_service.cc:1511] Processing CreateTablet for tablet 121cd579cbe1405a81ad6f0a8693538b (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:21.747591 12860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 121cd579cbe1405a81ad6f0a8693538b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.748071 12783 tablet_service.cc:1511] Processing CreateTablet for tablet 121cd579cbe1405a81ad6f0a8693538b (DEFAULT_TABLE table=soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:21.750299 12783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 121cd579cbe1405a81ad6f0a8693538b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:21.760111 13024 tablet_bootstrap.cc:654] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:21.742478 13023 tablet_bootstrap.cc:492] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:21.761751 13023 ts_tablet_manager.cc:1403] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.104s	user 0.014s	sys 0.000s
I20260709 13:06:21.746443 13022 tablet_bootstrap.cc:654] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:21.765192 13023 raft_consensus.cc:359] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.766026 13023 raft_consensus.cc:385] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:21.766323 13023 raft_consensus.cc:740] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:21.767134 13023 consensus_queue.cc:260] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.775945 13023 ts_tablet_manager.cc:1434] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:06:21.781296 13022 tablet_bootstrap.cc:492] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:21.781957 13022 ts_tablet_manager.cc:1403] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.051s	user 0.005s	sys 0.012s
I20260709 13:06:21.783879 13023 tablet_bootstrap.cc:492] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:21.785334 13022 raft_consensus.cc:359] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.786247 13022 raft_consensus.cc:385] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:21.786569 13022 raft_consensus.cc:740] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:21.789219 13024 tablet_bootstrap.cc:492] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:21.789825 13024 ts_tablet_manager.cc:1403] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.077s	user 0.004s	sys 0.013s
I20260709 13:06:21.791501 13023 tablet_bootstrap.cc:654] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:21.793226 13024 raft_consensus.cc:359] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.794080 13024 raft_consensus.cc:385] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:21.794384 13024 raft_consensus.cc:740] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:21.795270 13024 consensus_queue.cc:260] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.797878 13024 ts_tablet_manager.cc:1434] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:06:21.798969 13024 tablet_bootstrap.cc:492] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:21.800519 13023 tablet_bootstrap.cc:492] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:21.801184 13023 ts_tablet_manager.cc:1403] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.004s
I20260709 13:06:21.803524 13022 consensus_queue.cc:260] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.804818 13023 raft_consensus.cc:359] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.805663 13023 raft_consensus.cc:385] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:21.806246 13023 raft_consensus.cc:740] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:21.807243 13023 consensus_queue.cc:260] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.814834 13022 ts_tablet_manager.cc:1434] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.001s
I20260709 13:06:21.819329 13023 ts_tablet_manager.cc:1434] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:06:21.820165 13022 tablet_bootstrap.cc:492] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:21.820657 13023 tablet_bootstrap.cc:492] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:21.828181 13023 tablet_bootstrap.cc:654] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:21.830904 13024 tablet_bootstrap.cc:654] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:21.835850 13022 tablet_bootstrap.cc:654] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:21.841059 13023 tablet_bootstrap.cc:492] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:21.841866 13023 ts_tablet_manager.cc:1403] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.000s
I20260709 13:06:21.850812 13023 raft_consensus.cc:359] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.852651 13021 raft_consensus.cc:493] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.853542 13021 raft_consensus.cc:515] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.857810 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8751836b2c4b6a8102c3dfa18d8c6b" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
W20260709 13:06:21.859347 12765 leader_election.cc:343] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:21.860456 13021 leader_election.cc:290] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:21.862098 12871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8751836b2c4b6a8102c3dfa18d8c6b" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
W20260709 13:06:21.863793 12765 leader_election.cc:343] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:21.864468 12765 leader_election.cc:304] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc1fcc33b1c04369ab276792291f3331; no voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f
I20260709 13:06:21.866076 13021 raft_consensus.cc:2749] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:21.851943 13023 raft_consensus.cc:385] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:21.868049 13023 raft_consensus.cc:740] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:21.869094 13023 consensus_queue.cc:260] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.870991 13024 tablet_bootstrap.cc:492] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:21.871821 13024 ts_tablet_manager.cc:1403] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.073s	user 0.029s	sys 0.010s
I20260709 13:06:21.876394 13024 raft_consensus.cc:359] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.877463 13024 raft_consensus.cc:385] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:21.877905 13024 raft_consensus.cc:740] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:21.879037 13024 consensus_queue.cc:260] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.881557 13023 ts_tablet_manager.cc:1434] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 13:06:21.884213 13024 ts_tablet_manager.cc:1434] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:06:21.885408 13024 tablet_bootstrap.cc:492] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:21.887671 13023 tablet_bootstrap.cc:492] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:21.892045 13021 raft_consensus.cc:493] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.892812 13024 tablet_bootstrap.cc:654] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:21.892764 13021 raft_consensus.cc:515] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.896216 13023 tablet_bootstrap.cc:654] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:21.896633 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25c0c0ccabef4e72bf35ad3c1dc83f17" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
W20260709 13:06:21.899530 12765 leader_election.cc:343] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:21.901340 13022 tablet_bootstrap.cc:492] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:21.902034 13022 ts_tablet_manager.cc:1403] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.082s	user 0.020s	sys 0.032s
I20260709 13:06:21.905206 12871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25c0c0ccabef4e72bf35ad3c1dc83f17" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:21.905846 13022 raft_consensus.cc:359] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.906404 12871 raft_consensus.cc:2468] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 0.
I20260709 13:06:21.907003 13022 raft_consensus.cc:385] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:21.907487 13022 raft_consensus.cc:740] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:21.908447 13022 consensus_queue.cc:260] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.908644 12765 leader_election.cc:304] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 3a9fb085add94dd99d22ee2b1129932c
I20260709 13:06:21.920750 12948 raft_consensus.cc:2804] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:21.921386 12948 raft_consensus.cc:493] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.921880 12948 raft_consensus.cc:3060] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:21.911590 13022 ts_tablet_manager.cc:1434] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:06:21.923900 13022 tablet_bootstrap.cc:492] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:21.919440 13008 raft_consensus.cc:493] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.927670 12945 raft_consensus.cc:493] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.927379 13008 raft_consensus.cc:515] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.928561 12945 raft_consensus.cc:515] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.932709 12799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cca9fc63574e26a1b1aac317816794" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:21.932891 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cca9fc63574e26a1b1aac317816794" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:21.933560 12799 raft_consensus.cc:2468] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 0.
I20260709 13:06:21.934265 12945 leader_election.cc:290] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:21.933638 12872 raft_consensus.cc:2468] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 0.
I20260709 13:06:21.935436 13008 leader_election.cc:290] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:21.936157 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cca9fc63574e26a1b1aac317816794" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
I20260709 13:06:21.936332 13008 raft_consensus.cc:493] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.936404 12690 leader_election.cc:304] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:21.937518 12722 raft_consensus.cc:2468] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 0.
I20260709 13:06:21.940228 13039 raft_consensus.cc:2804] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:21.940730 13021 leader_election.cc:290] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:21.941483 12839 leader_election.cc:304] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:21.945096 13039 raft_consensus.cc:493] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.946075 13039 raft_consensus.cc:3060] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:21.955116 13008 raft_consensus.cc:515] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.959997 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee37120e5a4496bbea79d807e5ed39a" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
I20260709 13:06:21.959750 12948 raft_consensus.cc:515] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.960286 12799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cca9fc63574e26a1b1aac317816794" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:21.960834 12722 raft_consensus.cc:2468] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 0.
I20260709 13:06:21.961443 12799 raft_consensus.cc:2468] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 0.
I20260709 13:06:21.962265 12948 leader_election.cc:290] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:21.962546 12839 leader_election.cc:304] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:21.964730 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25c0c0ccabef4e72bf35ad3c1dc83f17" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:21.965775 12872 raft_consensus.cc:3060] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:21.968061 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25c0c0ccabef4e72bf35ad3c1dc83f17" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:21.968896 12722 raft_consensus.cc:3060] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:21.976909 13038 raft_consensus.cc:2804] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:21.985332 13038 raft_consensus.cc:493] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.985496 12948 raft_consensus.cc:493] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.985769 13038 raft_consensus.cc:3060] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:21.986102 12948 raft_consensus.cc:515] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:21.988281 12948 leader_election.cc:290] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:21.993742 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee37120e5a4496bbea79d807e5ed39a" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
I20260709 13:06:21.994521 12723 raft_consensus.cc:2468] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 0.
I20260709 13:06:21.996140 12871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee37120e5a4496bbea79d807e5ed39a" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:21.993930 13023 tablet_bootstrap.cc:492] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:22.001500 12871 raft_consensus.cc:2468] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 0.
I20260709 13:06:21.994344 13038 raft_consensus.cc:515] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.006846 13022 tablet_bootstrap.cc:654] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:22.007056 13038 leader_election.cc:290] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:22.008523 13038 raft_consensus.cc:2804] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:22.008914 13038 raft_consensus.cc:493] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.009297 13038 raft_consensus.cc:3060] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.002658 13023 ts_tablet_manager.cc:1403] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.115s	user 0.019s	sys 0.021s
I20260709 13:06:22.012639 13039 raft_consensus.cc:515] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.016427 12871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cca9fc63574e26a1b1aac317816794" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:22.017375 12871 raft_consensus.cc:2393] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a9fb085add94dd99d22ee2b1129932c in current term 1: Already voted for candidate 4a2c916532db4ec1b36070c705d7e07f in this term.
I20260709 13:06:22.017733 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cca9fc63574e26a1b1aac317816794" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:22.018523 12798 raft_consensus.cc:3060] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:22.010629 12839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:42057, user_credentials={real_user=slave}} blocked reactor thread for 48174.1us
I20260709 13:06:22.020002 13008 leader_election.cc:290] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:22.005148 12722 raft_consensus.cc:2468] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 1.
I20260709 13:06:22.020910 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cca9fc63574e26a1b1aac317816794" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:22.002465 12872 raft_consensus.cc:2468] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 1.
I20260709 13:06:22.021782 12723 raft_consensus.cc:2393] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4a2c916532db4ec1b36070c705d7e07f in current term 1: Already voted for candidate 3a9fb085add94dd99d22ee2b1129932c in this term.
I20260709 13:06:22.022730 13039 leader_election.cc:290] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Requested vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:22.011150 12765 leader_election.cc:304] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:22.023924 13040 raft_consensus.cc:2804] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:22.024573 13040 raft_consensus.cc:493] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.024806 13024 tablet_bootstrap.cc:492] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:22.024959 13040 raft_consensus.cc:3060] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.025416 13024 ts_tablet_manager.cc:1403] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.140s	user 0.028s	sys 0.080s
I20260709 13:06:22.029083 13024 raft_consensus.cc:359] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.029935 13024 raft_consensus.cc:385] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.030319 13024 raft_consensus.cc:740] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.031517 13023 raft_consensus.cc:359] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.031466 13024 consensus_queue.cc:260] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.033664 13023 raft_consensus.cc:385] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.033733 12799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cca9fc63574e26a1b1aac317816794" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:22.037482 13023 raft_consensus.cc:740] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.035849 12797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee37120e5a4496bbea79d807e5ed39a" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:22.035936 13024 ts_tablet_manager.cc:1434] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.005s
I20260709 13:06:22.040022 13024 tablet_bootstrap.cc:492] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:22.040146 13023 consensus_queue.cc:260] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.040412 13038 raft_consensus.cc:515] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.037169 12765 leader_election.cc:304] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:22.043284 13023 ts_tablet_manager.cc:1434] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.001s
I20260709 13:06:22.044380 12948 raft_consensus.cc:2804] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:22.044996 12948 raft_consensus.cc:697] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Becoming Leader. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:22.045837 12948 consensus_queue.cc:237] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.046952 13022 tablet_bootstrap.cc:492] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:22.047534 13022 ts_tablet_manager.cc:1403] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.124s	user 0.014s	sys 0.015s
I20260709 13:06:22.048729 13040 raft_consensus.cc:515] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.050295 12797 raft_consensus.cc:2393] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4a2c916532db4ec1b36070c705d7e07f in current term 1: Already voted for candidate fc1fcc33b1c04369ab276792291f3331 in this term.
I20260709 13:06:22.053572 12798 raft_consensus.cc:2468] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 1.
I20260709 13:06:22.055617 12837 leader_election.cc:304] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f; no voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331
I20260709 13:06:22.056922 13040 leader_election.cc:290] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:22.055090 12688 leader_election.cc:304] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 4a2c916532db4ec1b36070c705d7e07f
I20260709 13:06:22.058535 13039 raft_consensus.cc:2804] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:22.059038 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee37120e5a4496bbea79d807e5ed39a" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:22.059274 13039 raft_consensus.cc:697] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Becoming Leader. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:22.060048 13024 tablet_bootstrap.cc:654] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:22.060191 13039 consensus_queue.cc:237] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.063773 13023 tablet_bootstrap.cc:492] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:22.064433 12872 raft_consensus.cc:2393] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc1fcc33b1c04369ab276792291f3331 in current term 1: Already voted for candidate 4a2c916532db4ec1b36070c705d7e07f in this term.
I20260709 13:06:22.065474 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee37120e5a4496bbea79d807e5ed39a" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:22.066190 12723 raft_consensus.cc:3060] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.067209 13038 leader_election.cc:290] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:22.067838 13038 raft_consensus.cc:2749] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:22.070382 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee37120e5a4496bbea79d807e5ed39a" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:22.071381 12798 raft_consensus.cc:2393] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4a2c916532db4ec1b36070c705d7e07f in current term 1: Already voted for candidate fc1fcc33b1c04369ab276792291f3331 in this term.
I20260709 13:06:22.050721 13022 raft_consensus.cc:359] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.072136 13022 raft_consensus.cc:385] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.072708 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee37120e5a4496bbea79d807e5ed39a" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:22.072772 13022 raft_consensus.cc:740] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.078220 12723 raft_consensus.cc:2468] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 1.
I20260709 13:06:22.086491 12765 leader_election.cc:304] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 4a2c916532db4ec1b36070c705d7e07f
I20260709 13:06:22.088137 12837 leader_election.cc:304] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f; no voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331
I20260709 13:06:22.088912 13022 consensus_queue.cc:260] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.090214 13041 raft_consensus.cc:2749] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:22.091639 13022 ts_tablet_manager.cc:1434] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.000s
I20260709 13:06:22.092787 13022 tablet_bootstrap.cc:492] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:22.093559 12948 raft_consensus.cc:2804] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:22.094449 12948 raft_consensus.cc:697] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Becoming Leader. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:22.095506 12948 consensus_queue.cc:237] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.096733 12619 catalog_manager.cc:5697] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc1fcc33b1c04369ab276792291f3331 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "fc1fcc33b1c04369ab276792291f3331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.106792 13023 tablet_bootstrap.cc:654] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:22.108072 12617 catalog_manager.cc:5697] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3a9fb085add94dd99d22ee2b1129932c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.133605 13024 tablet_bootstrap.cc:492] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:22.134721 13024 ts_tablet_manager.cc:1403] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.095s	user 0.023s	sys 0.006s
I20260709 13:06:22.140164 13024 raft_consensus.cc:359] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.141350 13024 raft_consensus.cc:385] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.141810 13024 raft_consensus.cc:740] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.143081 13024 consensus_queue.cc:260] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.145931 13022 tablet_bootstrap.cc:654] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:22.152758 13023 tablet_bootstrap.cc:492] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:22.153347 13023 ts_tablet_manager.cc:1403] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.090s	user 0.007s	sys 0.044s
I20260709 13:06:22.153679 13024 ts_tablet_manager.cc:1434] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.002s
I20260709 13:06:22.155010 13024 tablet_bootstrap.cc:492] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:22.157023 13023 raft_consensus.cc:359] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.157924 13023 raft_consensus.cc:385] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.158247 13023 raft_consensus.cc:740] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.159180 13023 consensus_queue.cc:260] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.162173 13023 ts_tablet_manager.cc:1434] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:06:22.162585 13024 tablet_bootstrap.cc:654] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:22.163416 13023 tablet_bootstrap.cc:492] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:22.174683 13023 tablet_bootstrap.cc:654] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:22.178037 13024 tablet_bootstrap.cc:492] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:22.178736 13024 ts_tablet_manager.cc:1403] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.001s
I20260709 13:06:22.179116 13022 tablet_bootstrap.cc:492] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:22.179291 13041 raft_consensus.cc:493] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.179728 13022 ts_tablet_manager.cc:1403] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.087s	user 0.015s	sys 0.005s
I20260709 13:06:22.179840 13041 raft_consensus.cc:515] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.183418 13022 raft_consensus.cc:359] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.183075 13024 raft_consensus.cc:359] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.184579 13024 raft_consensus.cc:385] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.185189 12797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271e42be0a714018a752ea327df357ef" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:22.185583 13041 leader_election.cc:290] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:22.186257 12797 raft_consensus.cc:2468] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 0.
I20260709 13:06:22.185070 13024 raft_consensus.cc:740] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.188306 12837 leader_election.cc:304] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:22.185071 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271e42be0a714018a752ea327df357ef" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
I20260709 13:06:22.184417 13022 raft_consensus.cc:385] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.189695 13022 raft_consensus.cc:740] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.189884 13041 raft_consensus.cc:2804] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:22.190608 13041 raft_consensus.cc:493] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.191075 13041 raft_consensus.cc:3060] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.190622 13022 consensus_queue.cc:260] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.193058 12723 raft_consensus.cc:2468] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 0.
I20260709 13:06:22.199203 13022 ts_tablet_manager.cc:1434] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.001s
I20260709 13:06:22.189942 13024 consensus_queue.cc:260] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.201987 13024 ts_tablet_manager.cc:1434] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:06:22.203260 13024 tablet_bootstrap.cc:492] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:22.206753 13022 tablet_bootstrap.cc:492] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:22.211545 13041 raft_consensus.cc:515] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.215178 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271e42be0a714018a752ea327df357ef" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:22.215986 12722 raft_consensus.cc:3060] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.215488 13022 tablet_bootstrap.cc:654] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:22.226756 12722 raft_consensus.cc:2468] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 1.
I20260709 13:06:22.226413 12620 catalog_manager.cc:5697] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc1fcc33b1c04369ab276792291f3331 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "fc1fcc33b1c04369ab276792291f3331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.228489 12839 leader_election.cc:304] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:22.229605 13038 raft_consensus.cc:2804] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:22.230187 13038 raft_consensus.cc:697] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Becoming Leader. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Running, Role: LEADER
I20260709 13:06:22.234732 13023 tablet_bootstrap.cc:492] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:22.235539 13023 ts_tablet_manager.cc:1403] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.072s	user 0.033s	sys 0.021s
I20260709 13:06:22.238039 13038 consensus_queue.cc:237] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.244585 12797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271e42be0a714018a752ea327df357ef" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:22.245363 12797 raft_consensus.cc:3060] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.246707 13023 raft_consensus.cc:359] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.247534 13023 raft_consensus.cc:385] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.247798 13023 raft_consensus.cc:740] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.248561 13023 consensus_queue.cc:260] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.250519 13023 ts_tablet_manager.cc:1434] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.000s
I20260709 13:06:22.251619 13023 tablet_bootstrap.cc:492] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331: Bootstrap starting.
I20260709 13:06:22.251612 13041 leader_election.cc:290] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:22.256031 13024 tablet_bootstrap.cc:654] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:22.259243 13023 tablet_bootstrap.cc:654] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:22.264770 13022 tablet_bootstrap.cc:492] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:22.265370 13022 ts_tablet_manager.cc:1403] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.059s	user 0.010s	sys 0.004s
I20260709 13:06:22.268697 13022 raft_consensus.cc:359] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.269472 13022 raft_consensus.cc:385] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.269773 13022 raft_consensus.cc:740] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.272506 13038 raft_consensus.cc:493] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.273064 13038 raft_consensus.cc:515] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.263681 12619 catalog_manager.cc:5697] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195). New cstate: current_term: 1 leader_uuid: "4a2c916532db4ec1b36070c705d7e07f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:22.276616 13041 raft_consensus.cc:493] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.277436 13041 raft_consensus.cc:515] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.278666 13022 consensus_queue.cc:260] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.281255 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3111979238fe417aa43ba481ab84bfa0" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
W20260709 13:06:22.282790 12839 leader_election.cc:343] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:22.284214 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3111979238fe417aa43ba481ab84bfa0" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:22.284940 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8751836b2c4b6a8102c3dfa18d8c6b" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
I20260709 13:06:22.280539 13041 leader_election.cc:290] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:22.285806 12722 raft_consensus.cc:2468] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 0.
I20260709 13:06:22.287580 12839 leader_election.cc:304] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:22.288908 13038 leader_election.cc:290] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
W20260709 13:06:22.289140 12670 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:22.289664 13041 raft_consensus.cc:2804] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:22.293156 13041 raft_consensus.cc:493] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.293705 13041 raft_consensus.cc:3060] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.296043 13022 ts_tablet_manager.cc:1434] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.030s	user 0.010s	sys 0.001s
I20260709 13:06:22.290970 12799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8751836b2c4b6a8102c3dfa18d8c6b" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:22.301074 12799 raft_consensus.cc:2468] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 0.
I20260709 13:06:22.301964 13022 tablet_bootstrap.cc:492] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:22.285045 12798 raft_consensus.cc:2468] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 0.
I20260709 13:06:22.315042 12837 leader_election.cc:304] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 3a9fb085add94dd99d22ee2b1129932c
I20260709 13:06:22.315573 12797 raft_consensus.cc:2468] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 1.
I20260709 13:06:22.319314 13023 tablet_bootstrap.cc:492] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331: No bootstrap required, opened a new log
I20260709 13:06:22.319878 13023 ts_tablet_manager.cc:1403] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331: Time spent bootstrapping tablet: real 0.069s	user 0.018s	sys 0.048s
I20260709 13:06:22.320616 13024 tablet_bootstrap.cc:492] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:22.321506 13024 ts_tablet_manager.cc:1403] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.119s	user 0.013s	sys 0.016s
I20260709 13:06:22.322475 13038 raft_consensus.cc:2804] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:22.323030 13038 raft_consensus.cc:493] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.323514 13023 raft_consensus.cc:359] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.324326 13038 raft_consensus.cc:3060] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.324733 13023 raft_consensus.cc:385] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.325158 13023 raft_consensus.cc:740] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.326063 13023 consensus_queue.cc:260] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.327184 13022 tablet_bootstrap.cc:654] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:22.331606 13024 raft_consensus.cc:359] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.332644 13024 raft_consensus.cc:385] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.332981 13024 raft_consensus.cc:740] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.333990 13024 consensus_queue.cc:260] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.334115 13038 raft_consensus.cc:515] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.336524 13024 ts_tablet_manager.cc:1434] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:06:22.338153 13023 ts_tablet_manager.cc:1434] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:06:22.339295 13024 tablet_bootstrap.cc:492] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:22.328716 13041 raft_consensus.cc:515] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.344115 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8751836b2c4b6a8102c3dfa18d8c6b" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:22.344775 12722 raft_consensus.cc:3060] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.346109 12797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8751836b2c4b6a8102c3dfa18d8c6b" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:22.346762 13024 tablet_bootstrap.cc:654] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:22.346814 12797 raft_consensus.cc:3060] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.349438 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3111979238fe417aa43ba481ab84bfa0" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
W20260709 13:06:22.351262 12839 leader_election.cc:343] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:22.347680 13041 leader_election.cc:290] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:22.352394 12722 raft_consensus.cc:2468] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 1.
I20260709 13:06:22.353847 12839 leader_election.cc:304] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:22.354465 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3111979238fe417aa43ba481ab84bfa0" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:22.354827 12797 raft_consensus.cc:2468] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 1.
I20260709 13:06:22.355183 12798 raft_consensus.cc:3060] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.358799 13022 tablet_bootstrap.cc:492] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:22.359556 13022 ts_tablet_manager.cc:1403] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.006s
I20260709 13:06:22.359692 13024 tablet_bootstrap.cc:492] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:22.360271 13024 ts_tablet_manager.cc:1403] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.004s
I20260709 13:06:22.363240 13022 raft_consensus.cc:359] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.364151 12798 raft_consensus.cc:2468] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 1.
I20260709 13:06:22.365582 12837 leader_election.cc:304] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 3a9fb085add94dd99d22ee2b1129932c
I20260709 13:06:22.365676 13024 raft_consensus.cc:359] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.366462 13038 leader_election.cc:290] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 1 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:22.366485 13024 raft_consensus.cc:385] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.367282 13024 raft_consensus.cc:740] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.367453 13038 raft_consensus.cc:2804] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:22.368047 13038 raft_consensus.cc:697] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Becoming Leader. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Running, Role: LEADER
I20260709 13:06:22.368996 13038 consensus_queue.cc:237] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.368237 13024 consensus_queue.cc:260] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.366529 13008 raft_consensus.cc:2804] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:22.370697 13008 raft_consensus.cc:697] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Becoming Leader. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Running, Role: LEADER
I20260709 13:06:22.364233 13022 raft_consensus.cc:385] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.371392 13022 raft_consensus.cc:740] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.371598 13008 consensus_queue.cc:237] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.372422 13022 consensus_queue.cc:260] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.375522 13022 ts_tablet_manager.cc:1434] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:06:22.376609 13022 tablet_bootstrap.cc:492] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:22.380972 12948 raft_consensus.cc:493] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.381677 12948 raft_consensus.cc:515] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.384573 13022 tablet_bootstrap.cc:654] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:22.386121 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4318a4373b54622a1aa813bc53cfb44" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c" is_pre_election: true
I20260709 13:06:22.386627 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4318a4373b54622a1aa813bc53cfb44" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:22.384290 12948 leader_election.cc:290] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:22.387091 12722 raft_consensus.cc:2468] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 0.
I20260709 13:06:22.388103 12872 raft_consensus.cc:2468] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 0.
I20260709 13:06:22.388856 13024 ts_tablet_manager.cc:1434] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:06:22.389542 12765 leader_election.cc:304] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:22.391480 12948 raft_consensus.cc:2804] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:22.391964 12948 raft_consensus.cc:493] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.392434 12948 raft_consensus.cc:3060] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.395706 13046 raft_consensus.cc:493] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.396422 13046 raft_consensus.cc:515] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.390116 13024 tablet_bootstrap.cc:492] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f: Bootstrap starting.
I20260709 13:06:22.400455 13046 leader_election.cc:290] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:22.401449 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121cd579cbe1405a81ad6f0a8693538b" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:22.401635 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121cd579cbe1405a81ad6f0a8693538b" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:22.402455 12798 raft_consensus.cc:2468] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 0.
W20260709 13:06:22.403425 12690 leader_election.cc:343] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:22.404278 12688 leader_election.cc:304] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 4a2c916532db4ec1b36070c705d7e07f
I20260709 13:06:22.405799 13046 raft_consensus.cc:2804] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:22.406262 13046 raft_consensus.cc:493] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.406667 13046 raft_consensus.cc:3060] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.406783 13024 tablet_bootstrap.cc:654] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:22.411711 12948 raft_consensus.cc:515] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.415083 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4318a4373b54622a1aa813bc53cfb44" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:22.415802 12722 raft_consensus.cc:3060] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.415580 13046 raft_consensus.cc:515] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.416247 12620 catalog_manager.cc:5697] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195). New cstate: current_term: 1 leader_uuid: "4a2c916532db4ec1b36070c705d7e07f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:22.418599 12620 catalog_manager.cc:5697] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195). New cstate: current_term: 1 leader_uuid: "4a2c916532db4ec1b36070c705d7e07f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:22.419523 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121cd579cbe1405a81ad6f0a8693538b" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:22.420362 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121cd579cbe1405a81ad6f0a8693538b" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:22.420965 13024 tablet_bootstrap.cc:492] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f: No bootstrap required, opened a new log
I20260709 13:06:22.421139 12798 raft_consensus.cc:3060] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:22.421326 12690 leader_election.cc:343] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:22.421537 13024 ts_tablet_manager.cc:1403] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.009s
I20260709 13:06:22.425138 13024 raft_consensus.cc:359] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.426020 13024 raft_consensus.cc:385] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.426357 13024 raft_consensus.cc:740] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.427292 13024 consensus_queue.cc:260] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.429555 13024 ts_tablet_manager.cc:1434] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:06:22.431083 13046 leader_election.cc:290] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Requested vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:22.432312 12722 raft_consensus.cc:2468] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 1.
I20260709 13:06:22.434988 12765 leader_election.cc:304] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:22.436025 13021 raft_consensus.cc:2804] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:22.436542 13021 raft_consensus.cc:697] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Becoming Leader. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:22.437485 13021 consensus_queue.cc:237] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.438365 12948 leader_election.cc:290] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 1 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:22.442706 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4318a4373b54622a1aa813bc53cfb44" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:22.443168 12798 raft_consensus.cc:2468] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 1.
I20260709 13:06:22.443408 12872 raft_consensus.cc:3060] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.445890 13022 tablet_bootstrap.cc:492] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:22.446110 12688 leader_election.cc:304] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 4a2c916532db4ec1b36070c705d7e07f
I20260709 13:06:22.446522 13022 ts_tablet_manager.cc:1403] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.070s	user 0.014s	sys 0.002s
I20260709 13:06:22.447003 13046 raft_consensus.cc:2804] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:22.447544 13046 raft_consensus.cc:697] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Becoming Leader. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:22.448460 13046 consensus_queue.cc:237] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.452360 12872 raft_consensus.cc:2468] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 1.
I20260709 13:06:22.450315 13022 raft_consensus.cc:359] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.453552 13022 raft_consensus.cc:385] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.453979 13022 raft_consensus.cc:740] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.456274 13022 consensus_queue.cc:260] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.459368 13022 ts_tablet_manager.cc:1434] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 13:06:22.460754 13022 tablet_bootstrap.cc:492] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c: Bootstrap starting.
I20260709 13:06:22.467720 13022 tablet_bootstrap.cc:654] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:22.470444 12617 catalog_manager.cc:5697] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3a9fb085add94dd99d22ee2b1129932c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.474522 12620 catalog_manager.cc:5697] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc1fcc33b1c04369ab276792291f3331 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "fc1fcc33b1c04369ab276792291f3331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.479113 13021 consensus_queue.cc:1048] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:22.489383 13022 tablet_bootstrap.cc:492] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c: No bootstrap required, opened a new log
I20260709 13:06:22.490105 13022 ts_tablet_manager.cc:1403] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.005s
I20260709 13:06:22.493134 13022 raft_consensus.cc:359] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.493888 13022 raft_consensus.cc:385] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:22.494170 13022 raft_consensus.cc:740] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Initialized, Role: FOLLOWER
I20260709 13:06:22.494871 13022 consensus_queue.cc:260] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.496321 13046 consensus_queue.cc:1048] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:22.498148 13039 raft_consensus.cc:493] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.498759 13039 raft_consensus.cc:515] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.499089 13021 consensus_queue.cc:1048] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:22.499614 13022 ts_tablet_manager.cc:1434] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:06:22.504521 13039 leader_election.cc:290] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:22.504611 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e25d10382c5343c78b863083e4b4aa31" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:22.505443 12798 raft_consensus.cc:2468] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 0.
I20260709 13:06:22.506852 12688 leader_election.cc:304] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:22.507220 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e25d10382c5343c78b863083e4b4aa31" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:22.508013 12872 raft_consensus.cc:2468] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 0.
I20260709 13:06:22.508162 13046 raft_consensus.cc:2804] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:22.508751 13046 raft_consensus.cc:493] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.509181 13046 raft_consensus.cc:3060] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.520465 13046 raft_consensus.cc:515] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.529357 13046 leader_election.cc:290] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Requested vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:22.541872 12870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e25d10382c5343c78b863083e4b4aa31" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:22.542716 12870 raft_consensus.cc:3060] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.548241 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e25d10382c5343c78b863083e4b4aa31" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:22.549031 12798 raft_consensus.cc:3060] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.564332 12798 raft_consensus.cc:2468] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 1.
I20260709 13:06:22.567878 12870 raft_consensus.cc:2468] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 1.
I20260709 13:06:22.570289 13076 consensus_queue.cc:1048] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:22.573799 12690 leader_election.cc:304] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:22.574374 13077 consensus_queue.cc:1048] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:22.575695 13075 raft_consensus.cc:2804] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:22.576292 13075 raft_consensus.cc:697] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Becoming Leader. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:22.577282 13075 consensus_queue.cc:237] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.597419 12617 catalog_manager.cc:5697] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3a9fb085add94dd99d22ee2b1129932c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.619223 13075 raft_consensus.cc:493] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.619910 13075 raft_consensus.cc:515] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.621815 13040 consensus_queue.cc:1048] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:22.622628 13075 leader_election.cc:290] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:22.623821 12870 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3111979238fe417aa43ba481ab84bfa0" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2c916532db4ec1b36070c705d7e07f" is_pre_election: true
I20260709 13:06:22.625100 12797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3111979238fe417aa43ba481ab84bfa0" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fcc33b1c04369ab276792291f3331" is_pre_election: true
I20260709 13:06:22.626039 12797 raft_consensus.cc:2393] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a9fb085add94dd99d22ee2b1129932c in current term 1: Already voted for candidate 4a2c916532db4ec1b36070c705d7e07f in this term.
I20260709 13:06:22.642315 12688 leader_election.cc:304] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c; no voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331
I20260709 13:06:22.643391 13075 raft_consensus.cc:3060] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.653134 13075 raft_consensus.cc:2749] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:22.779102 13038 consensus_queue.cc:1048] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:22.791116 13041 consensus_queue.cc:1048] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:22.817699 12798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ee37120e5a4496bbea79d807e5ed39a"
dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
mode: GRACEFUL
new_leader_uuid: "3a9fb085add94dd99d22ee2b1129932c"
 from {username='slave'} at 127.0.0.1:39408
I20260709 13:06:22.818434 12798 raft_consensus.cc:606] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Received request to transfer leadership to TS 3a9fb085add94dd99d22ee2b1129932c
I20260709 13:06:22.824447 12798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25c0c0ccabef4e72bf35ad3c1dc83f17"
dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
mode: GRACEFUL
new_leader_uuid: "3a9fb085add94dd99d22ee2b1129932c"
 from {username='slave'} at 127.0.0.1:39408
I20260709 13:06:22.825098 12798 raft_consensus.cc:606] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Received request to transfer leadership to TS 3a9fb085add94dd99d22ee2b1129932c
I20260709 13:06:22.832535 12798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4318a4373b54622a1aa813bc53cfb44"
dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
mode: GRACEFUL
new_leader_uuid: "3a9fb085add94dd99d22ee2b1129932c"
 from {username='slave'} at 127.0.0.1:39408
I20260709 13:06:22.832791 13076 consensus_queue.cc:1048] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:22.833215 12798 raft_consensus.cc:606] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Received request to transfer leadership to TS 3a9fb085add94dd99d22ee2b1129932c
I20260709 13:06:22.834021 13041 consensus_queue.cc:1048] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:22.842391 12872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a8751836b2c4b6a8102c3dfa18d8c6b"
dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
mode: GRACEFUL
new_leader_uuid: "fc1fcc33b1c04369ab276792291f3331"
 from {username='slave'} at 127.0.0.1:38446
I20260709 13:06:22.843073 12872 raft_consensus.cc:606] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Received request to transfer leadership to TS fc1fcc33b1c04369ab276792291f3331
I20260709 13:06:22.850660 12872 raft_consensus.cc:3060] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:22.859212 13041 consensus_queue.cc:1048] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:22.871331 12870 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "271e42be0a714018a752ea327df357ef"
dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
mode: GRACEFUL
new_leader_uuid: "fc1fcc33b1c04369ab276792291f3331"
 from {username='slave'} at 127.0.0.1:38446
I20260709 13:06:22.872042 12870 raft_consensus.cc:606] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Received request to transfer leadership to TS fc1fcc33b1c04369ab276792291f3331
I20260709 13:06:22.876688 13076 consensus_queue.cc:1048] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:22.883323 12870 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3111979238fe417aa43ba481ab84bfa0"
dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
mode: GRACEFUL
new_leader_uuid: "fc1fcc33b1c04369ab276792291f3331"
 from {username='slave'} at 127.0.0.1:38446
I20260709 13:06:22.884050 12870 raft_consensus.cc:606] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Received request to transfer leadership to TS fc1fcc33b1c04369ab276792291f3331
I20260709 13:06:22.893751 13038 raft_consensus.cc:993] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f: : Instructing follower fc1fcc33b1c04369ab276792291f3331 to start an election
I20260709 13:06:22.894268 13038 raft_consensus.cc:1081] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Signalling peer fc1fcc33b1c04369ab276792291f3331 to start an election
I20260709 13:06:22.898773 12723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e25d10382c5343c78b863083e4b4aa31"
dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
mode: GRACEFUL
new_leader_uuid: "fc1fcc33b1c04369ab276792291f3331"
 from {username='slave'} at 127.0.0.1:37708
I20260709 13:06:22.899518 12723 raft_consensus.cc:606] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Received request to transfer leadership to TS fc1fcc33b1c04369ab276792291f3331
I20260709 13:06:22.900796 13021 consensus_queue.cc:1048] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:22.903800 12798 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3111979238fe417aa43ba481ab84bfa0"
dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
 from {username='slave'} at 127.0.0.1:39402
I20260709 13:06:22.904417 12798 raft_consensus.cc:493] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:22.904822 12798 raft_consensus.cc:3060] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:22.913729 13038 consensus_queue.cc:1048] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:22.913846 12723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "121cd579cbe1405a81ad6f0a8693538b"
dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
mode: GRACEFUL
new_leader_uuid: "4a2c916532db4ec1b36070c705d7e07f"
 from {username='slave'} at 127.0.0.1:37708
I20260709 13:06:22.916447 12723 raft_consensus.cc:606] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Received request to transfer leadership to TS 4a2c916532db4ec1b36070c705d7e07f
I20260709 13:06:22.920089 12798 raft_consensus.cc:515] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.924782 12797 raft_consensus.cc:1240] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Rejecting Update request from peer 4a2c916532db4ec1b36070c705d7e07f for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:22.928889 13038 consensus_queue.cc:1059] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:22.930261 13041 raft_consensus.cc:3055] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:22.930868 13041 raft_consensus.cc:740] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Running, Role: LEADER
I20260709 13:06:22.932128 13041 consensus_queue.cc:260] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.937491 13076 raft_consensus.cc:993] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c: : Instructing follower 4a2c916532db4ec1b36070c705d7e07f to start an election
I20260709 13:06:22.938340 13076 raft_consensus.cc:1081] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Signalling peer 4a2c916532db4ec1b36070c705d7e07f to start an election
I20260709 13:06:22.941423 12870 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "121cd579cbe1405a81ad6f0a8693538b"
dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
 from {username='slave'} at 127.0.0.1:38416
I20260709 13:06:22.946753 13041 raft_consensus.cc:3060] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:22.957651 12798 leader_election.cc:290] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 2 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:22.942304 12870 raft_consensus.cc:493] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:22.961609 12870 raft_consensus.cc:3060] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:22.965548 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3111979238fe417aa43ba481ab84bfa0" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:22.966388 12722 raft_consensus.cc:3060] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:22.969738 12870 raft_consensus.cc:515] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.972491 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3111979238fe417aa43ba481ab84bfa0" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:22.979488 12870 leader_election.cc:290] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 2 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:22.980494 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121cd579cbe1405a81ad6f0a8693538b" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:22.981206 12723 raft_consensus.cc:3055] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:22.981599 12723 raft_consensus.cc:740] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:22.982535 12723 consensus_queue.cc:260] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:22.983822 12723 raft_consensus.cc:3060] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:22.983984 13046 consensus_queue.cc:1048] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:22.992076 12723 raft_consensus.cc:2468] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 2.
I20260709 13:06:22.993517 12839 leader_election.cc:304] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:22.994750 12797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "121cd579cbe1405a81ad6f0a8693538b" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:22.995553 12797 raft_consensus.cc:3060] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:22.998664 13041 raft_consensus.cc:2804] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:23.001276 13041 raft_consensus.cc:697] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [term 2 LEADER]: Becoming Leader. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Running, Role: LEADER
I20260709 13:06:23.002254 13041 consensus_queue.cc:237] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.005488 13078 consensus_queue.cc:1048] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:23.007146 12797 raft_consensus.cc:2468] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 2.
I20260709 13:06:23.011408 12722 raft_consensus.cc:2468] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 2.
I20260709 13:06:23.014804 13038 consensus_queue.cc:1048] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:23.034205 13076 consensus_queue.cc:1048] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:23.038338 12765 leader_election.cc:304] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:23.015499 12872 raft_consensus.cc:2468] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 2.
I20260709 13:06:23.038166 12620 catalog_manager.cc:5697] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f reported cstate change: term changed from 1 to 2, leader changed from 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193) to 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195). New cstate: current_term: 2 leader_uuid: "4a2c916532db4ec1b36070c705d7e07f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:23.043516 13078 raft_consensus.cc:2804] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:23.045199 13078 raft_consensus.cc:697] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Becoming Leader. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:23.046073 13078 consensus_queue.cc:237] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.054582 13021 raft_consensus.cc:993] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331: : Instructing follower 3a9fb085add94dd99d22ee2b1129932c to start an election
I20260709 13:06:23.055097 13021 raft_consensus.cc:1081] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Signalling peer 3a9fb085add94dd99d22ee2b1129932c to start an election
I20260709 13:06:23.058295 12723 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4318a4373b54622a1aa813bc53cfb44"
dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
 from {username='slave'} at 127.0.0.1:37706
I20260709 13:06:23.058991 12723 raft_consensus.cc:493] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:23.059408 12723 raft_consensus.cc:3060] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.066210 13076 raft_consensus.cc:993] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c: : Instructing follower fc1fcc33b1c04369ab276792291f3331 to start an election
I20260709 13:06:23.066847 13076 raft_consensus.cc:1081] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Signalling peer fc1fcc33b1c04369ab276792291f3331 to start an election
I20260709 13:06:23.068773 12798 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e25d10382c5343c78b863083e4b4aa31"
dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
 from {username='slave'} at 127.0.0.1:39388
I20260709 13:06:23.069406 12798 raft_consensus.cc:493] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:23.069746 12798 raft_consensus.cc:3060] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.068555 12723 raft_consensus.cc:515] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.073707 12797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4318a4373b54622a1aa813bc53cfb44" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:23.074355 12797 raft_consensus.cc:3055] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:23.074715 12797 raft_consensus.cc:740] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:23.075677 12797 consensus_queue.cc:260] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.076779 12797 raft_consensus.cc:3060] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.074371 12723 leader_election.cc:290] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 2 election: Requested vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:23.081872 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4318a4373b54622a1aa813bc53cfb44" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:23.082695 12872 raft_consensus.cc:3060] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.084192 12798 raft_consensus.cc:515] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.084596 12797 raft_consensus.cc:2468] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 2.
I20260709 13:06:23.085924 12688 leader_election.cc:304] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:23.086812 13076 raft_consensus.cc:2804] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:23.087302 13076 raft_consensus.cc:697] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [term 2 LEADER]: Becoming Leader. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:23.087815 13038 raft_consensus.cc:993] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f: : Instructing follower fc1fcc33b1c04369ab276792291f3331 to start an election
I20260709 13:06:23.088333 13038 raft_consensus.cc:1081] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Signalling peer fc1fcc33b1c04369ab276792291f3331 to start an election
I20260709 13:06:23.088083 13076 consensus_queue.cc:237] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.091020 12798 leader_election.cc:290] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 2 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:23.091825 12871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e25d10382c5343c78b863083e4b4aa31" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:23.092465 12871 raft_consensus.cc:3060] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.099144 12723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e25d10382c5343c78b863083e4b4aa31" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:23.099787 12723 raft_consensus.cc:3055] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:23.100098 12723 raft_consensus.cc:740] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:23.100845 12723 consensus_queue.cc:260] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.100209 12798 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a8751836b2c4b6a8102c3dfa18d8c6b"
dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
 from {username='slave'} at 127.0.0.1:39402
I20260709 13:06:23.102102 12723 raft_consensus.cc:3060] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.104111 12871 raft_consensus.cc:2468] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 2.
I20260709 13:06:23.105322 12765 leader_election.cc:304] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:23.102116 12798 raft_consensus.cc:493] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:23.106345 12798 raft_consensus.cc:3060] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.106405 13021 raft_consensus.cc:2804] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:23.110276 13021 raft_consensus.cc:697] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Becoming Leader. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:23.110284 12723 raft_consensus.cc:2468] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 2.
I20260709 13:06:23.097961 12617 catalog_manager.cc:5697] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 reported cstate change: term changed from 1 to 2, leader changed from 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195) to fc1fcc33b1c04369ab276792291f3331 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "fc1fcc33b1c04369ab276792291f3331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:23.111330 13021 consensus_queue.cc:237] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.109424 12872 raft_consensus.cc:2468] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 2.
I20260709 13:06:23.108143 12620 catalog_manager.cc:5697] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c reported cstate change: term changed from 1 to 2, leader changed from fc1fcc33b1c04369ab276792291f3331 (127.0.107.194) to 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3a9fb085add94dd99d22ee2b1129932c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:23.118475 12798 raft_consensus.cc:515] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.119143 13078 raft_consensus.cc:993] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331: : Instructing follower 3a9fb085add94dd99d22ee2b1129932c to start an election
I20260709 13:06:23.119727 13078 raft_consensus.cc:1081] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Signalling peer 3a9fb085add94dd99d22ee2b1129932c to start an election
I20260709 13:06:23.120827 12798 leader_election.cc:290] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 2 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:23.123633 12724 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ee37120e5a4496bbea79d807e5ed39a"
dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
 from {username='slave'} at 127.0.0.1:37706
I20260709 13:06:23.124264 12724 raft_consensus.cc:493] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:23.124684 12724 raft_consensus.cc:3060] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.124763 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8751836b2c4b6a8102c3dfa18d8c6b" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:23.125495 12722 raft_consensus.cc:3060] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.128342 12872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8751836b2c4b6a8102c3dfa18d8c6b" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:23.129045 12872 raft_consensus.cc:3055] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:23.129359 12872 raft_consensus.cc:740] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Running, Role: LEADER
I20260709 13:06:23.136734 12724 raft_consensus.cc:515] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.135650 12722 raft_consensus.cc:2468] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 2.
I20260709 13:06:23.138013 12872 consensus_queue.cc:260] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.140687 12872 raft_consensus.cc:3060] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.142699 12871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee37120e5a4496bbea79d807e5ed39a" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:23.143450 12871 raft_consensus.cc:3060] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.148281 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee37120e5a4496bbea79d807e5ed39a" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:23.147675 12617 catalog_manager.cc:5697] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 reported cstate change: term changed from 1 to 2, leader changed from 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193) to fc1fcc33b1c04369ab276792291f3331 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "fc1fcc33b1c04369ab276792291f3331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:23.149142 12765 leader_election.cc:304] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:23.150063 13021 raft_consensus.cc:2804] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:23.150569 13021 raft_consensus.cc:697] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Becoming Leader. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:23.147578 12724 leader_election.cc:290] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 2 election: Requested vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:23.151467 13021 consensus_queue.cc:237] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.153225 12798 raft_consensus.cc:3055] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:23.153561 12798 raft_consensus.cc:740] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:23.154234 12798 consensus_queue.cc:260] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.155532 12798 raft_consensus.cc:3060] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.162555 12798 raft_consensus.cc:2468] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 2.
I20260709 13:06:23.163671 12872 raft_consensus.cc:2468] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 2.
I20260709 13:06:23.163897 12688 leader_election.cc:304] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:23.165344 13046 raft_consensus.cc:2804] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:23.165866 13046 raft_consensus.cc:697] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [term 2 LEADER]: Becoming Leader. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:23.166777 13046 consensus_queue.cc:237] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.172101 12620 catalog_manager.cc:5697] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 reported cstate change: term changed from 1 to 2, leader changed from 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195) to fc1fcc33b1c04369ab276792291f3331 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "fc1fcc33b1c04369ab276792291f3331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:23.174815 12871 raft_consensus.cc:2468] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 2.
I20260709 13:06:23.180371 13021 raft_consensus.cc:993] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331: : Instructing follower 3a9fb085add94dd99d22ee2b1129932c to start an election
I20260709 13:06:23.180811 13021 raft_consensus.cc:1081] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Signalling peer 3a9fb085add94dd99d22ee2b1129932c to start an election
I20260709 13:06:23.180512 12617 catalog_manager.cc:5697] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c reported cstate change: term changed from 1 to 2, leader changed from fc1fcc33b1c04369ab276792291f3331 (127.0.107.194) to 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3a9fb085add94dd99d22ee2b1129932c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:23.185165 12722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25c0c0ccabef4e72bf35ad3c1dc83f17"
dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
 from {username='slave'} at 127.0.0.1:37706
I20260709 13:06:23.185729 12722 raft_consensus.cc:493] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:23.186091 12722 raft_consensus.cc:3060] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.193996 12722 raft_consensus.cc:515] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.196746 12722 leader_election.cc:290] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 2 election: Requested vote from peers fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:23.197113 12798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25c0c0ccabef4e72bf35ad3c1dc83f17" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:23.197337 12871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25c0c0ccabef4e72bf35ad3c1dc83f17" candidate_uuid: "3a9fb085add94dd99d22ee2b1129932c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:23.197880 12798 raft_consensus.cc:3055] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:23.198029 12871 raft_consensus.cc:3060] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.198400 12798 raft_consensus.cc:740] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:23.199345 12798 consensus_queue.cc:260] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.200703 12798 raft_consensus.cc:3060] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.203861 12871 raft_consensus.cc:2468] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 2.
I20260709 13:06:23.205103 12690 leader_election.cc:304] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:23.205861 13046 raft_consensus.cc:2804] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:23.206329 13046 raft_consensus.cc:697] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [term 2 LEADER]: Becoming Leader. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:23.207183 13046 consensus_queue.cc:237] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.208366 12798 raft_consensus.cc:2468] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a9fb085add94dd99d22ee2b1129932c in term 2.
I20260709 13:06:23.215240 12617 catalog_manager.cc:5697] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c reported cstate change: term changed from 1 to 2, leader changed from fc1fcc33b1c04369ab276792291f3331 (127.0.107.194) to 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3a9fb085add94dd99d22ee2b1129932c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:23.278780 13008 raft_consensus.cc:993] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f: : Instructing follower fc1fcc33b1c04369ab276792291f3331 to start an election
I20260709 13:06:23.279358 13008 raft_consensus.cc:1081] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Signalling peer fc1fcc33b1c04369ab276792291f3331 to start an election
I20260709 13:06:23.281265 12798 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "271e42be0a714018a752ea327df357ef"
dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
 from {username='slave'} at 127.0.0.1:39402
I20260709 13:06:23.281813 12798 raft_consensus.cc:493] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:23.282150 12798 raft_consensus.cc:3060] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.288902 12798 raft_consensus.cc:515] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.290475 12798 leader_election.cc:290] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 2 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195:36913)
I20260709 13:06:23.291522 12722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271e42be0a714018a752ea327df357ef" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:23.291842 12871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271e42be0a714018a752ea327df357ef" candidate_uuid: "fc1fcc33b1c04369ab276792291f3331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
I20260709 13:06:23.292459 12722 raft_consensus.cc:3060] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.292776 12871 raft_consensus.cc:3055] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:23.293135 12871 raft_consensus.cc:740] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Running, Role: LEADER
I20260709 13:06:23.294167 12871 consensus_queue.cc:260] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.296166 12871 raft_consensus.cc:3060] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:23.301126 12722 raft_consensus.cc:2468] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 2.
I20260709 13:06:23.302611 12765 leader_election.cc:304] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:23.303610 13078 raft_consensus.cc:2804] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:23.304109 13078 raft_consensus.cc:697] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Becoming Leader. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:23.304911 13078 consensus_queue.cc:237] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:23.305716 12871 raft_consensus.cc:2468] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc1fcc33b1c04369ab276792291f3331 in term 2.
I20260709 13:06:23.318521 12617 catalog_manager.cc:5697] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 reported cstate change: term changed from 1 to 2, leader changed from 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195) to fc1fcc33b1c04369ab276792291f3331 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "fc1fcc33b1c04369ab276792291f3331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:23.464407 12871 raft_consensus.cc:1275] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Refusing update from remote peer fc1fcc33b1c04369ab276792291f3331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.465788 13078 consensus_queue.cc:1048] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.478704 12722 raft_consensus.cc:1275] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Refusing update from remote peer fc1fcc33b1c04369ab276792291f3331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.481423 12798 raft_consensus.cc:1275] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Refusing update from remote peer 3a9fb085add94dd99d22ee2b1129932c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.481339 13078 consensus_queue.cc:1048] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.483629 13046 consensus_queue.cc:1048] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.496088 12871 raft_consensus.cc:1275] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Refusing update from remote peer 3a9fb085add94dd99d22ee2b1129932c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.497731 13076 consensus_queue.cc:1048] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.523286 12722 raft_consensus.cc:1275] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Refusing update from remote peer fc1fcc33b1c04369ab276792291f3331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.524686 13021 consensus_queue.cc:1048] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.533634 12871 raft_consensus.cc:1275] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Refusing update from remote peer fc1fcc33b1c04369ab276792291f3331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.535368 13077 consensus_queue.cc:1048] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.548436 12798 raft_consensus.cc:1275] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Refusing update from remote peer 4a2c916532db4ec1b36070c705d7e07f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.549890 13041 consensus_queue.cc:1048] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.551677 12722 raft_consensus.cc:1275] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Refusing update from remote peer fc1fcc33b1c04369ab276792291f3331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.554390 13077 consensus_queue.cc:1048] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.568539 12872 raft_consensus.cc:1275] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Refusing update from remote peer fc1fcc33b1c04369ab276792291f3331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.571245 12722 raft_consensus.cc:1275] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Refusing update from remote peer 4a2c916532db4ec1b36070c705d7e07f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.571435 13077 consensus_queue.cc:1048] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.576169 13041 consensus_queue.cc:1048] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:06:23.631880 12797 raft_consensus.cc:1275] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Refusing update from remote peer 3a9fb085add94dd99d22ee2b1129932c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.633433 13076 consensus_queue.cc:1048] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.642912 12871 raft_consensus.cc:1275] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Refusing update from remote peer 3a9fb085add94dd99d22ee2b1129932c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.644309 13046 consensus_queue.cc:1048] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.687178 12871 raft_consensus.cc:1275] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Refusing update from remote peer 3a9fb085add94dd99d22ee2b1129932c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.688434 12945 consensus_queue.cc:1048] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.699723 12797 raft_consensus.cc:1275] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Refusing update from remote peer 3a9fb085add94dd99d22ee2b1129932c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.701406 13076 consensus_queue.cc:1048] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.712226 12724 raft_consensus.cc:1275] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Refusing update from remote peer fc1fcc33b1c04369ab276792291f3331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.714150 13078 consensus_queue.cc:1048] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:23.727099 12871 raft_consensus.cc:1275] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Refusing update from remote peer fc1fcc33b1c04369ab276792291f3331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:23.728772 13021 consensus_queue.cc:1048] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:25.923942 12618 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43224:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:06:25.944308 12618 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43224:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:06:26.061555   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:26.055003 12690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:36913, user_credentials={real_user=slave}} blocked reactor thread for 44537.2us
I20260709 13:06:26.210587 13150 tablet.cc:1724] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.231417 13145 tablet.cc:1724] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.259223 13155 tablet.cc:1724] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:26.269837 13146 log.cc:927] Time spent T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 13:06:26.277923 13151 log.cc:927] Time spent T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:06:26.281635 13141 log.cc:927] Time spent T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f: Append to log took a long time: real 0.082s	user 0.000s	sys 0.002s
I20260709 13:06:26.253384 13152 tablet.cc:1724] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.347050 13170 tablet.cc:1724] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.304339 13162 tablet.cc:1724] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.337015 13159 tablet.cc:1724] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.326442 13155 tablet.cc:1724] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:26.359719 13157 log.cc:927] Time spent T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331: Append to log took a long time: real 0.129s	user 0.002s	sys 0.000s
I20260709 13:06:26.364394 13145 tablet.cc:1724] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.388962 13171 tablet.cc:1724] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:26.393833 12839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42057, user_credentials={real_user=slave}} blocked reactor thread for 44264.3us
I20260709 13:06:26.399317 13173 tablet.cc:1724] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.336347 13166 tablet.cc:1724] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.402818 13168 tablet.cc:1724] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.414372 13168 tablet.cc:1724] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.416181 13165 tablet.cc:1724] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.422813 12796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17fa856455174351b44f0fa9466fbf4d"
dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
mode: GRACEFUL
new_leader_uuid: "4a2c916532db4ec1b36070c705d7e07f"
 from {username='slave'} at 127.0.0.1:43824
I20260709 13:06:26.423812 12796 raft_consensus.cc:606] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Received request to transfer leadership to TS 4a2c916532db4ec1b36070c705d7e07f
I20260709 13:06:26.435920 13150 tablet.cc:1724] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.438987 13145 tablet.cc:1724] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.442464 13152 tablet.cc:1724] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:06:26.492686 12765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:42057, user_credentials={real_user=slave}} blocked reactor thread for 197887us
I20260709 13:06:26.501616 13162 tablet.cc:1724] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.512044 13159 tablet.cc:1724] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.512856 13173 tablet.cc:1724] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.526180 13166 tablet.cc:1724] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.531777 13155 tablet.cc:1724] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.552922 13145 tablet.cc:1724] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.554584 13171 tablet.cc:1724] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.555912 12618 catalog_manager.cc:5789] TS fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543) does not have the latest schema for tablet 3111979238fe417aa43ba481ab84bfa0 (table soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6]). Expected version 1 got 0
I20260709 13:06:26.535697 13162 tablet.cc:1724] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.563267 12724 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56647efebe9c40b0bc6363de5264c359"
dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
mode: GRACEFUL
new_leader_uuid: "4a2c916532db4ec1b36070c705d7e07f"
 from {username='slave'} at 127.0.0.1:35516
I20260709 13:06:26.564000 12724 raft_consensus.cc:606] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Received request to transfer leadership to TS 4a2c916532db4ec1b36070c705d7e07f
I20260709 13:06:26.565845   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:26.566541   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:26.582754 13159 tablet.cc:1724] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:06:26.599104 13127 raft_consensus.cc:993] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c: : Instructing follower 4a2c916532db4ec1b36070c705d7e07f to start an election
I20260709 13:06:26.599812 13128 raft_consensus.cc:1081] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Signalling peer 4a2c916532db4ec1b36070c705d7e07f to start an election
I20260709 13:06:26.601936 12872 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56647efebe9c40b0bc6363de5264c359"
dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
 from {username='slave'} at 127.0.0.1:38416
I20260709 13:06:26.602566 12872 raft_consensus.cc:493] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:26.603104 12872 raft_consensus.cc:3060] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:26.614157 12872 raft_consensus.cc:515] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } }
I20260709 13:06:26.617249 12724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56647efebe9c40b0bc6363de5264c359" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:26.617944 12724 raft_consensus.cc:3055] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:26.618319 12724 raft_consensus.cc:740] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a9fb085add94dd99d22ee2b1129932c, State: Running, Role: LEADER
I20260709 13:06:26.619685 12724 consensus_queue.cc:260] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } }
I20260709 13:06:26.621157 12724 raft_consensus.cc:3060] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:26.632264 12872 leader_election.cc:290] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 2 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:26.633419 12795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56647efebe9c40b0bc6363de5264c359" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:26.634224 12795 raft_consensus.cc:3060] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:26.645460 12795 raft_consensus.cc:2468] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 2.
I20260709 13:06:26.647495 12724 raft_consensus.cc:2468] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 2.
I20260709 13:06:26.650027 12837 leader_election.cc:304] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a2c916532db4ec1b36070c705d7e07f, fc1fcc33b1c04369ab276792291f3331; no voters: 
I20260709 13:06:26.650967 13112 raft_consensus.cc:993] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331: : Instructing follower 4a2c916532db4ec1b36070c705d7e07f to start an election
I20260709 13:06:26.652506 13169 raft_consensus.cc:2804] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:26.653697 13169 raft_consensus.cc:697] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 2 LEADER]: Becoming Leader. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Running, Role: LEADER
I20260709 13:06:26.653997 12618 catalog_manager.cc:6242] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0: soft_deleted_table [id=6a5305a7a25641b6bc69f2759ecfb3c6] alter complete (version 1)
I20260709 13:06:26.654898 13174 raft_consensus.cc:1081] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Signalling peer 4a2c916532db4ec1b36070c705d7e07f to start an election
I20260709 13:06:26.657124 12872 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17fa856455174351b44f0fa9466fbf4d"
dest_uuid: "4a2c916532db4ec1b36070c705d7e07f"
 from {username='slave'} at 127.0.0.1:38430
I20260709 13:06:26.657765 12872 raft_consensus.cc:493] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:26.658161 12872 raft_consensus.cc:3060] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:26.658551 13169 consensus_queue.cc:237] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } }
I20260709 13:06:26.667219 12872 raft_consensus.cc:515] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:26.668897 12617 catalog_manager.cc:5697] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f reported cstate change: term changed from 1 to 2, leader changed from 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193) to 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195). New cstate: current_term: 2 leader_uuid: "4a2c916532db4ec1b36070c705d7e07f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:26.670605 12724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fa856455174351b44f0fa9466fbf4d" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3a9fb085add94dd99d22ee2b1129932c"
I20260709 13:06:26.670717 12872 leader_election.cc:290] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 3 election: Requested vote from peers 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057), fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543)
I20260709 13:06:26.672468 12724 raft_consensus.cc:3060] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:26.671797 12795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fa856455174351b44f0fa9466fbf4d" candidate_uuid: "4a2c916532db4ec1b36070c705d7e07f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc1fcc33b1c04369ab276792291f3331"
I20260709 13:06:26.674307 12795 raft_consensus.cc:3055] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:26.674630 12795 raft_consensus.cc:740] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc1fcc33b1c04369ab276792291f3331, State: Running, Role: LEADER
I20260709 13:06:26.675627 12795 consensus_queue.cc:260] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:26.676945 12795 raft_consensus.cc:3060] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:26.682642 12724 raft_consensus.cc:2468] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 3.
I20260709 13:06:26.684381 12839 leader_election.cc:304] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9fb085add94dd99d22ee2b1129932c, 4a2c916532db4ec1b36070c705d7e07f; no voters: 
I20260709 13:06:26.684590 12795 raft_consensus.cc:2468] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a2c916532db4ec1b36070c705d7e07f in term 3.
I20260709 13:06:26.685379 13169 raft_consensus.cc:2804] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:26.685906 13169 raft_consensus.cc:697] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 3 LEADER]: Becoming Leader. State: Replica: 4a2c916532db4ec1b36070c705d7e07f, State: Running, Role: LEADER
I20260709 13:06:26.687816 13169 consensus_queue.cc:237] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } }
I20260709 13:06:26.700615 12617 catalog_manager.cc:5697] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f reported cstate change: term changed from 2 to 3, leader changed from fc1fcc33b1c04369ab276792291f3331 (127.0.107.194) to 4a2c916532db4ec1b36070c705d7e07f (127.0.107.195). New cstate: current_term: 3 leader_uuid: "4a2c916532db4ec1b36070c705d7e07f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2c916532db4ec1b36070c705d7e07f" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36913 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:27.089402 12796 raft_consensus.cc:1275] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Refusing update from remote peer 4a2c916532db4ec1b36070c705d7e07f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:27.091035 13169 consensus_queue.cc:1048] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:27.109263 12724 raft_consensus.cc:1275] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Refusing update from remote peer 4a2c916532db4ec1b36070c705d7e07f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:27.111196 13183 consensus_queue.cc:1048] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:27.274296 12796 raft_consensus.cc:1275] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 3 FOLLOWER]: Refusing update from remote peer 4a2c916532db4ec1b36070c705d7e07f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:27.275820 13183 consensus_queue.cc:1048] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fcc33b1c04369ab276792291f3331" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:27.285656 12724 raft_consensus.cc:1275] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 3 FOLLOWER]: Refusing update from remote peer 4a2c916532db4ec1b36070c705d7e07f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:27.288919 13172 consensus_queue.cc:1048] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9fb085add94dd99d22ee2b1129932c" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 42057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:27.684111   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:06:27.711016 12839 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 8 similar messages]
W20260709 13:06:27.720050 12839 consensus_peers.cc:597] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f -> Peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Couldn't send request to peer 3a9fb085add94dd99d22ee2b1129932c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:27.749831   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:27.751588   431 tablet_replica.cc:333] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.753604   431 raft_consensus.cc:2243] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:27.755524   431 raft_consensus.cc:2272] T e25d10382c5343c78b863083e4b4aa31 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:27.757125 12765 consensus_peers.cc:597] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 -> Peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Couldn't send request to peer 3a9fb085add94dd99d22ee2b1129932c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:27.759621   431 tablet_replica.cc:333] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.760615   431 raft_consensus.cc:2243] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:27.761411   431 raft_consensus.cc:2272] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.764580   431 tablet_replica.cc:333] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.765372   431 raft_consensus.cc:2243] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:27.766773   431 raft_consensus.cc:2272] T f4318a4373b54622a1aa813bc53cfb44 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.770455   431 tablet_replica.cc:333] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.771626   431 raft_consensus.cc:2243] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:27.774159   431 raft_consensus.cc:2272] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:27.775538 12765 consensus_peers.cc:597] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 -> Peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Couldn't send request to peer 3a9fb085add94dd99d22ee2b1129932c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:27.779896   431 tablet_replica.cc:333] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.781561   431 raft_consensus.cc:2243] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:27.782666   431 raft_consensus.cc:2272] T 56647efebe9c40b0bc6363de5264c359 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.789392   431 tablet_replica.cc:333] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.790246   431 raft_consensus.cc:2243] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:27.791646   431 raft_consensus.cc:2272] T b7cca9fc63574e26a1b1aac317816794 P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.795310   431 tablet_replica.cc:333] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.796218   431 raft_consensus.cc:2243] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:27.797423   431 raft_consensus.cc:2272] T 17fa856455174351b44f0fa9466fbf4d P 3a9fb085add94dd99d22ee2b1129932c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:27.799763 12765 consensus_peers.cc:597] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 -> Peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Couldn't send request to peer 3a9fb085add94dd99d22ee2b1129932c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:27.802840   431 tablet_replica.cc:333] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.803915   431 raft_consensus.cc:2243] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:27.804652   431 raft_consensus.cc:2272] T f4fe0354684a4ea08b937a39cec88abb P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.808476   431 tablet_replica.cc:333] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.809362   431 raft_consensus.cc:2243] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:06:27.811277 12765 consensus_peers.cc:597] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 -> Peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Couldn't send request to peer 3a9fb085add94dd99d22ee2b1129932c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:27.811952   431 raft_consensus.cc:2272] T 51bbf038592a410da9f6958348a1934b P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.815184   431 tablet_replica.cc:333] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.815886   431 raft_consensus.cc:2243] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:27.816648   431 raft_consensus.cc:2272] T 121cd579cbe1405a81ad6f0a8693538b P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.819623   431 tablet_replica.cc:333] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.820377   431 raft_consensus.cc:2243] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:27.821501   431 raft_consensus.cc:2272] T a98c8d37fff242259cc617625b0f83b0 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.824977   431 tablet_replica.cc:333] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.825627   431 raft_consensus.cc:2243] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:27.826337   431 raft_consensus.cc:2272] T 271e42be0a714018a752ea327df357ef P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.829209   431 tablet_replica.cc:333] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.829862   431 raft_consensus.cc:2243] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:27.830554   431 raft_consensus.cc:2272] T 2fb3192718d94c2e850c178a9ad03e1d P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.833343   431 tablet_replica.cc:333] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.834048   431 raft_consensus.cc:2243] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:27.835563   431 raft_consensus.cc:2272] T 3ee37120e5a4496bbea79d807e5ed39a P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.838603   431 tablet_replica.cc:333] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.839345   431 raft_consensus.cc:2243] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:27.839978   431 raft_consensus.cc:2272] T d38859ad5d584abab1d3d1c93729a5f4 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.842772   431 tablet_replica.cc:333] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.843487   431 raft_consensus.cc:2243] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:27.844168   431 raft_consensus.cc:2272] T 3111979238fe417aa43ba481ab84bfa0 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.846745   431 tablet_replica.cc:333] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.847384   431 raft_consensus.cc:2243] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:27.848130   431 raft_consensus.cc:2272] T 86c4c48405d84950bc27d4972be8a04c P 3a9fb085add94dd99d22ee2b1129932c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.850943   431 tablet_replica.cc:333] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c: stopping tablet replica
I20260709 13:06:27.851485   431 raft_consensus.cc:2243] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:27.852537   431 raft_consensus.cc:2272] T bcf73d1d6c504892aca0e7c5abf13da7 P 3a9fb085add94dd99d22ee2b1129932c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:27.885840 12839 consensus_peers.cc:597] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f -> Peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Couldn't send request to peer 3a9fb085add94dd99d22ee2b1129932c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:27.929520   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:06:27.960211 12839 consensus_peers.cc:597] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f -> Peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Couldn't send request to peer 3a9fb085add94dd99d22ee2b1129932c. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:27.971745 12839 consensus_peers.cc:597] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f -> Peer 3a9fb085add94dd99d22ee2b1129932c (127.0.107.193:42057): Couldn't send request to peer 3a9fb085add94dd99d22ee2b1129932c. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:42057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:27.981349   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:06:28.003149 12837 consensus_peers.cc:597] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f -> Peer fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543): Couldn't send request to peer fc1fcc33b1c04369ab276792291f3331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:28.022166   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:28.023377   431 tablet_replica.cc:333] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.024492   431 raft_consensus.cc:2243] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.025512   431 raft_consensus.cc:2272] T f4318a4373b54622a1aa813bc53cfb44 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.028187   431 tablet_replica.cc:333] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.028861   431 raft_consensus.cc:2243] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:28.029927   431 raft_consensus.cc:2272] T e25d10382c5343c78b863083e4b4aa31 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.032608   431 tablet_replica.cc:333] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.033394   431 raft_consensus.cc:2243] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:28.034433   431 raft_consensus.cc:2272] T 5a8751836b2c4b6a8102c3dfa18d8c6b P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.037616   431 tablet_replica.cc:333] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.038313   431 raft_consensus.cc:2243] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.039000   431 raft_consensus.cc:2272] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.041360   431 tablet_replica.cc:333] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.042040   431 raft_consensus.cc:2243] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.042629   431 raft_consensus.cc:2272] T 56647efebe9c40b0bc6363de5264c359 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.046271   431 tablet_replica.cc:333] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.046993   431 raft_consensus.cc:2243] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.047662   431 raft_consensus.cc:2272] T b7cca9fc63574e26a1b1aac317816794 P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.050134   431 tablet_replica.cc:333] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.050769   431 raft_consensus.cc:2243] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:28.051780   431 raft_consensus.cc:2272] T 271e42be0a714018a752ea327df357ef P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.054216   431 tablet_replica.cc:333] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.054836   431 raft_consensus.cc:2243] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:28.055912   431 raft_consensus.cc:2272] T 2fb3192718d94c2e850c178a9ad03e1d P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.058418   431 tablet_replica.cc:333] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.059051   431 raft_consensus.cc:2243] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.059661   431 raft_consensus.cc:2272] T 3ee37120e5a4496bbea79d807e5ed39a P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.061992   431 tablet_replica.cc:333] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.062549   431 raft_consensus.cc:2243] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:28.063565   431 raft_consensus.cc:2272] T d38859ad5d584abab1d3d1c93729a5f4 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.065763   431 tablet_replica.cc:333] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.066337   431 raft_consensus.cc:2243] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.066838   431 raft_consensus.cc:2272] T 17fa856455174351b44f0fa9466fbf4d P fc1fcc33b1c04369ab276792291f3331 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.071312   431 tablet_replica.cc:333] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.071966   431 raft_consensus.cc:2243] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.072475   431 raft_consensus.cc:2272] T f4fe0354684a4ea08b937a39cec88abb P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.075273   431 tablet_replica.cc:333] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.075878   431 raft_consensus.cc:2243] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.076552   431 raft_consensus.cc:2272] T 51bbf038592a410da9f6958348a1934b P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:28.078100 12837 consensus_peers.cc:597] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f -> Peer fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543): Couldn't send request to peer fc1fcc33b1c04369ab276792291f3331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:28.079257   431 tablet_replica.cc:333] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.079906   431 raft_consensus.cc:2243] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.080845   431 raft_consensus.cc:2272] T 121cd579cbe1405a81ad6f0a8693538b P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.084496   431 tablet_replica.cc:333] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.085131   431 raft_consensus.cc:2243] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.085750   431 raft_consensus.cc:2272] T a98c8d37fff242259cc617625b0f83b0 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.088169   431 tablet_replica.cc:333] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.088730   431 raft_consensus.cc:2243] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:28.089736   431 raft_consensus.cc:2272] T 3111979238fe417aa43ba481ab84bfa0 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.092260   431 tablet_replica.cc:333] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.092833   431 raft_consensus.cc:2243] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:28.093772   431 raft_consensus.cc:2272] T 86c4c48405d84950bc27d4972be8a04c P fc1fcc33b1c04369ab276792291f3331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.097808   431 tablet_replica.cc:333] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331: stopping tablet replica
I20260709 13:06:28.098866   431 raft_consensus.cc:2243] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.099560   431 raft_consensus.cc:2272] T bcf73d1d6c504892aca0e7c5abf13da7 P fc1fcc33b1c04369ab276792291f3331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.146441   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:06:28.162094 12837 consensus_peers.cc:597] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f -> Peer fc1fcc33b1c04369ab276792291f3331 (127.0.107.194:36543): Couldn't send request to peer fc1fcc33b1c04369ab276792291f3331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:36543: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:28.190232   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:06:28.225638   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:28.226408   431 tablet_replica.cc:333] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.227295   431 raft_consensus.cc:2243] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.228092   431 raft_consensus.cc:2272] T f4318a4373b54622a1aa813bc53cfb44 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.230542   431 tablet_replica.cc:333] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.231249   431 raft_consensus.cc:2243] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.231839   431 raft_consensus.cc:2272] T e25d10382c5343c78b863083e4b4aa31 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.234371   431 tablet_replica.cc:333] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.235086   431 raft_consensus.cc:2243] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.235651   431 raft_consensus.cc:2272] T 5a8751836b2c4b6a8102c3dfa18d8c6b P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.237960   431 tablet_replica.cc:333] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.238665   431 raft_consensus.cc:2243] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.239300   431 raft_consensus.cc:2272] T 25c0c0ccabef4e72bf35ad3c1dc83f17 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.241604   431 tablet_replica.cc:333] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.242185   431 raft_consensus.cc:2243] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:28.243124   431 raft_consensus.cc:2272] T 56647efebe9c40b0bc6363de5264c359 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.245338   431 tablet_replica.cc:333] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.245908   431 raft_consensus.cc:2243] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.246383   431 raft_consensus.cc:2272] T b7cca9fc63574e26a1b1aac317816794 P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.249045   431 tablet_replica.cc:333] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.249606   431 raft_consensus.cc:2243] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.250097   431 raft_consensus.cc:2272] T 271e42be0a714018a752ea327df357ef P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.252249   431 tablet_replica.cc:333] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.252805   431 raft_consensus.cc:2243] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.253262   431 raft_consensus.cc:2272] T 2fb3192718d94c2e850c178a9ad03e1d P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.255443   431 tablet_replica.cc:333] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.256165   431 raft_consensus.cc:2243] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:28.257030   431 raft_consensus.cc:2272] T 17fa856455174351b44f0fa9466fbf4d P 4a2c916532db4ec1b36070c705d7e07f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.260558   431 tablet_replica.cc:333] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.261121   431 raft_consensus.cc:2243] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:28.261971   431 raft_consensus.cc:2272] T f4fe0354684a4ea08b937a39cec88abb P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.264191   431 tablet_replica.cc:333] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.264680   431 raft_consensus.cc:2243] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.265200   431 raft_consensus.cc:2272] T 51bbf038592a410da9f6958348a1934b P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.267360   431 tablet_replica.cc:333] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.267828   431 raft_consensus.cc:2243] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.268365   431 raft_consensus.cc:2272] T 3ee37120e5a4496bbea79d807e5ed39a P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.270460   431 tablet_replica.cc:333] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.271032   431 raft_consensus.cc:2243] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.271493   431 raft_consensus.cc:2272] T d38859ad5d584abab1d3d1c93729a5f4 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.273603   431 tablet_replica.cc:333] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.274104   431 raft_consensus.cc:2243] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:28.275017   431 raft_consensus.cc:2272] T 121cd579cbe1405a81ad6f0a8693538b P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.277287   431 tablet_replica.cc:333] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.277793   431 raft_consensus.cc:2243] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.278249   431 raft_consensus.cc:2272] T a98c8d37fff242259cc617625b0f83b0 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.280572   431 tablet_replica.cc:333] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.281088   431 raft_consensus.cc:2243] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.281615   431 raft_consensus.cc:2272] T 3111979238fe417aa43ba481ab84bfa0 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.283905   431 tablet_replica.cc:333] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.284407   431 raft_consensus.cc:2243] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.284803   431 raft_consensus.cc:2272] T 86c4c48405d84950bc27d4972be8a04c P 4a2c916532db4ec1b36070c705d7e07f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.286989   431 tablet_replica.cc:333] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f: stopping tablet replica
I20260709 13:06:28.287417   431 raft_consensus.cc:2243] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:28.287796   431 raft_consensus.cc:2272] T bcf73d1d6c504892aca0e7c5abf13da7 P 4a2c916532db4ec1b36070c705d7e07f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.335697   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:06:28.375016   431 master.cc:562] Master@127.0.107.254:35909 shutting down...
I20260709 13:06:28.400600   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:28.401283   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:28.401650   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P ec8e2d1e62004163b57ec94997e000a0: stopping tablet replica
I20260709 13:06:28.425745   431 master.cc:584] Master@127.0.107.254:35909 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (12430 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (25188 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (373664 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:06:28.487746   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 422 messages since previous log ~587 seconds ago
I20260709 13:06:28.487977   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 422 messages since previous log ~587 seconds ago
I20260709 13:06:28.488175   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 421 messages since previous log ~587 seconds ago
I20260709 13:06:28.488473   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:06:28.488649   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:06:28.488890   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 1 messages since previous log ~122 seconds ago
I20260709 13:06:28.489112   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/master/catalog_manager.cc:4755: suppressed but not reported on 1 messages since previous log ~79 seconds ago
I20260709 13:06:28.489391   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 11 messages since previous log ~0 seconds ago
I20260709 13:06:28.489711   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 26 messages since previous log ~6 seconds ago
